 So, you do testing for all your projects, right? I usually create a new issue, assign it to you, and then magically overnight my tests get written. It's the most wonderful thing. Yeah, for you. So, we did a previous episode on unit testing with things like Marker, and we looked at a whole other host of other ones. What do you think of when someone says unit testing to you? Oh, there's like in-browser testing, CI testing, there's like usability testing. Like there's an entire minefield. Yeah, so this is kind of I think the biggest issue that I have a lot of this stuff. Like when I first got into unit testing, it was like Marker and it was in the terminal and you get used to this thing of like this test pass, this one failed. And that was all cool, but it's JavaScript, it's not in the browser. You don't have the browser API, so it's a bit of me that got confused. It was like, well, okay, do I have to edit my code to be separate so it can run in Node even though it's going to run in a browser or what. That's kind of been cleared up because obviously with Marker you can run in the browser, which makes a bit more sense if you're building a browser library and you need those APIs. But there was always this weird tension because a lot of the work I've been doing super recently, I actually got to a point where Node, I was mocking out a lot of the browser APIs, largely to make the test super fast and super reliable because basically nothing was real. Everything was fake in a facade. So you don't have access to Windows, you don't have access to any of those APIs, do you? Yes, you end up kind of doing one or two things. Either you run it inside the browser with the Marker stuff and it will actually use those APIs, which for the most part are fine, it's good because if there's any browser differences, you get them. Or you mock all of them out, which means it's completely fake. It's normally super fast because it's not actually doing any work. You normally go, in this case, pass and this takes through an error and there. But it still kind of always confused me, like, okay, so I have these two things, but it doesn't feel like this is good on an ongoing basis because if I don't run the tests, nothing happens. So that's when we get into the Selenium world and it gets super interesting slash weird depending on how you do it. So for anyone who doesn't know, Selenium WebDriver is a way of launching a browser and controlling it. It's the most generic way of putting it. They have a ton of different versions, but this is the JavaScript Node version I've come to know and love. Let's look at super basic bare bones Selenium script. And this is it. So this is a Node version. Selenium has a ton of different libraries, which also doesn't help because documentation is so much stuff. But in the most bare bones thing, you create what's known as a WebDriver builder. At that point, you can basically say, okay, with the builder, I want it to be a Chrome browser. Or I want to do this and then you can bolt on additional things. You say, build me a driver. And from that point, that is basically, you can kind of think of it as like an instance of a browser. So that's what we got here. At that point, you can just start using it. So if we call get, in this case, I'm going to launch my Mocha test because we want to automate all of that stuff. We don't want to worry about it. And then we are going to say, wait until something happens. In this case, we're going to say, wait until Mocha results becomes a thing on the actual thing inside the window. And then get the Mocha results and do something useful for them. So this is like a really basic kind of Selenium thing, but it's already super helpful. So if you're going to run this, I'm just going to run it with Node, Test Selenium. You're just going to nod along when I'm just talking to myself. I'm just going to nod along. I assume you know what you're doing. Yeah. That's the safest way of doing it. So all it's doing, and you can kind of see it, is Chrome pops off and then it closes. And you briefly see the Mocha test running. This is super basic, but it's super nice because it means that you could put this, like because it's just a Node script, you can put it in MPM run or you could add it to Travis. You can do whatever you want. It's super nice and clean. The problem I always had with this approach is it gets super manual really quick when you want like Chrome and Firefox. So I created this thing called Selenium Assistant. And its entire thing was like, just tell me what browsers are available and then just give me all of those in a WebDriver thingy and then run it. Yeah. So it's kind of weird because you get available browsers and it can give you like stable, beta and unstable. We've done so much stuff with service workers and like brand new APIs that you wanted to test on all those ones because you knew breaking changes were coming in. Oh yeah. It's almost impossible to manually stay on top of testing that stuff. Yeah. So I built this thing and that's all it does is just simplify all those things. And in this case, I've kind of merged the two. So if we do it super raw with just Selenium Assistant, what it's going to do is we're going to get all the available browsers, open them up, go through the same steps of running Chrome and then return it. And that's super cool because you get into this super weird world where suddenly browsers open and close, open and close, open and close. And then you see it go into Firefox and then it's open and close, open and close. There's still one more big issue. It's almost as if you enjoy chaos. Well, it's great when it works. It's getting to that point. You have this mocker environment where everything builds, runs and it either passes or fails. Selenium just gives you the piece that opens up the browser. It doesn't actually give you the pass, fail. Like Selenium can open, do all those things. Things can go horribly wrong and it just goes, well, I don't care. And the worst thing with that is if you have a Selenium test and something goes wrong, Selenium doesn't quit. It doesn't close that browser. So you end up with a desktop full of like 20 different browsers opened simultaneously. The way around that is you kind of end up mashing mocker and Selenium together. And then you get super trippy inception where you've got mocker running your Selenium test, which is running mocker in the browser to then report back to your mocker Selenium thing. Which is what I'm doing here because you know what? That's true chaos and that's what you need. And it's kind of interesting. So you get the available browsers and you say add mocker tests for each individual browser. So we got for each add mocker tests. Inside of this, we're then saying, right, describe a new test suite. And then before each one, get the Selenium driver. So that's doing the builder stuff for me. And then after each, I just want to make sure I call quit on that browser. Because without that, like I say, especially when you start working on this, if you don't exit at the end, it's going to suck. So mocker tests, Selenium mocker system. And this time it's going to open the browser, listen for the results and come back, whether it works or not. But I've also added an additional step to this one where because we're now in mocker, we can do individual tests. I have that bunch of tests. It's like open the browser mocker, run through all the tests, the unit tests, and then I've added one on the end, which is like, okay, load another page, click on the about link, check that it actually went to the about page. And this is kind of the interesting thing with mockers. You start getting into more, which I think you'd class as an integration test. It's like, make sure the behavior is the actual end thing that I'm expecting. And I call them integration tests or kind of label them differently to kind of highlight the fact they're going to be long. Because I think that's part of the point is unit testing is going to be super fast, super reliable, which is why I'm kind of like, you could mock it out and it makes sense. Integration tests are the big, the long, scary, can take a while kind of things. And there's a ton of ways you can skin both of these cats. But I think it makes a lot of sense. So the super basic integration test, in this case, my website, I'm looking for a particular link, which I'm like executing in the page, clicking it, and then at the very end I'm just saying, okay, wait until the document title is actually what I expect it to be. How well does all this stuff play with ServiceWorker? So that's where I ended up getting into Selenium stuff because I was writing a marker test in the browser and that was great because it meant a really repetitive way of running tests against ServiceWorker APIs. And it was the only way of doing it in the same way, especially when you're learning because there's lots of edge cases. It was the then running them in each individual browser and repeatedly doing it, as well as then running it in the CI. You need all of that in a node script, which is why I landed on Mocha afterwards because you kick it off on the node side and then it's just JavaScript. And you're not testing any node stuff. You're only using Selenium to launch the browser, do certain things. And plus having that step where you can then launch it, perform certain tasks on a demo page. Right. And then check what the response is from the browser is insanely helpful, especially for figuring out like any possible browser differences or bugs in each one, of which there was many back in the day. So that's where I'm at. And I think I'm now starting to try and delineate between unit testing and integration tests because with Selenium, you are instantly in flaky territory. Like, WebDriver just does random stuff because of roles and funsies. Define random stuff. It will just... So the worst thing with a lot of this stuff is you will randomly get one browser updating and it will not work with its current version of its WebDriver equivalent. Your CI kind of goes out the window at that point. But it will often come up with a fix sooner or later. And it's very rare that that will land on stable releases. But there are just times where, let's say you're using Express or some other node modules start a local server. If for whatever reason the browser talking to Express causes an issue and something breaks, it breaks. And that may be a once-in-a-lifetime opportunity on the CI that you will never see again. So I'm finding I'm adding retries a lot for Selenium WebDriver tests. And generally that weeds out a lot of issues to the point where you're like, this is actually a bug on my code. What's the typical number of retries? I've been landing on like three. Okay. And that's just largely because nine times out of ten it's because I'm using like... I'm even going so far with integration tests of like testing against real-life networks. Like, you think of push. You need the browser to talk to like, whatever push servers they're going to talk to and then get a network response. And if that, like, if for whatever reason the CI's internet connection is broken that means my test is going to fail. It's going to get so far and then die. I think you've been tackling some of the sort of bleeding edge around unit testing dealing with service worker dealing with push and all of those other newer platform APIs. I don't think it's bleeding edge testing. I think it's just I've been testing a lot of bleeding edge which is why I'm reaching a lot of these weird situations and it's also the reason why I'm finding it so useful. Because the main thing is if I build these things I also don't want to babysit them. Because you just lumber me with an issue that I then don't want to have to keep on looking after. I just want you to deal with it. That's not going to happen. Yeah, I know. But it's interesting, I think everyone should be looking at this stuff and playing around with it. I definitely, like the next step for me is looking at like services like browser stack and Source Labs which I think now that I understand Selenium I'm at that point where I can appreciate what Source Labs does for me. But I think that's the interesting thing is I feel like everyone who does this starts off raw Selenium goes through the pain, realizes there's abstractions that exist and they want to use. And then they get to this point where they can do like follow up totally tooling tips episode on that topic. Boom.