 Welcome, everyone, to gaining confidence with Cypress Tests. Here's the part where I tell you I am definitely going to post the slides on my site tonight, and it's never worked out for me very well either. You can head to robrich.org right now, click on Presentations right here at the top, and here's gaining confidence with Cypress Tests. The code and the slides are online right now. Here's the code up on GitHub. While we're here on robrich.org, let's click on About Me, and we'll see some of the things that I've done recently. I'm a Microsoft MVP, a Docker captain, a friend of Redgate, and a Cyro developer advocate. AZ GiveCamp is really fun. AZ GiveCamp brings volunteer developers together with charities to build free software. We start Friday after work, Sunday afternoon, we deliver completed software to the charities. Sleep is optional caffeine provided. If you're in Phoenix, come join us for the next AZ GiveCamp, or if you'd like a GiveCamp here in Boston or wherever you're connecting from. Hit me up on email or on Twitter, and let's get a GiveCamp in your neighborhood too. I do a lot with testing and with JavaScript. I worked on the Gulp team in version two and version three, and one of the things I'm particularly proud of, I replied to a .NET Rocks podcast episode. They read my comments on the air, and they sent me a mug, woo-hoo! So let's dig in to Cypress testing. Now, I'm not quite sure if this demo is gonna work, so let me see if my site is gonna work well today. Let's fire up this suite and see if this site will behave. It's looking good so far. We got our TypeScript build. Yep, it looks like the site is doing okay. So this'll be a fun talk. Yeah, the site is working. Great. So Cypress. Cypress is browser-based functional tests or said differently, end-to-end tests. End-to-end tests, pardon me. It's an electron app and a browser plugin. It uses Mocha and Chai, and so if you've tested in JavaScript before, this may feel right at home. Like many good stories, we started in the middle. Let's start back at the beginning and talk about web testing. So when we look about web testing, there's lots of different tests that we may want to do. We may want to validate that this little piece of functionality works. We may want to test that this component will render correctly. We may want to test that this works in this particular browser. We may want to test backend APIs. As we look at testing each of these things, we'll use different tools to be able to accomplish these tasks. Now, as we're testing, we want some of each of these tests. So we'll probably use multiple tools for testing. As you grab these slides from robrich.org, you can click on each of these blue links to get to the details for that particular project. So first up, if I'm doing a unit test, now unit testing a service, I don't mean a web service or a component, but rather a small piece of business functionality. I can write a unit test with that with just regular Mocha and Chai or Jest. When I get those unit tests and I want to validate them in a particular browser, then that's where Karma is really helpful. Karma allows me to launch these tests inside of a browser and validate that they'll work in each environment. Now that we have HTML5, browser compatibility is less of a concern, but Karma is a great tool there. If I want to validate my components, I might choose to fire them up inside TestUtils. Now, there's a TestUtils instance for Angular, React, and View and others. So I can be able to mount my web components and either choose to render out my child components or to stub them with Fakes. If I want to test my API, I don't need a browser at all. I can just make requests to that API. Is it a REST API, GraphQL, or GRPC API? I can just make requests to it, validate the status code and the response, and that's how I can prove my tests. Super test is great for that. As we look at end-to-end tests, we have lots of choices. Cypress is one of them, but we also have Selenium, Test Cafe, and Playwright. Let's double click into end-to-end tests and take a look at each of these. Selenium is kind of the one that invented this criteria, this mechanism of end-to-end testing, and it's still the beefiest in this space. WebDriver has become more or less a browser standard, so with WebDriver technology inside of Selenium, you can test most any browser. Now, it is at arm's length type of test, and so really all it's doing is moving the mouse and clicking buttons. It doesn't really have very deep integration into the system, so you'll probably add a whole lot of weights in this system. If the DOM element isn't there, maybe the service is slow, then we'll wait a little bit more. If ever you have flaky tests, you're probably increasing that timeout, making the tests slower, so Selenium is known for slow and brittle tests. Next up, Cypress. Now, the beauty of Cypress is that browser plug-in so we can look more deeply into the content in the browser. Rather than waiting for a certain period of time, if an event happens, we can trigger right away. That's perfect. Now, Cypress being a browser plug-in requires that it have access to that particular browser. So right now Cypress works with Chrome and Chromium-based Edge and Firefox, but it doesn't work with Safari. So that's mostly a Safari thing. Cypress has a really elegant mechanism because it works great with Mocha and Chai, but I find its query syntax a little bit weird. It uses jQuery-like syntax, and so with jQuery you have a dot then, but it isn't a promise-based thing where you could await it. Cypress's dot thens are a little weird. Next up, Test Cafe. If you're using DevExpress for other things, you may feel right at home here. Now, the cool part is as you're selecting content from the browser, you can use standard CSS mechanisms, but it does spend a lot of time marshaling that browser content back and forth between the test. So you may write a method that it will inject in the page to be able to get at state, where with Cypress, you didn't need to do any of this marshaling process. Next up, Playwright. Now Playwright is built by the people who built Puppeteer, and if you've ever done any browser automation with Puppeteer, you'll feel right at home. Now it does feel like the assertion syntax is a little bit of a bolt-on here, but the cool part is that this project has lots of different languages. So if you want to build your tests in C-sharp or Python or Go, you definitely can, where with Cypress, you're writing in JavaScript or TypeScript. Again, you're marshaling content between the page and the browser to make this happen, and that's the nature of Puppeteer as well. So we've taken a look at various tools and we'll probably use many of these. I'll probably use Mocha with Chai or Jest for my unit tests. I'll want to run those unit tests in various browsers, so I'll also use Karma. I'll also want to test my API, so I'll use Supertest. I'll want to validate that my web components render correctly, so I'll use TestUtils. And then finally, I'll use end-to-end tests that validate the user experiences. So in this case, let's use Cypress. Getting started with Cypress, I'll just say npm install Cypress and then npxCypress open. Now, once I've done that, it will actually scaffold out a bunch of automated tests in their kitchen sync environment that allows me to see the various syntax of Cypress. So I've already got Cypress pulled up here and we can see all of those example tests right here. That's great. We could run each one by clicking on it and that will run only those tests. I'm gonna close this example folder. So, or we could come in here and run all the tests at once. Now, this is just the IDE. There's also a command line interface as well. I can choose what browser I want to run them in. Right now, I'm running them in Chrome, but I also have Firefox and Edge installed on my machine, so it discovered those and let me pick from those as well. It's telling me that I have a Chrome-specific setting in here. It also has Electron here. Now, this is a Chromium-based webkit browser that allows me to run the test just with the Cypress install if I didn't have any other browsers on my machine. Now, in this case, I'll run them in Chrome and I'll pop open this to do MVC test and let's take a look at how this test runs. Now, it does start a new instance of the browser and it doesn't use my current profile. So you'll notice that here's my regular Chrome browser and here's the browser that it's using to run the test. That means all of my history, my plugins, my user selections don't bleed into the tests that I'm running. Now, with each test, it's gonna go run the steps of that test, but you'll notice that it's running it inside of a browser and kind of iframing this side of the site into place. Now, this is just running content in the browser so I can switch windows or I can run multiple tests simultaneously. Looks like all those tests have passed. I can take a look at each step in the test and it will show me the content at that step. So it was trying to find this new to-do box and we can see this red dot showing where it chose to click in that box. And it's also doing both the before and after. It typed this content and so after we had some to-do. Now, this is a browser so I'm gonna open the F12 developer tools, flip over the console and we can see at each stage, it will actually show me the console output at that stage together with a lot of context inside of Cyprus. That's really powerful. Now, the other added benefit is if I flip over to the sources view, I can see my tests and I can set a break point. If I chose to rerun my tests, then I would hit that break point and I can just step through my code like I would any normal debugger. That's cool. Here in the IDE, we also have a mechanism. I can use this to go find things and it will help me identify the selector that I need. We'll come back to building more effective selectors. I can choose to rerun my tests and it'll also show me the details about the passes and the fails and how long that test took. So that's really cool. It's being able to validate my site and running all of the tasks that I chose to do here. Now, in this case, I chose to have a browser that's 1,000 pixels wide, but we could definitely set the viewport to mobile layout or any browser configuration that we needed. Okay, so that was a cool introduction to the Cypress IDE. We can also use the command line to run tests and we'll see that in a little bit. So let's take a look at the code associated with this project. The code is up on GitHub and so you can take a look at this project, clone it, fork it and learn more about the content in this project. Our first stop in this project is the Cypress.json file. Now, this was generated automatically when I said Cypress install and it identifies a few things. I have modified it a bit. The first thing it identifies is the plugins file. Now, in this case, my project is TypeScript, so it's an index.ts, let's go there and here is my integration file, my index file. Now, here in this index file, it specifies the location of all of the other Cypress resources. Here's my fixtures folder right here. Here's my integrations folder. That's where all my tests will be. My support file will come back to that one. That's really fun. And then the output. I chose to put both the screenshots and the videos in the results folder so that I could specifically add that to my .getignore file and not accidentally check in test results. Perfect. Now, this is a TypeScript project. By default, Cypress will scaffold out a JavaScript project, but I flipped it over to TypeScript. Here in my main project, I have the TypeScript configuration that I would need for my project and there's nothing in here about testing. It's just my standard TS config. But here in my Cypress folder, I also have a TS config that explains how to create a new root project so that my tests can run in TypeScript as well. Now, I'm gonna say extends that other TypeScript config file so I don't need to reiterate all the properties that won't change and then I'll add the Cypress specific details into this TS config. Now that I've got that TypeScript root, now we can run some tests. Now, I've created inside my package.json some steps to be able to run interesting things. So Cypress open, opens the IDE and Cypress run is that headless mechanism. I can also specify which browser I wanna run. In this case, I'm running headless Chrome and headless Firefox and headless Edge. And then for NPM test, I just chose to string all of those together. So it'll run all of my tests in all three browsers. And if all of my tests pass in all three browsers, then it'll move forward and continue with my build. Great. So we've gotten a feel for how TypeScript kind of initializes itself. Let's take a look at the code associated with this project. Let's start with to do NBC. Now we looked at, nope, here. Let's run these tests again. This to do NBC is based on this to do NBC project. Now this project is a smidge dated, but it's a great way to be able to compare frameworks. They provide a standard HTML and CSS file and invite various vendors to build this to do app in their framework. So we can compare the results in various places. So in this case, I've built some tests around that external site. And let's take a look at the code for those tests. Here in VS code, we could flip between various tests. I'll run the AngularJS version right now. Now I'm gonna go grab the site name just that I can display it here in my described block. That'll be cool. And then before each test, I'm gonna go visit that site URL. Now I could definitely do this at the top of each test, but I'm gonna do it in a before each so that I don't need to worry about it. By the time I'm done with this step, I know that the page is loaded. Notice that we're not awaiting it. By the time this command finishes, the browser is ready to go. Next, let's do our first hello world test. Should visit the page. Now I'm grabbing this URL and I'm validating that that's the page that we landed on. We didn't get 301 or 302 to a different page. As I started in this test, they actually flipped from HTTP to HTTPS. And so this test failed to help me understand that I wasn't going to the right page. That was perfect. So let's level up again and start interacting with the page. Sci.get. Sci.get allows us to select content in the browser. So we're gonna give it this CSS selector or this jQuery selector. In this case, I'm gonna go look in the to-do list for any LI items. And I'm gonna validate that there aren't any. I haven't created any, so that makes sense. With an empty list, we shouldn't have any to-do items kicking around. Next up, let's start to interact with the browser types and content. So I wanna create a new to-do. Here's my right Cypress test content and that's the to-do that I wanna create. We can see this range act assert pattern that allows us to set up the conditions associated with our test, run the content and then finally assert the results. Now technically, pardon me, technically this visit is part of the act stage but we've done it previously. Okay, so I'm gonna go find that new to-do box. There's my jQuery selector again and type in it this new to-do text. And then I'll push enter. Notice that this one has a dollar sign before it and this one does not. We can use this curly brace and type in various things to type various function keys or shift alt control. We can get at any kind of weird keys on the keyboard with this curly brace syntax. So we'll type our new to-do, we'll push the enter key and then as soon as we finish that, it should create a new element in our list and clear out this box. So let's go look for that list and go see if the ally items contain this new to-do. If we have a new to-do, then we'll be able to find it this way and now our test should pass. Let's also go look at the new to-do box and validate that its value is blank. Okay, so we've set up this test and we're able to go create a to-do. Now let's mark a to-do as completed. We'll go type in this new to-do. I'm gonna start by typing in a relevant to-do so that I make sure that the one that I'm completing is the one that I want and not maybe the first one in the list or the last one. So I'm gonna create an irrelevant to-do. I'll go type my new to-do and push enter and then go look in that view for the to-do that I'm looking for. This is kind of like jQuery's find syntax where I pass in a query and then a filter selector. That's a little weird, but I get it. So once I found that thing, then let me go find the toggle button. That's how I choose to complete it and I will click it. Now once I've clicked that complete button, now looking through this list, we should have that one element marked with the completed CSS class. Now in this case, completed means strike through, but I'm not validating that my CSS is specified correctly, only that my JavaScript is functioning the way it should. Okay, so we've marked a to-do as completed. Now let's delete a to-do. I'm gonna go create an irrelevant to-do. I'm gonna go create my new to-do and then I will go look for that to-do that I want to delete and go find the destroy button. Now in this case, the destroy button only shows up when we do a mouseover and then we can click it to delete it. And so because I'm not running the CSS events inside this test, I'm only running the JavaScript events, then it never is visible. Well, I can still click it, I just have to say force is true and when I say force is true, it will click it even though it's not visible. In this case, the button wasn't visible, so I said force is true and I marked that, I deleted that one to-do. So now my to-do list should only have one item in it and it should contain that irrelevant to-do, not the to-do that I marked to delete. Perfect, let's level up again. Now we spent a lot of time typing new to-dos and pushing enter and some more to-dos and pushing enter, that's getting kind of redundant. Instead, let's create an action. Let's say cy.to-do add and add some content there. Now I just made that up, but that centralizes that business logic associated with that task. So I'm gonna come here into my support file and take a look at the index file and this imports those commands. So here's that command and I built a command called to-do add. To-do add will take in a text, type it in that box and while I'm here, I may as well just validate that once I pushed enter that box is now empty. Perfect. Now in JavaScript, that's all I would need to do, but here in TypeScript I need to tell it that this is a valid command. Okay, let's head off to the Cypress docs and we'll take a look at the details of associating commands with TypeScript. Not only do I build this function that identifies how it works, but I also need a TypeScript declaration file explaining how this thing works and that will help TypeScript validate the types associated with my command. So back in VS Code, I have this TypeScript declaration file. Nope, this TypeScript declaration file that will show me the interfaces that I've built. So here's my to-do add function, takes in some text and returns that element. So now I'm able to make this test and kind of describe what's going on in my test. I also have a to-do complete command that specifically goes and finds that toggle button and clicks it. What's really elegant is I'm starting to describe my business logic. My test is much more terse and those steps that I need to do are abstracted away into one place. We can build up our test this way by creating commands for each of the tasks that we want to do and then just really quickly describe the business logic here in our test. Now that makes our tests a lot more legible. We're describing the intent of the user, not necessarily describing all of the steps it takes to get there. The steps are often the commands. So I create three to-dos, I complete one of them and then I go click the active button. So now I should only have two left in my list. Now I chose to create three because, well, was I showing the active ones or the inactive ones? I wanted to validate that I was showing the active ones. Okay, so let's use that same mechanism. We've created three to-dos, we'll mark one of them complete and now let's go filter by the completed button. Once I've got that completed button, now I should only have one item in my list. Now this mechanism of selecting things, dot filter slash a completed is a little bit awkward. We've built this CSS selector that kind of navigates through the DOM, but if ever I change the HTML associated with my website, that selector might break. Instead we should upgrade to going by specific CSS selectors on our content or even better based on IDs associated with that content. Even better still, we could create a data dash CY element and that very specifically says that this attribute is focused on testing. Then in our test, we can validate, we can just grab that based on that data dash CY element and we know we're getting exactly the element that we need. That's perfect. Now in this case, it's not a site that I can control so I'm just using the CSS selectors. Once we've quit completed, now let's go validate that we only have one thing in the list, only our completed item. To fill out our tests, let's go clear the completed list. So we'll create a new one, we'll complete it and then we'll go click the clear completed button and now we should have nothing in our list. And we can see based on these tests that each of those passes and we're able to validate that our site accomplishes each of these tasks. So for example, to do delete, we're gonna type a new to do and then we will go market as done and then we validate that we only have one left. Perfect. So now we were able to get started with Cypress in a really elegant way. Let's level up and talk about network requests. So I'm going to start up a new suite of tests and let's talk about hacker new PWA. Now much like to do NBC, hacker news PWA is kind of that spiritual successor where it reads hacker news and shows it in various applications. So here's the react hacker news PWA and here's the another react hacker news PWA and a pre-act one, there's a view one, here's a spelt one. And so we can run similar tests. In this case, we're running them inside of Angular. So let's take a look at that test. As we fire up the test, similarly we could switch between various sites. I chose the Angular one in this case and I'll just grab the name for the described block. Now we'll go visit that test. I'm glad my first hello world is still a go validate that's the page that I landed on because if ever we move pages within our website it'd be nice if our tests would fail to validate that we aren't testing the page that we expect. Next up, we should have a specific top story. I want to validate the rendering of my component. Now in this case, the component could be well, whatever that news story is. I don't know what hacker news's top story is. And so as we're building this code we should instead put in some fake data. So I'm gonna say cy.intercept, we could put in a string but in this case I chose a regular expression and I'm gonna say add the access control allow origin header because it is cross domain and then replace it with this fixture. Here in the fixtures folder, here's my hacker news JSON and I have specific stories that allow me to make some assertions. So this is the first story. I know that one will be the first one in the list and I can come back here and I can say, okay go grab the first story and validate that it should contain this specific title. If I want to validate the rendering of my components I can use fake data, these fixtures to be able to specify the specific details and then validate that it renders correctly. That would be really easy for me to lie to myself and pretend that that's how the API works. So we have a second test that actually does really go hit the real API. Now I can assume that there will probably be 30 or more stories in hacker news and this API pages at 30. So let me just go grab the real things and validate that I have 30 in that list. Now I don't know what the stories are. I can't validate that it goes in the right place but that's why I have these other tests that mock out the details when I need to test specific things. Now if this API took a really long time I could name it in this case get stories and then specifically wait for that task. In this case it ran fast enough so I didn't need to. That's perfect. So let's look at this test and we can see that we visited the page correctly. We looked for a specific page and a specific result and noticed that these are the fake results and then our real test that hits the real API and validates that we have the content in place. So it looks like all of those tests passed this one actually canceled the real request in favor of the fixture and so we got to see some really elegant mechanisms for getting started and leveling up through various places inside of Cyprus. Let's take a look at some best practices. Now we talked about how we could build CSS selectors and definitely find content in the page anywhere we needed to but it's better to level up to a specific class or even better an ID so that we know that we're hitting exactly the thing that we want. If we rearrange our HTML we want our test to still work. Even better still create a data-cy attribute on that element and then go select that specific data-cy attribute. In this case we were testing a public website so we weren't able to do that but that's a really elegant way of not only capturing exactly the element that we need but also documenting in our project when testing needs a particular thing. So if I'm modifying some HTML and I see a data-cy element I know that I'll probably also need to modify a test. Next up, next best practice. Let's use fixtures for data that we want to use in interesting ways. Oh, this is about commands. In commands we do need to specify that TypeScript declaration file to be able to show not only to JavaScript the function that needs to happen but also to TypeScript the definition of that. Commands are a really helpful way to abstract away the particular details of implementation of these features into a central place we might choose to reuse in many tests and instead focus on describing the user's journey in our unit tests. These commands are really helpful. If ever you've used Selenium's page object you may feel right at home here with commands. Next up, don't log in at the beginning of every test. Now if your site is an authenticated website you might want to do a before all where you go grab an authentication token. You do want one suite of tests that will validate the login experience but you don't want to do the login at the beginning of every test because that's a lot of screens to click through just to get to the next step. Another best practice, use fixtures. We use fixtures to be able to swap out the top stories. You could use fixtures to specify specific users or company or customer data and grab the data out of that fixture anytime you want to do that. That makes your tests a lot leaner. Next up, we mocked an API so that we could validate how a particular control can render. Now mocking an API is great and using fixtures for that is wonderful but don't just mock the API. You want to be able to run the real API as well to validate that your assumptions in the mock are indeed true that the API surface area hasn't changed. We took a look at Cypress, how to get started, how to level up in interesting ways whether you're using JavaScript or TypeScript. Cypress can, pardon me, Cypress can be a really fun place to work. The slides in the code are online right now. If you're watching this video later hit me up on Twitter at Rob underscore rich. For those of you who are here live join me in that spot the conference has designated and let's discuss Cypress some more. Thanks for joining us. Thank you, Rob. It was very informative session. If any questions, please feel free to ask. I don't know, my voice is repeating. I'm actually surprised we're able to do questions live. All the other sessions they just shuffled me off into another session. This is cool. Yeah. What have you found helpful in reducing test fatigue when the design requirements change a lot? Good question. The purpose of tests is risk mitigation. And if the designs are changing a lot then we may want to take a higher level path through the tests maybe only validate the critical functions or only validate those things that have kind of baked some. If our goal is moving fast and not risk mitigation then we may only have a few tests. That's a great question, Lenny. Thanks. In relation to it how do we implement shift left testing in situations where the design requirements may change in response to customer feedback? Yes. Cypress is great at shift left testing because you can run these in a very automated way either right at the beginning of right in the DevOps process or even on your local work station to be able to understand those pieces. Sometimes when I'm developing a feature I'll develop the Cypress test to help me validate that feature and then I'll actually build that feature out and kind of use a TDD approach with Cypress. Great question Lenny. How do you suggest we test things like requirements, design specs, et cetera? That is great. Ideally you can take your design specs and you can turn those into a suite of tests. So we've described the user's journey through the application. Let's build a test that validates each of the steps along that journey or validates that entire journey as a whole. One of the things that I love to do is build a test that will launch the website, get into the product catalog, pick one, put it in the shopping cart and get to the purchase page. Now I'm not gonna complete the purchase but I've validated that whole critical journey to validate that my site is behaving correctly there. Then I'll run that test in production every few minutes or every hour. At that point I validated that my production website is behaving the way I expect. Now in the case where my site goes down, my Cypress test running in production will probably tell me before users do. That's really helpful. I'm able to validate those critical requirements and validate it running in production as well. I'll run it as part of my DevOps build as well but if ever any connectivity issues happen between my services, I validated that critical path and not actually checked out, not charging money but got myself all the way to that step and I can run that test in production really frequently. What metrics have you found useful for web applications with regard to software quality? Such a great question, Lenny. The easiest one to measure is code coverage and I found that that metric is pretty useless. 0% code coverage can tell you a lot that you don't have any tests but 100% code coverage doesn't tell you very much more. If you're in the 80s, 70s or so, maybe 90s, it tells you that you're working pretty hard in that spot but if you get to 100, usually it tells me that you've gained the system. Instead, let's look for scenario coverage. Exactly that, let's take the user's requirements, let's flesh them out into tests and let's be able to execute each of those tests. If I have an if block in my code, I have two different tests, one to flex each side of that scenario. If I have two if blocks, I have four possible conditions. If I'm only measuring code coverage, two tests would get me at 100% code coverage but it wouldn't validate all of the scenarios through my code. If we can look at scenario testing instead of code coverage, I think we can get better metrics here. Now the hard part is scenario coverage is a really hard metric to measure. So we tend to lean on those easy metrics like code coverage. Ultimately though, it's about risk mitigation. And so whenever I find an outage in production, I'll first build a Cypress test that will validate that that function should behave the way it expects and that test will fail. Then I'll validate, then I'll build a fix for it and set that in place. Now I'm confident that that production outage will never happen again. These are really great questions. Thanks Lenny. Yes Rob. Thank you so much Rob. Most definitely. Thanks for having me here in Boston.