 Hi everyone, I'm Tiffany. I hope that the weather is good to you wherever you are. I'm going to be talking today about user-centric testing and how Expedia Group specifically was able to build a testing pipeline that is resilient to change. So in Expedia Group, we have thousands of developers and millions of users every day. Our changes have to be airtight, easily refactorable, and consistent to users overall. Over the year that I've been working on Expedia, or rather for the past two years now, I was able to build a user-centric testing suite that is now the standard at Expedia. I want to share how you could also build one following a lot of the philosophies that we've adopted. In present day web interactions are quite complex. Accessibility is a priority, especially if you're a company that wants to follow accessibility guidelines and legal guidelines and performance is essential as well. UI updates also will always keep us on our toes from external libraries. How can we have confidence that these changes won't cause unwanted side effects while minimally burdening devs so that they can move on forward and make new things? I want to walk through a modern end-to-end testing suite which puts users first for a maintainable and scalable code base. At the end of this, you should be able to identify who your users are and test for their individual needs while balancing cost and confidence in your systems. It's also to be noted that the texts used within Expedia are that would be presented are specific, but the philosophies can be used for whichever testing frameworks that you have in place. So now that's out of the way. Maybe a little bit of an intro on me. So I'm Tiffany and I'm an engineer at Expedia Group and also a new speaker. I've been an engineer there for maybe the past two years and I work mainly on the UI platform teams there and also on building landing pages so if you just that you look up when you are googling top 10 hotels in Miami. So I am also a mentor for Fico Camotralla and I am one of the founders technically of Starship but I'm not really good at Rust. And that's about it for me if you want to follow me or see what I'm up to. I'm not super active on Twitter but what day could be so you can find my links here down below. Let's go into what a good testing pipeline is. For us a good testing pipeline is one that captures unintended changes affecting your users. It's also one that gives you confidence that production won't break with new changes, does promoting a culture of quality, and finally one that provides quick and effective feedback to your developers. This means that when something breaks you want to be immediately into knowing why that happened and immediately into how to fix it. Let's talk a little bit more into what this means. Here we have a wizard or a typical place that you would find yourself if you were looking for a property. You would go in this wizard, put in a few dates, put in a few of rooms and guests that you want to be in, and then you would search for a property. Here we have a test that was able to catch that something has changed on the page. For our beautiful National Benf National Park image with a place with one of the screening frog. This is something that might affect the user. Maybe they don't want to be searching for a hotel anymore in a place that has screening frogs. This is a little bit debatable, but other examples would be for if, for example, the date picker didn't allow you to input your dates or if the header, Fine Hotel's near Benf National Park in Alberta, has either changed or didn't show. These are things that are actually impacting the user experience, so we want to catch those. Here's an example of initially the same page, but now we see that it was actually an each one wrapped with a div for this this header here. And then there was a change made by the developer and what they did is they just removed the div because they saw it was redundant. There was no change that was visible or affecting the user. However, the test still broke. This is a bad catch. It's fragile and it annoys developers of discourage refactoring. This is the same for if you were to check for class names or the internal workings of the views, some prop that doesn't actually affect the way that your component is being interactive. A lot of this follows the philosophy of Ketzi Das, who a lot of you may know as the creator of testing library or React testing library and also a phenomenal speaker. So he tweeted out saying, the more your tests resemble the way your software is used, the more confidence to give you. And this is a philosophy that I want you all to keep giving your hearts because it's the one that is going to be following us all throughout this pipeline. For Expedia. As I said before, Expedia is home to a billion dollar industry with millions and millions of travelers. So we're always pushing changes, we're always refactoring and we want things to be consistent and of course not break. So let's take again our page. Our data analysts have found out that actually Screaming Frogs have brought us more users and so we're going to keep that. We still have again the wizard where you can enter your check updates, check updates and your guests and search as well as some miscellaneous links. So who would you say are the users of this page, this wizard? They are all of our customers. So this is anyone that wants to come in, they're going to want to search and enter your dates and search for properties. No matter how what kind of accessibility means that they may have. If you need any more reasons to handle accessibility use cases and follow accessibility guidelines other than it's the correct thing to do, there's of course a lot of data showing that it's actually good business-wise to handle and support these customers. Then we have data analysts. These are people that are not really our customers but they are going to see what are the track events of the customer through the phone making sure that our page is running properly and seeing if there's any anything that we can make better. These are the people or rather the non-people that are going to be crawling through our pages and giving us a score making sure that we have good rankings and that we're following all the proper practices as well. What they want is good metadata, performance and accessibility. This is very important to us like just media because we want to make sure that we are ranking as high as possible in the search engines. Now onto the brass tacks. We have here the pipeline that is most common at Expedia. Initially we start with static checks so these are any checks that are running while you're programming or while you're programming or right before you're pushing code such as if you're having a text script or any linters that would make sure that you're catching any really coding mistakes for example for getting to initialize a variable before using it before you push your code. Then as you go up into the pipeline once you submit it for us then you jump into a lot of the tests that we want to run parallel. We have here the end-to-end tests and sorry the unit and integration tests the happy facts with end-to-end. There's a regression test with critical flows, performance and accessibility tests with lighthouse and then that ends up getting deployed into non-breaking smoke tests with which will test for extremely critical flows across device tests and across device and browser tests. Keep in mind that there is a certain performance at hand that we have to keep in mind. At the top really really fast you have integration tests which do not need to be running an entire application to run your tests. Then you have the anti-test which are actually building rendering and then going on the actual browser in order to test your application which of course is slower, synchronous and can also be a little bit flaky as well. Then you have visual regression tests which will actually take screenshots of your page and compare them to different versions that you had before. Then finally you have performance and accessibility tests as well as any other checks that you might have. These speeds depend but for performance and accessibility tests that we use it is quite slow because you are again having to run your application and run these lighthouse performance accessibility tests on each one and it has to go through the entire dome. We're going to talk about this a little bit more in detail in the following slides. We want to make sure that we are covering the cost versus how much confidence we can get from each stage of testing. It's part to know where we can focus, where should we focus our time as well as to make sure we're balancing the cost of the test versus the confidence it gives us. When you're going as you can see from the previous graph as you go more into testing the way the user was with an actual device and how they would interact with the page then you get into something that gives you more confidence but it's also much slower and more expensive. Expensive can be monetary but it also means that it's something that will cost your developers all the time. At the very very top we have cross-device which is actually running on a real device most of the time or it's going to be running on on Selenium and and is going to be having a lot of flakiness as well. And you have visual tests again that are quite slow and end tests and then you see in the middle bracket where we have most of our tests is integration. It's a little bit hard to define what integration what a unit is and what an integration is inside of UI code. In most cases we try to avoid those terms and kind of just say their functional tests or their integration tests but there is of course use cases for pure unit tests. Then we go to the static checks as well which do cover a lot of use cases. So those are things that you want to keep in mind when building a pipeline of what it is actually going to cost you to add these tests and where you should put them. You always want to have more confidence over coverage as well. Alright, so let's look into unit tests. Hooray! So we use just testing library with the react one and just on external expect. If you're starting from scratch I would definitely recommend using testing library which is framework agnostic and naturally encompasses the best practices talked about here. If you cannot change your tools then try and keep in mind that user-centric philosophy as we talked about here and go forth with what you have. So if you're for a unit test for example this would be checking the input label and all image that are inside the document and track them just getting sent as well as error states. For example, taking in account the wizard that we had beforehand we can check that the inputs and the images are in the DOM. By checking that the label is there for checking and also checking that our image the screen frog is there by its alt text. Another example would be checking that the track events are sent on click. For example here we are going to get the button by the search button by its text search and then we're going to click on it and then once that's done we are just going to check that it has been called and that the track events have been called with the proper arguments. Other examples would be checking error states from bad customer input and bad idea responses. If you want to cover SEO in here and have metadata you can also check that or check that the proper tags exist as well if you wanted to do this here and not in the lighthouse stage. As you can see here these are pretty good at covering accessibility guidelines. Of course not everything can be covered by this but the fact that there's not really better way to be testing this other than using the actual alt text or a label text ensures that you are actually using those well during your development which is great. It makes it so that accessibility is not an active dot and it is something that your developers can get used to as well. Other examples would also be entering dates and entering the number of guests and anything here really is where you would have the many different experiences that your user would have as much as you can really. We have end-to-end tests and you're actually building and running the app locally or in a CI and interacting with the page the way a user would. So this would be for example entering dates and searching and making sure it goes to the proper URL. Again, end-to-end tests are quite slow. Everything is happening synchronously because it has to wait until you have to wait for something to appear before going forward most of the time. For example, if you're clicking a button that opens a dialogue you should wait until the dialogue appears before running that test. That's why we only really wanted for a happy pass and we use Cypress because it acts the least like a black box and runs on an actual browser. We also use Cypress testing library for additional Cypress commands that will match the just ones that I showed previously. Let's go into an example. In here we are going to be doing what I just said which is we're going to find the check-in and check-out dates, enter Rs and click on the send button and then make sure that it sends us to the correct URL with the correct query terms. What we do here is syntax very thin, familiar to the last slide. We find the label text check-in and then we type in June 23rd, 2020. Then we go and we find the check-out input and then we type June 24, 2020. After that, since Cypress is running synchronously, we know that these tasks have been done. Now we can click send by finding the button and clicking it. Once we've done that, it's going to check that the path name and the search has been changed correctly so that it has gone to a different URL that has hotel search in this case and it was given the proper check-in and check-out dates. It's all very human readable and very easy to go through and then see how it is that broke as well. I want to show here an example of how Cypress would be run. Let's wait a little bit until this person is done writing your test. And there you go. You can see that, you can go back a little bit here. You can see that it was able to write its test, make it go visit to a page and an actual browser will be open for you to interact with. This is for development purposes. It makes it very easy to debug because there is a console that you can interact with as well. And it shows you step by step the test running and what happens once it gets to each step. In this case, it went in, it went to a URL, it found some inputs, typed in it, and made sure that the values change. Visual regression. Most smaller applications don't need this. So that's something to keep in mind. It's useful when you have external UI libraries and when you have applications that you do really want to check any UI changes that might have cosmic side effects. For those that don't know, a visual regression test is one that compares a base image to a new image taken once changes have been made and make sure that there are no differences between them. In this reference that we have, we have a page that says welcome to a dog with a dot. Then once there was a change to change the dog to a candy cane, a different image was taken showing where were the pixels that changed. We're pretty sure that the app did not need to change the dog from up dog into a candy cane. And now we know that that did happen. And visual regression is really making sure that those those unintended side effects are intentional and not just having updated a library and maybe not realizing that it propagated to some little page down below. We use Cypress again for taking the screenshots. And we use Cypress screenshot diff for matching screenshots and running visual diffs. There are a lot of different tools that you can use for this. We use Cypress screenshot diff because, well, I made it so this is the kind of self plug and it is also allows you to have multiple different screenshot folders so that's useful for our repos that are smaller repos. And the way it works is you run a simple command, Cy.match screenshot, which is going to act pretty much the same as match snapshot that we have inside of jest. And what you can do is very simply go visit the URL, go find whatever element you want and take a screenshot of that. So it's very, again, human readable, which is nice. We have a rather, almost like we have other checks. This means lighthouse for web audits that check for performance, accessibility best practices SEO, and also size limit for bug size, making sure that when we are upgrading libraries or making changes, we're not going to be going over our size budget if you happen to have it. This is really going to cover a lot of what we want with bots as well as making sure that we are conforming to the standards of the web really. Well, actual finally, we have cross browser and cross device testing. I do have to say a warning that I say whenever someone goes into cross browser to cross device testing, most of our code isn't read by us anymore. React is tested cross browser and will handle events for you so that to make sure that any events that you use are cross browser compatible. We also have a lot of poly fields everywhere that make sure that code is compatible cross browser. There's a lot of static tooling that you can use to ensure that you're not using APIs that are missing poly fields, such as ESLint plugin compact. And if you really are in a position where you need or want to use cross browser and device testing, then you should still make it only a small part of your your testing pipeline, because that typically would mean that you need to use browser stack or rather Selenium. And that is extremely flaky and has a lot of weights and it's very difficult to do test and develop it with. So it shouldn't really not be the bulk of your application or rather your test pipeline. That being said, our applications are typically large enough and have enough users that we want or need it for a few of our use cases. And so we do have it as a smoke test that is not breaking. That means that it has been deployed and is running on the deployed version. And that notifies us if it has broken. But it is not something that would stop it from deploying because they're usually false positives. There's a lot more to this topic. Each section of this could probably be its own talk. But with the time that we have, and this is kind of to give you an overview and hopefully give you an idea of what you should be looking out for when building a testing pipeline, as well as any of the texts that you could apply for them. Thank you all. And I think that is time for questions. If you ever want to go all deeper into this, feel free to reach out to me and I'm sure we can have a pleasant chat about it. Thank you.