 Hi, everybody. My name is Andre. This is Joel. We are Chromium engineers working on DevTools and Puppeteer. So we know this is the third day of Google I.O. And it might have been overwhelming. So thank you for staying awake and keep coming to the talks. Today, we will talk about modern web testing and automation with Puppeteer. We will split our talk into a few parts. First, we will introduce you to Puppeteer. Then there will be a surprise announcement. Then we'll talk about testing the modern web with Puppeteer. And then we'll wrap up with a community. So what is Puppeteer? So Puppeteer is a browser automation solution. It brings power of web to Node.js with simple to use and easy APIs. So with Puppeteer, you can do many things. You can open pages, navigate to websites, evaluate JavaScript inside these websites, and many, many more other stuff. So in fact, we encoded this in our logo. So you see over there is a browser frame. This is Chrome. Up there, the green thing with bars and strings. This is Node.js that controls Chrome through the APIs. So seeing once is better than hearing. So I want to actually show you the Puppeteer. And for this, let me introduce you to Joel's personal Merchandise website. So on this website, Joel sells t-shirts, coffee mugs, stickers, all with his name on it. So my personal favorite is this t-shirt. But unfortunately, it is a little bit pricey for me. It is $50. So every once in a while, I keep coming to Joel's website in hope for a discount. But unfortunately, it doesn't work like this. And being a lazy developer, I thought that this is a perfect task for web automation. So let's see how I did this, this Puppeteer. So the first thing I do, I install Puppeteer with NPM. And when Puppeteer is getting installed, it downloads the browser. And this already eliminates the pain point of browser automation. So this Puppeteer, there is no need to configure, link, or otherwise manage the browser installation. I just do NPM install, and I'm ready to go code in. So let's code. So this is Node.js script. So the first thing I do is require Puppeteer. So now, all my code will go in between these parentheses. I use the async function so that I can use the async await syntax of the JavaScript. So first two lines. I will launch a browser with one line, and I'll open a new page in this new browser. So in the top right corner, you can see the browser frame. This is actually what it will look like. Next, I will navigate the page to the Joel's website. And then I know that Joel has his price in this specific CSS selector. So I will use this Puppeteer API to fetch the price and output it to Node.js console. And yes, I will not forget to close the browser. So this is as easy as I guess. This is just six lines, one NPM install, and I have a browser automation for Joel's website that pulls the price. It's really simple. So now, let's see how it worked. I had it running for a few days before I.O. And this is a chart of Joel's T-Short price. As you see, the price, unfortunately, didn't go down. But in fact, increased, which is fair given the I.O. demand. I want to make some money at I.O. Yeah. However, I have one last hope. I have heard that certain websites change prices depending on the browser I use. But Puppeteer is all about Chromium. But we are happy to say that there is another browser that is joining Puppeteer family. So with this, let me introduce you to Puppeteer Firefox. So Puppeteer Firefox is currently an experimental project. It is available on NPM as a Puppeteer Firefox. And it is a work in progress, but it already supports 90% of Puppeteer API. We are working with Mozilla to bring the project out of experiment and support more Puppeteer APIs. So let's see what it takes to convert my script from Puppeteer to actually Puppeteer Firefox. So again, the first thing I do, I do NPM install Puppeteer Firefox. And similarly to Puppeteer, it downloads the browser. But this time, it is not a Chromium. This is Firefox. So after that, with just one line, I change require from Puppeteer to Puppeteer Firefox. My script works with Firefox. So let's see how it actually works. OK, so this is the Firefox window. It does resemble Chromium. But actually, something's going on over there. Let me zoom it in for you. Gosh. OK, so it looks like Joel's website doesn't work in Firefox. So clearly, my automation adventure didn't succeed. But at least we found a problem. So Joel is a great web developer. And Firefox is a great web browser. But clearly, certain things require testing and require cross-browser testing for this kind of thing. So Joel, can you please tell us why didn't you test your website? Calling me out at IO for not testing my website. So I was scared to test my website because I thought that it would be really slow to test and complicated. And I was worried that I'd spend more time maintaining my tests than actually writing my website. But of course, this being the Puppeteer testing talk, we hope that with Puppeteer, we can solve these problems for testing on the web. So with Puppeteer, there's no complicated setup. It's really simple. You just do npm install. And it works the same as any other npm library, like ESLint or TypeScript that you're probably ready using. If being a Node.js library, it supports any JavaScript test runner, like Jest, Java, Mocha, Karma, QUnit. There's a lot of them. But people are using all of them with Puppeteer, and it's working great. So getting Puppeteer working on your desktop environment is really simple. But it also works on your CI. We personally test Puppeteer on Travis and AppVayer on our GitHub. But it works on others too. And it works on all the cloud providers, GCP, AWS. And if you need to, it works in the Docker container. So you can run it pretty much anywhere. So web tests have a reputation for being kind of slow. But why actually is that? Because I spend a lot of time making my website really fast, hopefully your websites are fast. So then why are your tests slow? One reason tests can be slow is because you're launching a new browser for every single test. This is actually a really good idea, because it keeps your test isolated from each other. Otherwise, tests might interfere, like cookies could be set, network caching. So running a new browser is great for isolation. But it has a downside, because it makes your tests really slow. With Puppeteer, we have a special API that we call BrowserContext that we hope fixes this for you. It's based on incognito mode in Chrome. And it isolates your pages from each other. But you don't have to launch an entire new browser. So this gives you all the benefits of launching a new browser, but way faster. It's actually 100 times faster. So we really encourage you to use BrowserContext. And let's take a look at some code and see how you do that. So here I have a test that I just verify that a pay button exists on my Merck site. That's a very important button. At the beginning of the test, I'm launching a new browser. And at the end of the test, I'm closing that browser. Let's look at how we make it work with BrowserContext. So first, I move the browser launch call outside of my test, because I'm going to share the same browser between all of my tests. Then in my tests, I create a new incognito browser context and close it when I'm done. It's that simple, and it gives you a huge performance win. So web testing has another bad reputation, which is for being flaky. Sometimes your test pass, and sometimes it's fail. Personally, for me, I feel like a flaky test is actually worse than having no test, because then I have to deal with the flakiness and the problem in my code. A common fix is retrying tests, but there are some other better solutions. So here's a test for my website that checks that I can click the pay button, and then the purchase is successful. But this test is actually flaky. My pay button appears asynchronously after the page load, so sometimes it doesn't appear yet when I try to click it. And sometimes my purchase hasn't completed when I check for the successful purchase button. So this test right now fails about 70% of the time. A common solution for fixing this, if you've done web testing, is to sleep for a bit in between asynchronous tasks. So with Puppeteer, we're waiting for a second after the page load for the button to appear, and then we're waiting for another second for the payment to complete. This makes the test less flaky, it's still a little flaky, and it slows it down a lot. But with Puppeteer, we have an even better solution. So with Puppeteer, we have a set of reactive APIs that we call our wait for X APIs. You can wait for DOM elements to appear. You can wait for network requests to be sent and finished, and many more. So let's look at the right way to fix this test. So instead of waiting for a second after our load for the pay button to appear, we're just going to wait directly for the pay button. Then instead of waiting for a second for our payment to finish, we're going to wait for the payment network response to come into the website. Then we know we can check for a successful payment element. OK, so let me recap. So Charles didn't ask his website because he was scared, but turns out that if he were to use Puppeteer, his fears will not apply. So with Puppeteer, tests are fast because of headless Chrome and browser contexts. So testing with Puppeteer is reliable because of all the reactive APIs we have. And testing with Puppeteer is actually simple, because it's just one npm install out of you, away from you. So now with Puppeteer and Puppeteer Firefox, you can actually test for different browsers, for Chrome and Firefox. Let's see how we can use Puppeteer to test the modern web. And modern web is not what it used to be like 10 years ago. Every other website today is in a web app. We spend a lot of time to make sure the performance is good and that the website is accessible. So how do we actually test for this? Let's go and see certain features that Puppeteer lets you do. So first up, mobile. So it is 2019, and we as web developers spend a lot of effort to make sure that our websites work on both desktops, mobiles, and tablets. So and we do it different ways, right? You can do fluid layouts, responsive designs, or we can even build separate websites for different devices and sniff as a user agent. So how do we test for this? Well, manually, you can pull off your phone and load your website over there. It's easy. Or you can all load your website on a desktop, open DevTools, and enable device simulation to see how it works over there. But how do you make sure that your website doesn't regress over time? How do you automatically test this? Well, it turns out it's really easy with Puppeteer. So let's write a test together that makes sure that Jill's website works on iPhone and that there is a pay button there. So first, I will open the page. And it's very important that in the bottom right corner there is a browser window. It looks like Chrome. Now, with the one comment, I will emulate an iPhone 8. And now the page becomes an iPhone. So now whenever I navigate the page towards the Jill's website, his website is absolutely sure that it's been loaded in an iPhone. Now, I can easily assert that there is a pay button. Now, when I say that the website is sure that this is a real iPhone, I really mean it because our emulation is really extensive. They emulate not only the viewport size, but also the user agent, DPI, touch support, landscape mode, all the things you shouldn't care about, but the devices are real. And what is even better? We have more than hundreds of these devices available there. We ship them. So with Papetier and mobile emulation, you can make sure that your important checkout workflow works in mobile. This is really great. OK, next up, offline support. So offline is really critical to certain types of web applications. For example, phone maps or subway maps. But it turns out that regular websites, every other website, can actually take advantage of supporting offline mode. So let's see what Joel did on his website. So when the user goes offline, his website shows a big red banner that defined me that I cannot actually purchase a t-shirt. So how do we test the offline mode? Well, manually, I can load it on my phone and enable airplane mode. Or, for example, I can open DevTools and trigger an offline mode in DevTools. Or I can even turn off my Wi-Fi on laptop. So automatically, I can do the same thing with Papetier. So let's write a test that makes sure that this website shows this red box whenever it goes offline. So as usual, I'm going to open a new page. I will navigate the Joel's website. And now, with this next command, page.setOfflineMode, I will transition the page from online state to offline state. This will trigger all the necessary APIs inside the page. And if it were to do any request, it will fail with no network connection error. So now it's really easy for me to assert that the offline alert banner is over there. That's it. So for me, this test is not only easy, but actually a pleasure to write. Now, if you deal with offline, chances are high that you actually have to do with Service Worker and you know what it is. So Service Workers are this big, very important part of the web platform today. And they help us to do caching on the web. Now, how do I actually make sure that Service Worker registers? To help me with my demo, I want to introduce you to the other website. So this is pptr.dev. This is a website with this documentation available online, for popular documentation. It is actually fuzzy searchable. It works offline, supports devices and everything. It's a really modern application. And it does support Service Workers, which is important for us. So since Service Workers are so complicated, we use Workbox to help us configure the Service Worker. But how do we actually do know that Service Workers work? Well, the only way today is to go open DevTools and navigate to the applications panel and find Service Worker over there. If they're there, it's good. But automatically, you can do the same thing with Pptr. And it's actually even easier. So let's try to test to verify that Service Worker registers for my page. Again, I open a page. And now I navigate it to the pptr.dev. Now, something really important is actually happening here. So not only is the website being loaded in the page, but also Service Worker is getting registered in the browser context of the page. So with this, I can actually wait for it to appear in the context with these three lines of code. And this is already a good test. It makes sure Service Worker gets registered. But there is actually more. And to show you more, let me bring you to another slide. So now, when we have the Service Worker target in the hands, we can actually attach to it. And once I attached, I can evaluate JavaScript inside the Service Worker global scope. Now, with this, I can make sure that, for example, my logo is properly cached. But there is more. And this actually opens the limitless possibilities for Service Worker testing. OK, let's talk about JLocation. So the web is global. And your application is global as well. So there are many different cool applications for JLocation. For example, if you have a map and you want to show where a user is. Or if you have a use feed on your website and you want to serve the relevant content. So how do I actually test these things today? Turns out, I don't do things. First, I navigate to my website. And I grant the permission to query for my JLocation. Well, next, I'll open DevTools. And I will emulate some location. For example, pretend I'm in London. Now, with Puppeteer, I can do exactly the same thing. So let's write a test that makes sure that JL shows pounds when we are in London. The first thing I do, I set up my browser in context. And I make sure there is a permission for JL's website to query JLocation. Now I can navigate the page. And with this one cool page that said JLocation, I can convince the page that it is actually in London. So now when the JL's website is getting loaded inside the page, you can query for the JLocation. And it actually thinks it's in London. And we can query the price. And it's actually 40 pounds. So good job, JL. OK, off to the really powerful stuff we have in Puppeteer. This is network monitoring. So network is usually a big point in different browser automation things. And the golden standard for network monitoring is DevTools, DevTools network panel. So DevTools network panel gives you a waterfall of all the requests and all the network activity that is actually happening inside the page. So you can see requests, responses, you can see status codes. You can even get the content of the responses. Now what if I tell you that you can get the same network monitor, but with Puppeteer? So Puppeteer gives you two events, request and response. And with this, you can recreate the same DevTools monitor, but programmatically. So there are many different APIs available over there. You can query for headers, for post data, and you can even check if the response is coming from service worker or from cache. This is really powerful. But there is more. Puppeteer can not only do network monitoring, but also network modifications. So let me introduce you to the request and reception. And to demonstrate this feature, let's substitute every image on the web with a cat. So if you were to go to the developers Google com, we will have cats all around. Let's understand how it works. So say I have a browser, a page on the left, and servers on the right. And there is an image on the website that actually issues the request. Now the request is getting intercepted by the Puppeteer, and we can inspect it and see that it is actually coming from the image. So we can fulfill it with a random cat image. Now if there is a style sheet in the page, we can again intercept the request and see that we don't have anything to do with it, right? We care about only images. So we can continue and let it hit the service. Now in code, it looks really simple. So first thing we do, we enable request interception. Then we subscribe to request event. And every request actually gives me the type of the resource it is attached to. So I'm going to connect to image. And if it is an image, I will respond with a random cat image. Otherwise, I will just continue to the server. So we really like this catification. So this is a video of what it looks like if you've got to browse Twitter with this functionality. Looks much better. So what does it all have to do with testing? So Joel's website has a piece of functionality that handles the case when payments don't go through. So in this case, Joel shows this red banner, payment failed. So it turns out request interception is a really powerful thing to do server mocking. So the same thing, we're going to subscribe to request event. And for only the request that hits the payment endpoint, we will abort it. Otherwise, we will just continue. And with this, it is really easy to mock a server and APIs. So let me hand it off to Joel to tell us about more input injection and other stuff. Thanks. So I'm talking about keyboard and mice and user input, which you might think is simpler than the network stuff Andre was talking about, but it's still pretty complicated. If you've done web testing before, you're probably familiar with the Dispatch Event API. This lets you send fake JavaScript events. It calls all your event handlers, your on mouse down callbacks. This works great for making sure that your site works for events. It works all in JavaScript. But this actually misses some of the story of real user events on the web. When real mouse and keyboard come into Chrome, they get sent to Blink, where they affect your CSS, like your colon hover, pseudo-selectors. They go into DOM, and they change your text selection and your focus. So with Puppeteer, we send events directly into Chrome in the same place that user events come in. And they have all the wonderful side effects of user events. Like they go through Shadow DOM correctly. They go through iFrames. You can resize a text area. And our keyboard actually sends the right text in with the keys. So let's look at a test that actually types something in with Puppeteer. If you remember our documentation site before, it has an awesome feature where you can search. So we're going to open a new page. We're going to go to our documentation site, ppdr.dev. And then we're going to find the search element. And we're going to type JavaScript into it. Then all we have to do is hit Enter. And we should see the results of the search. Let's look at it in action. So we actually slowed this down so that you can see it happening. Puppeteer is much faster. But you see, it types into the page, and it progressively performs the search. So it's not enough just to make sure that your site works correctly. We also want to make sure that it works as best as possible. So performance testing is really important. And Puppeteer has a lot of great APIs for you to automate performance testing on your CS. My favorite API is Page Metrics. It's really simple. You just give you a nice list of all sorts of metrics on your site, the same as the performance monitor and dev tools. My favorite number down there is the JS heap size used. Some of you might complain about how much memory your website uses. Maybe you blame a certain web browser. But with Puppeteer, you can monitor your own memory usage and make sure that you don't regress. If Page Metrics isn't enough for you, you probably want to look into Chrome tracing. Tracing is a feature in Chrome where we instrument the entire browser, like CSS, DOM, animation frames, network. And you're probably familiar with it in the performance panel in DevTools. When you want to take a performance trace in DevTools, you hit record and stop and get the trace. And with Puppeteer, we have the same API. So let's make a performance trace for this test that selects the mug item on my site. Before the test, I'm just going to call Page Tracing Start. And I'm going to specify a path where I want the trace JSON to go. Then when my test is done, I hit stop. And then I can drag that JSON into DevTools and see the trace just like I took it in DevTools. So code coverage is another really powerful tool in DevTools. It tells you exactly which CSS and JavaScript were used on your site so you can make sure that you only ship what was needed or that the right code ran. And just like we have tracing in Puppeteer and DevTools, we also have code coverage in Puppeteer. You can start and stop CSS code coverage and JS code coverage the same as tracing. So your site working fast is not the only aspect of it working correctly. We also want to make sure our sites are accessible. You can test manually for accessibility with screen readers and other assistive technologies. But it's very hard to automate accessibility testing. So that's why we have this awesome API in Puppeteer called the Page Accessibility Snapshot. This gives you the accessibility tree out of Chrome. The accessibility tree is what's sent to assistive technologies, and they tell them what content is on your website. So it very closely matches what is actually read to users or displayed. This is the accessibility tree of my Merch site. As you can see, it has the title of the site. It has an image of my shirt. And of course, it has the buy button, the most important part of the site. Let's look at how we can use the accessibility tree to write a test to make sure that my site is accessible. So I'm going to query for the button on my site, the buy button, just regular download element. Then I'll take a snapshot of the accessibility tree for just that button, and I'll verify that it has the correct Google Play name. I had a lot of fun with accessibility trees in Puppeteer, so I actually built a tool called the Accessibility Renderer. So this takes the accessibility tree that was generated from your page, and it generates another web page from just the tree. So you can see if they match up, and if all your content is displayed correctly in the accessibility tree of your website. So here's my Merch site, and it's actually its own web browser. You can navigate to any site and see its accessibility tree rendered as its own web page. So here's Google, and you can try your own sites and make sure that they look how you expect them to look. You can find the accessibility renderer on GitHub, and of course, you can get your accessibility trees through Puppeteer. OK, thank you, Joel. So this is what we did for the features, so let's talk about community. So Puppeteer has been around since 2017, and it's available on GitHub, and many different companies actually use it. So Google uses Puppeteer internally, but I guess no surprise here. Netflix uses Puppeteer internally, and also Facebook uses Puppeteer to drive their performance analysis. But not only companies. There are many different people who use Puppeteer. So there is a Slack channel where thousands of people communicate with each other and help each other to solve questions related to Puppeteer. But community not only talks about Puppeteer, but actually embraces and contributes to it. So more than 100 people have stepped up to help us maintain our API docs. So our API docs are really good quality. We have code snippets for all our API methods. Go check them out. Really good stuff. All thanks to the community. But really important, and we're really proud of it, that 20% of all contributions to the core library are actually community contributions. This is a big number for an open source project. So last but not least, Puppeteer is an NPM native project. There is a huge ecosystem of packages that depend directly on Puppeteer, more than a thousand of them. And we believe that Puppeteer spawned a new wave of next generation web developer tooling. So I want to give you a few examples of what is possible with Puppeteer and what people do with it. So first is a project called Size Limit. And Size Limit uses Puppeteer to measure how much time it takes for a website to load and run in JavaScript. So with Size Limit, web developers stay aware of the size of their website, and it makes them ship less code, and websites load faster for all of us. This is good. So same thing, but for CSS, it's called Penthouse. Penthouse uses Puppeteer to extract the critical CSS, or only the CSS, which is absolutely necessary for the web page to load. So with this tool, you can again ship less CSS and make your website load faster, and your users happier. So this is the end of our talk. Thanks for sticking around. So summary, Puppeteer is the official Chrome API. We are very excited about experimental Firefox support, and our collaboration is Mozilla. And yes, the modern web is testable with Puppeteer in both Chrome and Firefox. So please test the websites. Thank you. This is Andrey, Joe.