 Yeah, so in this talk we're going to talk about Selenium browsers problem solutions to frameworks not three We'll talk about what happens next and we'll do a Q&A In this talk, we're not I'm not going to convince you to delete and restart. I'm not getting paid commission At least I don't think so We're going to skip over mobile. We're not going to talk about all the alternatives out there and we're going to ignore Python and Java So let's talk about in the beginning. So this is life after Selenium Let's talk about in the beginning in the beginning. There was Mercury and Mercury created QTP and that was created by Mercury interactive in 1998 It was one of the first if not the first browser based UI automation tool. It was closed source proprietary $4,000 per copy and that was in 1998 dollars. I should say USD So that's probably more maybe almost double that And it had a limited browser support It only I believe only worked with IE 3.0 or 1.0 and it only ran on Windows And so Selenium came from that Selenium is the cure for mercury poisoning and that was what the idea for that's where the name comes from It's created in the year 2004 long after that net is now in 2021 maintained by a Selenium HQ Organization is open source and open standard Cross it was designed to be cross browser instead of just working one place They wanted to work everywhere. So they created the abstraction that we know today, which is web driver So in today in 2021 Selenium is the infrastructure and rule the tooling to run web driver But web driver is now the open standard. It's built into every browser now It's part of the W3C specification any browser you using today now has web driver built in And so therefore it is cross browser by definition So what does that mean? Well, it's an open standard. So it's maintained not necessarily maintained by the same folks So while preparing for this talk today my in you've all probably seen this my Chrome driver binary got out of sync with Chrome In which it was out of sync with web driver and you know in the Selenium test itself This is the problem with being an open standard, but um, let's talk about what web driver itself is driving So the best way I know to do that is with a real-world application. And so this is the Cypress real-world app Available on my GitHub repo is not created by me. It's built by the Cypress team But everything we know and love this testers. It's got a login page. So, you know username password sign-in button So this is Chrome and on the right is the Chrome dev tools on the right with all of the network traffic that we're going to capture here So I've entered in the username. I've entered in the password. We click the sign-in button in my case You can see here that we were successfully logged in we got a successful request for login now I'm going to go to the new button up here and you can see what happens, right when I got new I got more information more rendered on the screen and when I start searching I'm going to search for Devon Becker. I'm Searching and getting a payload back so you can see here This is the request that the browser is sending to the API server Which happens to be a local host and that is getting me a response Of a JSON payload that is then simply Rendered on the screen. So again, you log in you go to the new button You wait for the new button to exist and then you click on it Then you wait for the search bar to exist you click on it and you enter in the string Devon Becker the web browser Receives this JSON payload from the API and renders it on screen and that's just like the simplest Thing I can think to demonstrate Selenium. So let's talk about Selenium. So in a review login page Authentication via an XHR API request. We're routed to home We see the new button we click on the new button and it routed to a new URL we the the browser renders the search bar and then we search by firing off requests and receiving a payload and that request is seen by the The user so I've got a Selenium test here that does just what we just did by hand. So we instantiate a browser We in our case, we were just doing a simple check to see that the name of the application is up We are going to wait and use the Selenium best practice for Waiting and polling until this statement is true. So it's going to pull on internal implicit waiting For 10,000 milliseconds or 10 seconds and we're waiting for the username field to exist when it does exist We move on in the test logic. We send some keys the username password. We click the button and then we Then we click on the the new button that we just saw at the top right and then we're going to find the search bar They're usually called queue for some reason Query I guess and then we're going to send the keys Dave on Becker and assert that we've received Dave on Becker text in the little drop-down. So this is a test So again, if you want to run this for yourself, please check out the GitHub repo. So we've instantiated the browser We've routed to local host We've see the sign in page we've logged in and oh we failed and why have we failed? Well in our case There is no such element For the new button and if you look at this, it's because we immediately went from the login page To the new button, but we have no idea from Selenium standpoint. What happens in between And that's the problem. That's what we're here to talk about today So why is Selenium flaky it's because you're effectively stuck here In your in your test code. You don't really know what's going on Between the web driver and the browser You have to pull until true and it's stateless. It's restful. It's stateless. It it becomes true It's not event driven and you have zero insight into what's going on between the browser and the web app with Selenium So you're effectively it's not event driven. It's polling versus listening for events There's this loose cup and coupling because you're effectively stuck here at the web driver level or the you know Your client library in our case JavaScript and there's just generally a lack of observability here So let's do that same exact test, but we're going to add Additional latency in our case something that's closer to the real world will add You know the difference between Jenkins and browser stack for example a hundred millisecond So let's run that exact same test and What you'll see here is because we're pulling until true It continuously pulls until this is available Polls until this is available on a given interval and then it moves on and fails in the same way It fails in exactly the same way But we've we've made it even the problem even worse by not running locally with local hosts We've made it the problem even worse by running like you would in Jenkins, which is a huge problem So to summarize some of the problems is an open which creates some dependencies without ownership like you saw from the first failure message I showed you where Chrome driver couldn't talk to Chrome because it was the wrong version That's because it's an open standard, you know, Chrome doesn't necessarily have to support Every binary version of the web driver by there's abstraction So there's because it runs everywhere that creates a problem, which is you are observing You're stuck on the outside which caused speed and performance issues It's expensive to run because you're waiting you're spending a lot of time, but also It's inherently flaky due to this elusive coupling between the test code and what you actually are trying to test Which is the web application So let's talk about the solutions here So the solutions all come from what's happened in the web space. So in 2004, which is off the screen here back here Chrome didn't exist Chrome didn't exist until about this portion here and then it took off and during that time Everything else kind of cratered. So the definition of what multi browser means has changed the definition of what you know Going from IE to Chrome That kind of has kind of shifted the entire landscape to What we're looking at today, which is puppeteer. So puppeteer is how the Chrome team Tests Chrome and that's hugely important because if you're going to test Chrome, you need a good interface You can't have this lack of observability. So they created an API Using that Chrome DevTools protocol that we looked at which is then, you know then everything that you saw me do manually and you saw the network request come across is now exposed to your tests in the puppeteer and to further, you know They've also made the API almost exactly match web driver in terms of the way that it looks in the way that it feels So it's almost interchangeable in that sense, but it gives you everything you kind of need here Also, it's evergreen. It always works If you've got a version of Chrome installed then you've got an associated puppeteer binary that works It's easy to get you MPM install puppeteer. It's now got multi browser support I remember that was a big deal for Selenium through the nightlies and Firefox But I do want to call out it is not a framework just a Selenium replacement as puppeteer is not a framework It doesn't necessarily have assertions and that sort of thing But yeah, so let's just spend one more slide on this to say puppeteer is now event-driven The WebSocket interface is now provided instead of a stateless REST request interface And you're getting everything you kind of need within the it with you know kind of within Chrome here and that it provides even more power than I think is immediately obvious that we're going to look at at the end here But you are now inside the browser and you're getting real-time information streamed back to your test About the application and if you look at it head to head effectively you've moved from here with WebDriver to here You are inside the browser. You know everything there is to know about the web app You know everything there is your test can know everything there is to know about what the web app is doing Which is where we want to be as testers. So a quick summary It's not a W3 standard. It's real-time instead of polling Very few dependencies to manage npm install puppeteer It's cross browser ish just Firefox now. It's not inherently flaky again. It's not that abstract Because you're inside the browser and it's very lightweight very little that bring to bring with it And that's a good thing And let's take a look at the best example I can say the best example I know of the impact that CDP and puppeteer have had is now that WebDriver IO Which was created in 2013 is now switched from WebDriver bindings to puppeteer by default It is so powerful that WebDriver IO needs a new domain name Because it's it's no longer appropriate to really call it WebDriver.io But let's take a look. It's a it's an open source framework But let's just let's stop talking to the look here. So I've got a WebDriver IO test looks very similar to the Selenium test We looked at you. You're you know, you're waiting for the username to appear You are setting the values just like typing You know typing keystrokes you're clicking in exactly the same way and all of this is using WebDriver under under the hood You're waiting for it to exist and you're you know, you're just following the standard things. So Here we're again, we're at the search where we set the value in the search bar, right? And then at the end of it, we are going to assert that Devon Becker appears in that search bar So let's give it a shot. So I'm going to run This same test, but I'm going to use WebDriver IO and the WebDriver bindings that the framework provides So boring boring boring boring But not so boring it failed and that it failed for the exact same reason that this Selenium test failed, which is it failed here It doesn't necessarily know what has occurred between the search and it doesn't necessarily know what's occurred between And what's received and visible by the DOM in the clock for the client to see the browser to see Because this is still using the await mechanisms given by WebDriver You could use an anti-pattern which is like putting in pauses, but that that's not really what we want. So let's let's instead of that Let's give this a shot. Let's switch from WebDriver to puppeteer Which is what's so cool about this is it's almost interchangeable if your framework supports it, which you know this does here So what's a little bit different here is remember We the test is the same until this point But what we're going to do is we're going to intercept the network request search term for Dave on Becker and guarantee that a 200 comes back from the API And then we are going to move on only after this becomes true only after this becomes true and let's see what sort of effects that has on the outcome of the test I'm going to save my I'm going to switch to the other one to save my changes And you'll see in the the code base just how easy it is to switch between puppeteer and WebDriver So we're going to instantiate another browser same thing except now we're using CDP instead of WebDriver Bingo and it's done it worked it did everything it needed to do because And it did it as quickly as possible basically real time because of the it has the information it needs to know when to transition between states Which is super cool super powerful and kind of there's really no going back like once once you're in the browser and you need you have access to all the information you need why go back So unfortunately we don't have enough time to demo playwright today in this abbreviated version I am going to talk about it and I'm going to show a screenshot from it but just think about it as a problem It's cross browser testing done right it's open source but it's managed by Microsoft it's not using the same CDP interface exactly it created a shim so that they can use other browsers It's controversial though and it best kind of illustrates what's happening in the space which is Microsoft hired the puppeteer team they just bought them they poached them and moved them in And now we you know now you have Chrome now now you have puppeteer and you have playwright but it's causing issues because playwright is using a fork of a fork of Firefox puppeteer but not the same one that Firefox is using now So that makes your head spin yeah it should it's now very confusing and it's because of their shim that they've added here it can run one cool thing is one of the reasons they did this was so they could run on Safari which is effectively you know WebKit Which means you can run you know your your UI tests on Firefox Chrome and now Safari effectively through WebKit And I'm going to give a picture of that but effectively it's not a framework right now it's a fork think of it more of a fork of puppeteer so if WebDriver.io wanted to I think they could very easily change their their their interface from the CDP puppeteer interface to the the shim here and I'm sure there's probably a package out there that does just that for folks that are using WebDriver.io tests will want to run it with playwright And what does that give you will really right now today that gives you This which is WebKit the ability to run Safari tests anywhere you can run it on a Jenkins Linux node you can run it anywhere you can cover Safari effectively which is awesome And unfortunately we don't have time to demo it but what I do want to take the time to demo is Cypress IO so a little bit different Not using CDP directly it kind of is but it's proprietary so we're going all the way back to the mercury days is proprietary It's way more than a framework it's kind of a holistic all in one solution to UI testing you just npm install Cypress and you have everything you need they have a freemium OSS model that has a dashboard service that gives you test analytics and everything you need to kind of complete the total feedback loop for tests This cross platform now Firefox and Edge are now supported But otherwise for most testers it's going to become clear kind of like where the power is with this tool here in a second in the demo and they go ahead and show you so I'm going to show you the Cypress test So the Cypress test here So in here we are going to do the same thing that we had been doing previously you can notice that it has built in waiting it's waiting for this to be be present before moving on but it's all basically the same except we now have access to this information We are going to wait for this to be true before moving on into the test very simple very straightforward Yeah let's just run it so to run it we're going to do npm run test Cypress open and this is the headed version you can run it endlessly if you'd like to as if it were running in CI This is the GUI that comes with it I'm going to select the test I want to run I'm going to run it against canary so let's run it So what makes Cypress unique and differentiates it from the rest is what you're seeing on the left hand side so right now the test that we looked at is running and it's identical to all the other tests that we saw previously It's you know entering the username entering the password clicking waiting for this to exist and clicking on it in it's failing in the live demo which is great let's run it again Let's try that one more time right now I don't know if you can see but my computer is slowing to a crawl yeah so unfortunately this is exactly but we can see exactly what's not working right now That it is not exactly this this element is not exactly ready because the URL is not exactly correct and you can see the XHR requests as they go across the wire So that's kind of kind of the power of Cypress is it gets you the inside of you know why did your test fail and you can see where it failed you can get a snapshot in the DOM of what it saw when it failed and in our case this XH this link is actually bad This link that we're looking for is actually bad because it's seeing two versions of that so let's move on and pretend that didn't happen it's a live demo this is the way it goes And I'll figure out what went wrong after the after the demo but let's let's do one step better and use Cypress 6.0 and take a look at what we can do as testers when we have access to everything in the browser and in my case I'm going to replace the payload that comes back from the API with my name And a first name and last name which is pretty cool I'm going to modify it on the wire so let's do that yeah that's also failing Let's let me go ahead and do what we're not supposed to do and let me go ahead and add a weight I think it's just been bogged down by the fact that I'm presenting so let's put a weight which is the anti pattern from Cypress let's go to give it a shot There it is there goes and what you can see is you can see you have access to all of this information so you can see everything as it comes across the wire it's available in the test and you can see I I just took and I made up a payload You know real time in the test which is awesome what that means is where we are now with Cypress 6.0 as we are here we're actually on the other side of the web app or on both sides of it we can modify it on the wire. We can we can make the test be anything we want so it's really powerful for example if you have an external service that's you don't want to necessarily create all the time you can mock it out sub it out or you know make the You can make this test fail any number of ways you know every once in a while you need to test that a 500 error code is handled by the the API well here's your chance you can do this however you want So in summary it's real time without flake now these all the solutions are fewer dependencies less maintenance they're all cross browser but they're open source with an issue that we're going to talk about which is kind of the future right now which is where we are today with CDP. We've now gotten to a situation we never been in where we have multiple competing standards multiple competing organizations Microsoft Google Cypress WebDriver IO all kind of driving towards the same goal but they were no longer all happy with Selenium WebDriver. That is to say until now so the one thing that I want to say is that Selenium 4 is up or the beta is being released you can see it here it's using an open source W3C bidirectional CDP standard. That's where they're going that's very new should be happening in the next few weeks so keep an eye on that. And this event based and effectively yeah we've gone kind of full circle here and now Selenium is now right back to you know solving a lot of the problems that that we talked about earlier. So I'm going to stop so I can leave some time for Q&A but in effect we talked about Selenium we identified some of the problems walk through two of the solutions and with Selenium 4 we kind of talked about what happens next with their bidirectional CDP replacement interface.