 Welcome everyone to the session. Why does building a robust automation framework matter by Cameron Bradley? So we are glad Cameron can join us today. Thank you for the lovely introduction and a big hello to everyone out there. I'm coming to you from Australia and I'm definitely really excited to present why does building a robust automation framework matter. So I've been about myself so I currently lead the QA practice at Bunnings for those that haven't heard of the Bunnings name, you've definitely heard it if you're Australian because it's the number one retailer in Australia, particularly in the home tools segment. I began my career as a software engineer working across iOS and web and during this time I discovered a passion for ensuring what we develop is being delivered with quality and speed to our customers. I followed this passion and shifted my career to focus on improving the quality function. I guess a couple of quick highlights is our team at Tabcorp, which was the company I previously worked for, was a top nominee for best agile test team in the world at global competition judged by our peers. And we're alongside such global companies as Google, Microsoft and Amazon. And finally I founded Testing Talks. And Testing Talks is a wonderful company, pretty well known brand name in Australia. It's all about providing high quality conferences, online events and test automation courses for our community. So these are some of the takeaways I'm hoping that attendees actually get from this talk. I want to really give takeaways of, you know, why does automation framework architecture matter, the importance of building a proof of concept for your automation framework, leveraging engineers and your team to help build a robust automation framework, some key object-orientated design principles to use across your frameworks, how this approach can support multiple automation frameworks and finally some observations, some successes and challenges. So I guess, you know, why does automation framework architecture matter? Now, many arguments could be made about the order of these, but from the top we've got accommodate team skill sets. So, you know, we want to build a framework that should enable our most technical but also our least technical team members to be efficient automation engineers. We want it to be easy to maintain and update over time. We want low complexity code that is not intimidating to fresh eyes. We want to enable best practice and iterations on the framework, both the architecture and code with ease. And we want a consistent architecture approach that also supports things like training, pair programming, skilling up even across teams. And finally, we want low code and easy to reuse. We want to spend less time doing code and more time doing automation. So a little bit on the importance of proof of concepting. So, you know, we want to understand supported features. You know, does it support iFrames, for example? So this is really getting the idea out there of, you know, let's try and do a little bit of a proof of concept of our framework before we actually decide on a solution. And this is about, you know, looking at different opportunities, you know, different frameworks, different architectures, you know, things like building team buy-in. This is something you can actually develop from a proof of concept. You bring everyone in the team into it. You get as many opinions as possible. You get your best engineers in the room. And you really start having those stimulating discussions, which we all know get very passionate when it comes to automation. Some frameworks, you know, they actually cater to certain software better. You know, do you have long forms in your application? Again, do you have iFrames? These are the sorts of considerations you have to make when you're actually picking your framework. This is one that could be debatable. Architectural discussions, you know, something I really love to do is actually stimulate architecture discussions as part of the proof of concept. Really get that kind of thought process happening as our proof of concept forms. We actually start to think of the architecture we'll use in our sort of solution we go with in the end. Finally, you know, quick wins generate momentum. And that's the thing, right? Like as a proof of concept, you can actually prove it out. You can kick some goals. You can start to really develop a sort of message of success and where this automation framework is going to go. Some of the benefits the team are going to see. And then finally, measure and review, you know, which is actually choosing the framework that you go forward with. You know, maybe you've done two proof of concepts and you've proved out the benefits. You've noticed which ones it's going to work with well, which system it's going to work well with and you actually kick the system. So, you know, one of the keys for our proof of concept and how we came up with this robust automation framework we're going to talk about very shortly is that we brought the team on the journey. You know, we discussed team-wide as a group. What was important to our automation framework architecture? I remember I was in my first month at Bunnings and I literally invited, I said to the top test engineering lead, I said, who are the top engineers in the company? I want them to be the most passionate people in these meetings where we discuss architecture and our proof of concept. I want them to challenge us on everything. We set out with a goal to make the architecture so generic we could replace the underlying framework. And I'll explain what I mean there. But really what we're trying to do is create a framework that was, again, easy to use for our most technical and our least technical that our best technical people could really engineer the framework and engineer the improvements but that our least technical could not only build their knowledge and learn the framework but be efficient automation engineers from the get go and we'll speak to that more as well. As I said, we invited the top engineers in the team to a teams channel as well and we asked that they actually review all our proof of concept pool requests and we told them to go hard. We said, you know, if you don't like the code if you don't like the architecture, tell us, let's have a meeting, let's pair program on it. We want to get the best possible proof of concept. We can get out of this at the end of the POC and then we performed multiple iterations of showcases, meetings, pair programming and code review to refine our framework architecture and I really want to highlight that is showcases. Incredibly important. Just like this presentation right now it's about constantly bringing the team on the journey showing the team your wins. What are you all doing? What's the architecture? Who's in on this? Because ultimately that's what's going to really lead to momentum as you build out your framework with the team later. So here's something we decided to use Cucumber. So we didn't use Cucumber because of the BDD benefits. You know, if we talked to the founder of Cucumber and told that person why we use Cucumber they'd probably throw up. It's not something that was originally intended for Cucumber. We actually use Cucumber to enable generic architecture for two key reasons. We wanted a shorter learning curve so we could actually reduce duplication of code and have generic step definitions that could work across any feature or functionality, any interaction or assertion. And then we wanted strong collaboration. So we actually wanted developers and testers to both contribute to this framework. And finally, we have a large practice at Bunnings and as I say, we have a broad skill levels from someone that's never done automation to someone who's done a lot of automation and comes from a very extensive engineering background. And we wanted to build a framework that enabled both. And Cucumber definitely allowed us to do that and I'll show you how soon. So the original proof of concept we actually put together was Cypress versus Playwright. And I've just added a few videos out of fear that potentially running both the frameworks at the same time during a live presentation might be a dangerous idea. So all those demos and the code is all in the slide packet, just for safety, really. But as you can see, the proof of concept is just basically running through the Bunnings website. That one's in Cypress, which took 19.39 seconds. Just your basic end-to-end scenario that logs into an account pretty much standard across most web applications these days. And then on the other side, Playwright. Right away, you can actually see something really interesting there, which is the Cypress one is a lot slower. And for those that have used both and those that understand the architecture and how Cypress works, it's really got a lot to do with those network requests being completed. When we started automation at Bunnings and we were really starting from scratch, we were just automating against the UAT environment just to get this proof of concept up. Yeah, we wanted to focus on local host automation and mock servers and all that great stuff as well. But we just wanted to get a starting point to build that framework. But as you can see, from two proof of concepts, just on time taken alone, we can pretty much see that Playwright might be a better option for the Bunnings website because the Bunnings website has a lot of network requests. And because Cypress awaits those network requests, that simple test is double the time, which is really interesting. So what's so special about this POC though? I mean, everyone's done a POC, right? Well, the answer is the framework architecture was the exact same. And I want to show you how. So on the left, you've got Playwright. On the right, you've got Selenium. And they're the exact same framework. So basically you've got one that's Cucumber, Selenium, and TypeScript. And then the other one you've got Cucumber, Playwright, and TypeScript. And thanks to Cucumber and that use of generic step definitions, what we can actually build is using a lot of object- orientated design principles is the exact same architecture in both. Obviously substituting out what's required to start Playwright and what's required to start Selenium. And we'll dig into that in a second as well. So as you can see, we basically have the exact same Cucumber scenario for our home page. And then we have our exact same code instead. And you'll notice there's only really one difference here. And it's in the Playwright test, we're passing Page because basically Playwright runs off the browser context and Page versus Selenium, as many of you will definitely know, runs off that driver. But as you can see, the architecture of the step is the exact same. There's no code difference except for passing of the Page and the driver. Then you go to the support function where we actually navigate to Page. And again, you can see, we're actually got the exact same code again. So we're basically bringing in that Page or that driver for Selenium. We're determining our host. We're passing in our host, which determines our URL. And then you'll notice again, that difference in Playwright. We're using Page.goTo, which is the sort of default function to navigate. And in Selenium, we're using driver.get, right? So basically there's two differences, but the architecture of the framework is the exact same still at this point. Let's keep moving through the code. So let's go to the last bit of this test. So we've got our package.json. So where we actually kick off the actual tests themselves. And once again, you'll see, they're pretty much exactly the same. In fact, you could have this be the exact same. I left a common config file in one and one doesn't. And that's just because one was designed for Windows and one was designed for Mac. But generally speaking, they run the exact same executable commands. And I will show you how all this works very soon. That's when we're really going to dig into the code and see how this all ties together. But what I want to stress you at this point is what was very unique about our proof of concept was that it was the exact same code, exact same architecture running the exact same test. The only difference was one had Playwright and one had Selenium. Finally, what was the difference? Because obviously there was a difference, right? And again, I just wanted to show you that that's in our initial lives. So basically we initialize the actual framework on runtime obviously for Playwright. We have to set up that browser, set up that context in that page. Whereas in Selenium, we need to set up that driver. And that's the only difference. So that init function is literally the only thing that's actually different between both those frameworks and the use of obviously page.goTo and driver.navigate, et cetera. Okay, so what did we discover for our proof of concept? So we discovered a lot. So we discovered things like about cross origin capability and Cypress being quite complex, navigating between different URLs, external of the actual main web application. We discovered that there wasn't great iframe support. You could use some external libraries in Cypress, but it wasn't native to Cypress itself. And that's totally understandable with the underlying architecture of Cypress. Underlying files weren't native in Cypress. And I just got to take a pause here because generally speaking, when we went into this proof of concept, everyone thought Cypress was going to be a clear winner. Like Playwright, this was over 10 months ago, right? So we thought, you know, Playwright so new, barely anyone seems to even be talking about it yet. Everyone's loving Cypress. You know, we'd all used Cypress a little bit and we're really excited to explore it more and use it more in depth. But this proof of concept completely changed our minds. And again, it goes back to things like, you know, that automatic await network response completion with Cypress. You know, really wasn't the right candidate for the Bunnings website, which had a lot of slow responses. You know, that issue is, you know, things like no need for scroll on long forms in Playwright. You know, Cypress is really good at that as well. But we noticed when we did this in Selenium, because keep in mind, we did this across Selenium, Playwright and Cypress. Selenium was pretty much off right away because unfortunately it still has those issues with scroll that we all know about on long forms. And while we know we can get around those with a few scroll twos and a few different ways to do it, it's just flawless in Playwright. And that really surprised us. But again, it was also flawless in Cypress that. So it wasn't, that wasn't the clear winner. For us, really, it just came down to the fact that the Bunnings website had a lot of internal iframes. So hidden iframes and Cypress did not deal well with that. Like we had to do a lot of sort of hacks, if you will, to actually get this framework up and running in our website using Cypress. And that didn't seem like a good thing if we're only in the proof of concept on top of that. That is network requests. As I keep going back to it, it was so important because it basically showed that every test was going to be running a lot slower. On top of that, Playwright had really good support for Safari, which is something our customers use a lot. It also was really flawless when it came to parallel browsers, particularly on our local host. So needless to say, our outcome was, and it surprised us all, that Cucumber, Playwright, and TypeScript was actually the framework that we would go forward with. It was the clear winner, much to the shock of all of us, indeed. So how did we build this framework architecture? What OOD principles helped us build our robust automation architecture? So we leveraged a mixture of OOD principles in our design and it really started with how we initialized Cucumber World. So for those that haven't heard of Cucumber World, it's actually a really interesting stuff. So basically when you use Cucumber, you can also introduce this concept of Cucumber World, which is within the actual Cucumber framework, and what it enables you to do? This actually enables you to pass segregated global configuration between scenarios. So I don't know about you, but definitely in my early days as a QA, I would define global variables and potentially set those global variables throughout my automation suite. Well, that was obviously dangerous, right? Because if I was running multiple tests in parallel that set that global variable at the same time, it might cause a test to be on my actual test suite to be unstable. So with Cucumber World, you can actually pass all these segregated global configurations all at the same time. So you can be setting that global variable in every test running in parallel, and it's going to pass because it's completely segregated. So highly recommend if you're out there and you're using Cucumber to look into Cucumber World because you'll see the benefits of it throughout this sort of presentation. So as you can see, where do we start? You know, we start in our hooks, right? That's kind of where the automation framework will always start when you're using Cucumber, that before hook. So in Cucumber, you know, you've got your before, you've got your after, you've got your before all and your after all. And as they say, basically this code will execute before your scenario, before all your scenarios, after your scenario, or after all your scenarios. And as you can see here, we've got our before. So this is actually going to execute before every single scenario. So what do we do? We basically console log out running the Cucumber scenarios just for a nicety. And then we set our context options. And as you'll see there, we're actually leveraging Playwright. So this is a minor change to the architecture that's different because Playwright offers you the ability to video record for each scenario, which is really cool. And then we actually initialize Cucumber world. So this enables us to actually create our new browser context and page before every test. Now that was really challenging to work out when this framework was first worked on. And it's because there was limited Playwright documentation in those days, particularly for Cucumber. Don't get me wrong. You could Google, you know, Cucumber Playwright TypeScript Framework and it would show you how to connect a basic test. But there was nothing online for how to use Cucumber World with Playwright. So we literally had to figure this out for ourselves by digging into the actual Playwright framework and working out how we could basically segregate the browser context and page. So then we could actually fully leverage Cucumber and Cucumber World. So this is actually really cool stuff. But as you can see, as we initialize we basically close any existing browser context and page that might exist. We open that new browser. We create that new browser context and we create that new page and we return a screen object that basically will be available throughout our entire Cucumber scenario. But again, I want to stress this because we're using Cucumber World and because we're using Playwright we're in this beautiful scenario right now and we've barely kicked off this framework where we've got segregated global configuration and we've got segregated browser context and page per browser. So nothing shared. You've got a perfectly self-contained test on every test. Whether it's running one at a time, two at a time, five at a time, 20 at a time, everything's contained within its own scenario. So we're often awesome start. All right. Now we go ahead and determine our browser. So Playwright in those days really the main support was for Chromium, Firefox and WebKit which is actually amazing because for those of us that use Selenium a lot as well, that Safari binary isn't quite as good as WebKit in that WebKit will give you parallel browsers from the get go. It will give you headless mode from the get go and then obviously Chromium and Firefox are great binaries as well and obviously really representative of the latest Chrome and Firefox browsers. But basically what this function does and as you can imagine and keep this in your mind is it will show you the framework. It's basically allowing us to pass an environment variable which will show a bit more as well soon of either our Chromium, Firefox or WebKit and that will determine what our actual automation framework, what the browser is that we will run on. So next we actually leverage our index.tx to execute a runtime in source via the CucumberJS and this was one of the most interesting things the CucumberJS with additional customization is actually not designed to work with TypeScript just yet. So let me explain a bit more. So the CucumberJS as it sounds is very well designed for JavaScript. It's not very well designed for TypeScript. Now that won't hurt you if you're doing an automation framework using Cucumber without Cucumber World once again or without wanting to customize your CucumberJS runtime arguments you'll get away with it. But if you want to actually update those arguments unfortunately you are going to need to come up with a solution that converts your TS files into JS files. So the index basically which will contain your custom CucumberJS runtime arguments will execute and I'll show you that in the next slide because obviously there's a bit there but basically it works beautifully you just have to convert it to JavaScript to really get the full benefits and unlock the full benefits of the CucumberJS. So as you can see some minor work done in our package Jason to basically introduce Babel. So for those that haven't heard of Babel it basically allows you to convert your TS files into JS so converts TypeScript to JavaScript pops it in a file of your choosing as you can see here into the disk folder and then you'll basically find all the TS that we've done as part of our TypeScript converted over to JS including our index dot TS which for those that do a lot of TypeScript will know is that first initialized file called so if you initialize source it's actually going to look for an index dot TS by default and if that contains your CucumberJS runtime arguments that's what's going to be executed so let's look at that so I don't want to over confuse. So let's see how it really looks so we use Babel as I said to compile our TypeScript to JavaScript on runtime so we could use and then customize the CucumberJS to enable us to leverage CucumberWorld so to actually use CucumberWorld you actually need to pass it in as dash dash world parameters inside the CucumberJS arguments but what we want to do is we want to have the ability to have customize config for our hosts, our pages our element mappings all those key things that make up a sort of world class automation framework that can do all the things you would expect to do in any web application so we need to pass that hence why we needed to basically have our CucumberJS TS because this is a TypeScript framework as you can see there on the right but this is the trick right so in our index so in our CucumberJS this would sit basically in your root directory it will actually just exports and require dot slash dist and so when we actually run this framework it's going to look in the dist and it's not going to find the Cucumber TS that's right it's going to find the Cucumber.js and that is exactly what we need so we can write our framework in TypeScript and unlock all the beautiful benefits of TypeScript but when it comes to compilation we're going to have this bad boy running in JavaScript which is going to play beautifully with the CucumberJS run up and get us in a position where we can run the framework as is and that's again something that was a big challenge there was no documentation for this this is something we had to work out as a team and thank God we'd run those proof of concepts because we had those developers on board we had these questions we could bring in our best engineers they were part of the architecture they were part of the POC they had that buy-in we talked about so they were on the journey and they helped us come up with this solution by leveraging Cucumber world parameters we can then pass in our host configuration page configuration and element mappings on our run time so segregated between each scenario so just on the left there this is what you call your world.ts and this is a default file that basically the Cucumber world or all the initialization and the constructor sits within and it's actually where you define the things that you want to have as segregated global configuration now you can see we've got global config and we've got global variables so to quickly explain what they are think global config that's where you're going to have your pages your home page, your login page you're going to have your route there, your rejects you're going to match it up you're going to have your host are we going to our local host are we going to our production.com.au host and then you're going to have your elements your login button your submit button and then we've got jason's and I'll show you that in a minute as well finally we have our global variables and this is where we can do things like store e.g. a stored value what about those tests where you don't know what the balance is going to be maybe you're running these on production as well and you want to store your balance at the start of the test and assert that it's $5 less after you do something or you spend $5 by using this approach that will be segregated data so you won't have an issue running that scenario 20 times at the same time doing all different types of updates everything's going to work because we've put in the hard work to initialise QCumble World and have that segregated context we can then leverage config.jason's to create a set of host config page config and mapping config and that's what we just talked about so this is where you have this really clean folder structure right so you've basically got e2e config, mappings in there you've got all your element mappings so you've got your home page mappings maybe it's your header logo maybe it's your sign in button and then you've got your comment jason right because we're dealing with single page applications you're going to have element mappings that sit across multiple things e.g. your headers and your footers and then you're going to have your host.jason this is again where you're going to have things like your local host, your production URLs because you want to run this in local host you want to run it on multiple in test environments you want to run on production right and then you've got your pages all the different pages within the application itself so we can navigate around the application or jump straight to a part of our application or even just know where to draw our page elements from on the left there you can basically see in that index.ts remember converted to js on runtime we've got all the things around getting that jason out of those files based on a host path or a URLs path that will define an environment variable in our comment.n so then we can actually leverage config.jasons to create a set of host configs page configs at runtime so you've got your pages.jason you've got your home you've set it to your slash right home page usually it's just a dead route it's just a slash then you got your host.jason simple enough we'll run this on our local host which is our htp local host 3000 we've got our first element identifier we want to interact with it's our contacts header data ID contacts that's what's in our html that's how we'll reference this element and finally what is this all console log well it produces an object that looks a lot like this so it tells us that we've got a common.jason a home.jason and in that we've got our header logo and our contacts header and as you can imagine we'll be able to soon draw from that using the parameters we pass through with cucumber so let's finish out our automation test let's get this bad boy up and running so let's go back to our cucumber scenario we've got our home page up we've navigated to our home page we've spun up the framework we've got cucumber world initialized we're about to hit our first page hallelujah just hit our home page congratulations let's move on to the next step so we want to go and the header logo should be displayed so we just want to assert that the home page has a header logo so let's go ahead and step into that step so what we can do here is we can actually leverage the config.jasons to create a set of host configs page configs at runtime once again and we do that because it's already initialized at run time it's it's actually initialized within the framework it's there but we're actually going to pass it via scenario world so you'll see their async function this scenario world that's going to draw in that object we just saw in the console log directly into this step definition for that scenario completely segregated once again so as you can see within log out what we want to log out through a debugging perspective but this is a cool line and this can work across any set for the rest of our framework const element identifier equals get element locator page element key global config so we go off and retrieve our element locator based on that parameter header logo and where did we just see that we saw it in our console log because that was what we draw down at the start when the framework ran and we stored in our global config from from our jason files we also have our custom wait for function there and we retrieve elements true or false and assert if the element is visible accordingly we decided to not use playwright tests for those wondering why we decided to use native assertions and it's because the default playwright test library as it did not work perfectly with cucumber so this is really interesting as well right like it has support for cucumber because playwright still sort of in its infamy it's still a relatively new framework it's not perfectly set up for it yet so we actually found that using native assertions was a better approach we also discovered that by using negate it would enable us to have our step do two things instead of one so our step is actually the blast should not be displayed or the blast should be displayed it can actually do both which is really cool then we retrieve our element identifier based on our cucumber header logo so this is just the logic behind that get element locator as you can see it's really simple stuff you think that would be doing a lot of complex code but it's not it's literally just looking in that page elements it's a mapping object we saw console logged out before it's going hey which page are we on okay we're on the home page so we need to go in and then grab our element identifier from our home.json really simple stuff but really clever what it's doing so next we actually retrieve our page based on our current URL so this was actually something amazing that was done by one of our top engineers so we originally were setting our page ID via the cucumber you know on the home page if I click the login button then I'm on the login page this engineer was like why aren't you doing this dynamically why aren't you setting a rejects in your pages.json and then creating a function that can actually go off grab your URL match it against your pages and tell you what page you're on so we removed that entire need to have that step to tell us what page we're on to determine where we get our actual element mappings that was there for us we created this function that literally goes off and retrieves the page we're on and stores that in the framework and allows us to pull down our mapping element identifiers this was so cool and just another backtracking benefit of that proof of concept getting the team on board those showcases that passion we're bringing and that determination to build something that's never really been built before so to get element function as you can imagine so this was sitting our HTML behavior and this is where the majority of the differentiation would have been eg between a playwright or selenium framework this is where we actually use playwright to get element and as you can see there we're just finding an element otherwise we return now in that function all right there you go so and the header logo should be displayed we can use this single step to assert on any visible or not visible element on our website and if you're thinking wait you could use this on any website you're absolutely right so you could copy and paste this entire framework into your web project right now update that element identifier in the mappings update the pages in the configs any JSON updates we're talking about copy and paste this step and this will work for you as well and that's really a really genius part of this framework so let's finish it out so now our final step for our scenario then the contacts should contain the contacts let's go ahead and step in so notice now we can leverage all our existing html behavior functions to serve an entirely new purpose right so we're using that negate again we're using that get element locator exactly the same function we're using our wait for selector and we've got a new get element text which basically will sit in that html behavior thus giving us new functionality by using playwright functions right but isn't that amazing we've basically got a completely different step one was asserting against an element now we're asserting against text we literally had to add no code we've got a new step but we haven't actually had to add any new code really to our functionality except that get element text so let's go ahead and look at that so our get element text function to retrieve our text pretty standard again it just goes off users playwright.text content and retrieves our text from our element but a bing but a boom the contacts should contain the text contacts up and running working as expected and we've just created an element that could assert on any text in our application or any text on any application in the world as long as we update that mappings in that config folder so let's review some of these key concepts we've just introduced because we've done a bit so we've got Cucumber World so we leverage Cucumber World to enable segregated page host elements mappings for each scenario across our automation suite we introduced Babel for TS to we leveraged Babel to enable Cucumber JS customization while still being able to build our automation framework with TypeScript check we initialized our mappings on runtime we are setting all our pages host and element mappings as objects we can easily match via our Cucumber parameters dynamic pages so we created a function that determines what page we are on actually from our pages .json based on the URL that the framework actually pulls as part of that step we don't even have to tell what page we are on it tells us and then it knows where to draw down those page element attributes based on what the .json is called whether it's home.json or login.json generic steps we created generic steps and reusable Cucumber steps that can be used for the same purpose across both interactions and assertions so you can use these steps across the entire application you can get to a point where you've got hundreds of tests for your web application potentially automated from less than 50 step definitions we have reusable functions so we created generic and reusable functions that can be leveraged across the entire framework and service the functions of many steps so success we have built a framework that is so architecturally sound we can literally substitute out Playwright for Selenium we don't lose any tests we don't lose any step definitions or functions and here's an actual example of both we created a small bare bones react application a few of us got together and built this as part of a hack date it's been amazing actually it's literally a real bare bones react application it's got some of the major components you'd expect to see inside the playground if you've got a test suite you can validate through the entire test application you're really validating the entire framework works because this framework is so generic in architecture and can be copied and pasted from project to project it's actually architecturally sufficient enough that you could create a single framework and you could actually write the features on each project and have a single common framework between multiple projects and that's something we did at TabCorp with a similar approach 3,000 automation tests across 16 web applications running off a single framework that was maintained by a technology team of over 100 at Bunnings this is a new framework we've got it up and running now for Bunnings we've got it up and running for a new system called Hybris so we're in the infamy of getting that common framework going across both applications now this framework architecturally has just gone leaps and bounds and again it goes back to that POC I think when we did the version of it at Tab and that was using Protractor and TypeScript no actually sorry it was using Cucumber Protractor and JavaScript we did it predominantly as a test team with some additional help from developers especially as we worked together to build out the suite but at Bunnings it's been hand in hand with the engineers and we've seen the dividends from that just on the right we've also got the play right version of the framework running as well I decided not to include the Cypress one I just wanted to show you that we did get multiple frameworks so again Cypress, Selenium and play right so yeah on the next slide as well I just wanted to also show you the structure and show you how it's the exact same because that's the thing right it can literally be the exact same you can have the exact same inf features you can have the exact same reporter step definitions setup and support now under that code what's different as we saw there'll be some minor changes in the hooks as we initialize our suite whether it's running on a driver or if it's running on the page you're going to have some minor differences around the world because we're going to have potentially different browsers we use you know in Selenium you're going to have Chrome and Safari you know in play right you're going to have Chromium and WebKit for example the steps themselves exactly the same literally no difference the support functions that's where the real change is right although you've got the exact same code one is going to be calling Selenium functions and one is going to be calling play right functions so key successes just to round this out so we saw increased collaboration we saw significant increases in collaboration across our team using the framework approach that was easy to use and enabled all our team members to be contributors we saw massive improvements in code quality since the code was so minimal and generic we found maintaining code quality through collaboration and pull request review to be very easy and straightforward many tests from minimal code this framework approaches enable us to create hundreds of automation tests across multiple applications with very little code less time coding and more time adding automation tests leveraging across multiple projects we are able to literally copy and paste this framework into a new project then update our hosts and pages and element mapping JSON files for the application and have a fully functional automation framework we can just get started copying and pasting those steps right away key challenges selling the engineers on Cucumber many engineers have had bad experiences with Cucumber in the past you know maybe they've had debates about how it should be used or if BDD is the right way or not what we found was those showcases and that POC solved all those problems the engineers saw that we wanted to use Cucumber for its architectural benefits for its enablement benefits and they got fully onboard and were very supportive and then of course proving the upfront effort is worth it you know as you can see with this framework approach there's a bit of upfront effort right like you've got to build out that kind of you know actually getting Cucumber world working for example like we saw you know we had to overcome some hurdles we had to build out those initial step definitions but of course once you get over that hump once you've paid down that initial debt and you're literally in that low-code no-code scenario where you're basically able to just add Cucumber steps and build out those tests so you might be saying well how can I learn about this framework well fortunately I have gone off and it's been about a hundred plus hours or more developing a course on Udemy and it's a best-selling course which is fantastic the community really seem to love the course it's got a very high rating as well and you can actually learn as we say how to build a world-class automation framework in either Cucumber Playwright or TypeScript or Cucumber Selenium TypeScript so please check it out we offer full support as well so you know if you want to join up we'll invite you straight away to our Slack we're pretty much happy to help you know we love this stuff I'm a tester through and through I love this community I'll be your direct point of contact as your help and guidance for that course as well let's go on that journey together a little bit about testing talks conference so you know I established testing talks conference in 2019 and what it's all about is similar to Selenium Conf bringing amazing people together amazing people from our community working hands-on doing amazing things but on top of that it's also about swag fun games prizes it's shaping up to be a massive event we've got some incredible sponsors on board some that are obviously joined in with this conference as well from browser stack to aptly tools to lambda test and it's going to be a big day the way we think of it is we want it to be a party for our community a day the community a day the community can look forward to to please join us it's going to be an amazing day amazing speakers and so much fun finally let's connect on LinkedIn you know I love to connect with community I'm pretty much available around the clock to have a chat have a yarn have a coffee if you're in Melbourne definitely add me up you know I live and breathe this stuff and I love the community I love testing so go ahead and connect and I'll definitely connect back thanks everyone any questions so we can see some questions in Q&A let's check that out is BDD implemented in the whole team or is just the QA team using it this is a fantastic question so we are using this across the team now so when you think about BDD traditionally you're thinking about the way we define our business requirements you know give an eye when I do that then I see that we don't use BDD in its traditional sense so you won't actually see our test cases read as a story per se we use it for that generic step definition architecture you would have seen throughout this presentation so it's a little bit different but we definitely follow the idea of readable tests just not for the traditional purpose of BDD now the developers are definitely across it because they're across the whole process doing BDD they're using Cucumber for its architectural benefits and easy story writing and the enablement of the team so it's a little bit different no question but enjoyed listening full of energy did you take a breath when I was good question I did take a breath I should have drunk more water but I was too excited to present here today thanks for that my friend what about Dashboard in Playwright is that are you asking if there's a Dashboard equivalent similar to Cypress in Playwright I'm not quite sure I'll speak to both so basically I found Playwright in Spectre really good it's pretty cool it's like this tool that basically with a simple environment variable set will spin up a debugger that you can basically view as the test runs and shows you what elements you're hitting and what it's doing in the HTML I feel like it's still in its early days it's a little bit temperamental like you'll be running a test and it'll just shut down and you'll have to run your test to see it again but it's super beneficial overall and we definitely teach it in the course as well because we see the benefits of that as far as Dashboards in Playwright you know we're using Cucumber so we use the Cucumber HTML reporter which is sort of the go to reporter for Cucumber so we didn't use any sort of Playwright Dashboard per se also it doesn't have a Dashboard like that it doesn't have the equivalent of Cypress either it's really just a test reporter nonetheless really cool stuff though awesome I think that's all the questions okay that's all the question one more if you want I think we're out of time mate up to you okay yep you can take yeah I've got time okay can you compare performance with Selenium and Playwright Playwright came out ahead so if you run our exact framework on our bare bones React application with Playwright and you run the same framework with Selenium running on the exact same architecture like you just saw running on the same browser binary we actually saw about a 10 second improvement overall across the 50 tests when running in Playwright so minor improvement however very similar to be honest I found them both extremely beneficial and that's why I did a course on both of them I think it comes down to personal preference the only thing I will say is Playwright handles long forms a hell of a lot better still than Selenium like I still found with the Selenium framework we had to add a lot of those scroll to elements just to get us over the hump but generally speaking both really great frameworks in Selenium Selenium 4.0 has has come a long way highly rated and I think it's back back it's it's definitely back other stories written in Gherkin so you just have just to implement it in the framework or your adoptions in the scripts unnecessary stories written in Gherkin so when we write our business requirements we work with our BAs to produce them in a sort of a given and when equivalent but they don't break down the steps into generic step definitions like we we actually add them in our tests so we haven't looked at aligning our our tests our tests perfectly with our business requirements yet it's just not something we've we've endeavored to do and I'm not sure if we will I think the process is really working right now and I think the testers and developers are happy to let a little bit of exploration happen when they write their tests rather than just trying to replicate the BA requirements coming through on the cards and thanks Cameron for sharing your experience with us today. Thanks everyone