 Hello and welcome to OpenJS Foundation's World 2021 Talk on UI Automation with WebDriver IO and Cucumber, a robust modular BTT framework. I'm Olga Smullyar representing InterSystems. UI Automation has historically been regarded as a brittle undertaking, requiring considerable effort to safeguard against unreliable results and flakiness. At the same time, there's been a desire to shift various automation suites leftward in a DevOps or QA pipeline. This in turn increases pressure on the automation suites to be more reliable and more robust. WebDriver IO has emerged as the leading JavaScript-based platform for functional and UI testing. WebDriver IO is highly extensible and supports an array of protocols and services. It also addresses some of the instability inherent to UI automation intrinsically via built-in strategies such as its automatic weight, locator weight strategy, and other techniques. However, harnessing the full power of any UI automation tool requires the use of additional strategies to build a framework on top of the interaction layer provided by its APIs. We're going to talk about this in the context of the UI automation framework we use at InterSystems. For a bit of background, InterSystems is a leading provider of data technology across a range of sectors. Some of our main products are InterSystems Iris and Iris for Help, which are data platforms, and Health Share and Track Care, which are product suites specific to the healthcare industry. The Quality Development Organization consists of about 70 people. The data platform's division runs a little over 5,000 Selenium tests and many tens of thousands of other backend tests. And the Health Share division, which uses an agile methodology, consists of nine scrum teams running about 700 Selenium tests currently using Selenium Grid and Jenkins. We also have a Selenium Community of Practice, which meets biweekly across these various scrum teams and divisions, and hosts presentations, roundtables, workshops, and code reviews. It has also been instrumental in extending the framework which we're going to look at. Some of the framework features and strategies we'd like to share include separation of concerns across a range of levels. The use of BDD principles, which allow, among other things, streamlined test data for input and validation, and a number of guardrails that we've put in place to address the reliability aspects mentioned earlier, including implicit constructor-based page validation, stateful navigation, and static typing of various entities. Finally, we'll look at some utilities that we've developed for efficient data input and validation, including get and set universal for type-agnostic element interaction, get and set properties for bulk population of inputs, and a custom table validator. Separation of concerns is implemented via modularization and object-oriented principles, which confer the obvious advantages of maintainability, reusability, flexibility, and readability. The approach we take is to maintain a separation between page-level document object model and business logic, between navigation and validation, between input data and functionality, and between code expertise and business domain knowledge. This is a schematic of the structure of our UI automation's architecture. Because this slide is a bit busy, we'll go through it layer by layer. At the bottom is the open-source layer, including WebDriver.io, of course, and a number of services, including TypeScript, Cucumber, and the Allure Reporter, which another breakout session in this conference will be addressing also. Next is the UI map layer, which contains locator maps that resolve elements based on selectors and give them names. Additionally, we define custom commands to extend WebDriver.io to add logging and optional parameters, as well as new commands, such as get and set universal, get and set properties, set checkbox and radio, and many others. We also define common utilities, including page stack and process table, and other commonly used functions. These rectangles shaded in blue are part of a common repository that teams can map their code to to take advantage of the latest developments in common code, whereas the rest of the framework is project or team specific. The page object layer facilitates logicless interaction with elements. Page object classes extend locator maps and define custom atomic methods on elements if needed. They also include page validations in their constructor for implicit validation. They can also refer to common widgets and panels. The business layer contains the application's navigation and validation logic and is implemented via so-called business UI service classes. These extend page object classes and implement workflows within and across pages. They also model the relationships among pages and the flow within the application for one thing by having their navigation methods return an instance of the landing page, meaning anytime an action results in navigation to a new page, the method that performs the action instantiates and returns an instance of the child page class. And the shaded rectangle is the class hierarchy that comprises the page object as a given page object and represents a page or panel in the application. Depending on the complexity of the application, this can be simplified and condensed to just one or two classes or files per page if there are not many elements per page or the business logic is not very complex. But we find this separation quite useful, especially in order to provide flexibility for multiple teams testing different types of applications. Business UI services reference business option type definitions, which specify the available parameters for their methods. And likewise, there are business option classes that specify defaults and construct the custom cucumber world object. The cucumber world object is designed to maintain state among steps in a scenario. It contains business options and other data. Finally, the test layer, which is powered by cucumber, consists of feature files and step definitions. Feature files use those so-called Gherkin syntax to define test scenarios and scenario outlines and to specify test data. Feature files call steps defined in step definition files, which map steps to UI service methods. They can be used to chain navigation and validation commands for readability. The test layer is isolated from dependencies on downstream pages by the mechanism described here, where business UI services return instances of pages that they navigate to so that rather than having to import and depend on every page a test interacts with, it usually only has to import the login page and all the rest are organically generated during navigation of the workflow. Next, we'll take a quick look at our use of BDD principles, which is basically about cucumber. And what is cucumber? It's a behavior-driven development test framework, which defines a test layer on top of a test framework written in other languages for which cucumber provides findings. It provides a business-facing language, which gets mapped to code via step definitions. If you look at cucumber's website, they build themselves as an open source tool for executable specifications that merges specification and test documentation into one cohesive whole. So this describes an ideal of BDD development where end-to-end traceability from requirements to code is implemented using GERKIN, but it can also be used to simply write tests and more in the spirit of test-driven development than write the code, both application and test code, to make the test pass. And this is more what InterSystem strives for. This is a simple example of a feature file with standard sections. The feature provides the name of the feature and a free form description. The background section is optional and gets executed once before each scenario, whereas scenario is basically a test. It's one set of steps with no examples table, whereas a scenario outline is a set of steps that gets repeated for each row of an examples table, like a for loop. The keywords given when, then, and and are used to provide logical flow for the scenarios. Additionally, most IDEs have plugins that support cucumber with step text auto completion and very useful keyboard shortcuts for formatting tables. Individual steps can also be configured to accept a table as an input parameter. This is a capability of cucumber that we find particularly useful for streamlining and organizing our data visually. When steps accept a data table, they are passed an object with methods that can be used to access the data. Depending on the type of the format in which the step data table is presented, along with the method that's used to parse it, different data structures are generated. The most common one, table dot rows hash, simply takes two columns interpreted as keys and values to create an object. Similarly, table dot hashes creates an array of objects corresponding to the rows of a table. And we'll come back to this one when looking at our process table, validation utility. Scalar parameters can also be embedded in line in the step definition. Next, we'll take a look at some of our guardrails. Firstly, implicit page validation within page constructors allows page validation on navigation. This implicitly enforces that correct navigation has occurred before continuing with the test. This is an example of a constructor that calls a validation method, which can validate any aspects of a page. And in this case, it's just looking for the title. Context.page is a global that constructors also take care of that keeps track of the current page for stateful navigation. By this, we mean that step definitions and methods can check whether the current page class matches where they expect to start. And if not, they can navigate to it. Additionally, all UI service constructors contain a param, at least one parameter called validate. And this allows UI, the classes to be imported into other files without the compiler trying to execute page validation while loading the files, which would cause an error. Instead, the validation only gets executed when the constructor is invoked by the page getting instantiated by another page's navigation method that passes the validate parameter as true. Additionally, divergent paths for expected landing pages can be parametrized using multiple values or constructor parameters as shown here, where a login page can expect to land on the home page, if given valid credentials, or to land on an error, and the appropriate validation pathways can be controlled using these parameters. In addition to the current page, we also maintain a page stack, which we refer to as stack p, an array that keeps track of the page navigation history. And this was devised to support backwards navigation while avoiding circular dependencies. These would arise if each forward navigation action imports and constructs an instance of its child landing page, then backward navigation would require the child page to import and instantiate its parent, and this would of course cause an error. Instead, we devise these utility methods, push p, pushes the current page onto the page stack when it's called in the page constructor, it sets context.page to the current page class, and prints out the page stack as seen here on the right. Poppip pops the last page off the page stack when navigating backwards from a page, for instance by closing a modal, clicking a browser back button or using breadcrumbs. This can also optionally validate that we've landed back on the parent page, and sets context.page to the correct page. It also prints the stack. So on the right here we have a snippet of our console output showing the custom commands logging that logs the action and selector for each interaction or weight command, and the page stack on each navigation. And taking together these provide a great snapshot of what the test was doing if an error occurs during development or troubleshooting. We mentioned that we use TypeScript for ensuring type safety across the framework, but in particular we use it to define business options, which correspond to the properties applicable to a specific functional area or workflow or page. In this example here, role business options represents all the possible input values that might be provided when creating or editing a role in an admin application, whereas role validation options list the validation touch points that we might want to harvest while navigating through the workflow and then check later to see if they match expected values. And this is one way in which we separate the validation and navigation logic. The type definitions are then used to build the context interface, which gets used as the type of Cucumber World's context property. And multiple type definition files can extend the context interface. Next, business option files are used to construct the Cucumber World object, whereby their constructor initializes the business options with defaults were applicable, and these can get populated or overridden within step definitions by data passed in by the feature files. The classes get instantiated and merged into Cucumber World Object's context property during world construction for each scenario. For a closer look at the Cucumber World object, this is provided to maintain context between and among steps within a scenario. This is needed because, for instance, if one step appends a timestamp to a unique entity that's being created, other steps will be able to have access to that same timestamped value using the World object. This gets regenerated automatically by Cucumber using the World constructor, which can be overridden or extended as follows. In this example, we have our business option classes getting instantiated, merged, and populated into the context property. And the custom World constructor can be tied to step definitions, as shown here. The World object is exposed within Cucumber Step definitions, as the this keyword, and also within certain hooks as a parameter, but it's not accessible anywhere else in the code, such as the page object hierarchy. So individual business options sets get passed by step definitions to UI service methods, which provides an additional layer of access control and avoids the use of globals. Another result of this is that step definitions must be defined using function syntax, not arrow syntax, in order to use Cucumber World because arrow functions don't have this keyword. So finally, we'll look at some of the utilities that we use for efficient data input and validation. Get and set universal extend WebDriverIO element, and they first determine the element type using tag name and type, and then they call the appropriate command to get or set the value based on the element's tag name and type. And this is useful because it precludes the need for page objects to define accessors for every element based on the element type. They're also harnessed in these two important custom commands, get and set properties. Set properties calls setUniversal iterating over an item array of properties from which it constructs the pages locators and the corresponding business option value for each. So this notation takes advantage of JavaScript's accessor properties and accessor notation, which of course is what makes this construct possible because the language that doesn't allow accessing properties using variables would not be able to perform this type of interpolation. So in order to make this work, names of locator definitions must match the business option names. So an example of how we would use set properties to fill in this input form with one command, we would first define the locators whose names have to match the business options, and my UI service class would define an edit array listing all the business options. And then in order to use set properties, we would pass an instance of the current page, which gives access to the locators along with desired business option values, user options or validation options here, and the array of locator and option names to iterate over. So then get and set properties call, get or set universal for each item in the property array, referencing the corresponding locator and business option value if provided. And this is why we mentioned in the beginning that page objects only define accessors if they're needed. So if an accessor needs some kind of custom handling, such as first expanding a panel or switching a tab on a page or any kind of custom weight or other custom parameter, then we define an accessor in the page object and otherwise we use the UI services to simply set groups of standard input elements all at once and similarly for the getters. Finally, the process table utility is used to validate data provided in cucumber step data tables and it can parse an HTML table given only the table selector. It processes a given subset of rows and columns and returns two arrays of objects, text and elements from the table both corresponding exactly to cucumber's hashed input table format and the resulting data structure. What this allows us to do is to define a validation step like the one shown here and the expected data for row numbers X, Y, Z is retrieved and then as the input step, we simply indicate the portion of the table, the columns and rows that we want to validate and then use process table to compare the actual table contents to the expected subset. The process table also accepts faux selectors to accommodate non-standard table structure. Finally, we mentioned that we use lower reports so just a quick plug for them as being intuitive and interactable and allowing very transparent troubleshooting by allowing you to drill down to scenarios and steps in order to view stack traces and screenshots for failures. They also provide a variety of reports and metrics and integrate with Jenkins and other build managers. Here's an example of a top-level sweet view of an allure report with each row here corresponding to a feature file and failures classified into product defects or assertion failures and test defects or broken tests. You can drill into these colored bars to get to the scenario view which lists the names of scenarios in a feature file on the left hand column and if you select one of them it shows the steps of the scenario in the right hand panel along with screenshots and stack traces for any failed steps. Some takeaways that we hope you'll gather from this talk are one the importance of separation of concerns including the use of BDD principles, use of guardrails to model and enforce application logic, common utilities for efficiency and flexibility, and the use of tools for debugging and analysis that allow rich logging and reporting. Thank you very much for your time.