 Next, I'd like to welcome Joe Lust with one of our partners, Mabel. Mabel focuses on intelligent testing automation. Application security testing isn't the only type of testing that software factories want to shift left, and Joe is going to show you how quality testing can also shift left. By using a review app, kind of like what we do with dynamic application security testing, Mabel has used GitLab to shift quality testing earlier, delivering results to the developer before the code ever leaves their hands. We invite you to engage with Joe in the chat anytime during his presentation or after. Thanks for joining our session today. We'll be talking about ubiquitous quality and how you can achieve that with continuous testing. I'm Joe Lust, an engineer at Mabel, where we build testing software solutions. I've been building for the cloud for the last 10 years, for the web for about 20, and if you'd like to keep up with me, check me out at Lust Coder on Twitter. But enough about me, what about our talk? We're going to talk about three ideas. One, I'm going to convince you that you need continuous testing. Two, I'm going to convince you that you can achieve that on GitLab, and three, I'm going to show you how to do that. Let's jump into it. So continuous testing, what is it? Continuous testing is testing at all levels of your code life cycle, not just testing the code that went out to production, not just testing in QA or in your CI CD pipeline, but testing at every commit in every merge request. Continuous testing is pervasive testing, and as we chase the ideal continuous testing, we have to understand why we haven't gotten there yet. To understand that, we need to look at the pre DevOps era. Most of you here today, you're DevOps experts, but you can remember the way things used to be. Before DevOps, we actually didn't test that often. We might have worked at companies where testing occurred monthly or quarterly, and oftentimes, this testing was paper-based. It was manual. Somebody followed the script, and this meant we didn't really get a lot of coverage because it was tedious and costly to do that testing. And the places we did that testing, those were sparse. Maybe we had a UAT environment or a QA environment, but there weren't many of them, and we had to be careful not to break them. And the result of all this testing was often reams of log output. Maybe we've talked with many testing professionals. I remember one call with Sarah, and Sarah's story was about how her typical Monday was getting to work, getting her coffee, opening Jenkins, and just looking at logs for several hours to figure out if the tests were good or bad. That's the way in the pre DevOps days we did testing. But now, in the post DevOps days, harnessing the skills you have with serverless in the cloud, we can do continuous testing. We can test instead of every quarter or every month, we can test every commit that's thousands of times a month. And we can use fancy new automation tools, self-healing tests, end-to-end coverage, all of the cutting edge tools that are now available. And these can harness, as I said, the cloud and the serverless tools in your toolbox to give us unlimited test environments. And finally, with the new insights we get from AI and ML, we can boil down Sarah's reams of server log outputs and Jenkins and give you the developer just the correct insight you need to fix that bug. As we move into that post DevOps period of testing, we move closer to the source of the bug. And I admittedly must say people like myself, developers, we are the source of bugs. And so we need to shift left. We need to get closer to the development life cycle to the source of those bugs. We don't want to catch the bugs out in production. We want to catch them where they're made. We catch them before the merge. We get the greatest savings. And I won't get into the details, but as you probably have already heard, there's tons of research that shows that further towards production, you get many fold increases in the cost. And now when you and your developers are no longer fighting fires, you can actually get more work done. You've probably come to work on a Monday. You'd have that list of things you're going to achieve and you don't because of all the fires you have to fight. Well, we can get rid of this whole section of fires. And finally, your QA team, when they're no longer chasing the silly bugs that I created, those regressions that have been broken four times in a row, because we've got better automation that catches them at the source at each commit. They can actually focus on the needy, hard to solve, hard to test business logic problems that you've got. All of this because we're shifting as far left as possible. So that was continuous testing. How can you implement it at GitLab? I'll show you. First, we're really going to need two things. We're going to need a way to scale our application under test. We've got to deploy that code somewhere to test it. And we have hundreds or thousands of these applications to test for each commit you made. Then we need to use something highly scalable, for example, like GitLab or VWAPs. And second, once we've got our test code out there, we need a highly scalable test running solution. Now you can go with a SAS solution like Mabel, which we much prefer. Or if you feel very proficient using tools like Kubernetes, containerizing browsers, things like that, you can also use these modern tools like Puppeteer, and you can combine them in your own cloud skills, and you can self host. Regardless, you still need a place to scale out and run all of those tests. So a quick word. What's Mabel? Well, Mabel is an intelligent test automation platform. Our core tenant is we want to make everything easy, make offering tests easy. So we use a Chrome extension, when you can click around and you've got your test and now it can be replayed on many different browsers and very easily retrained, maintained, auto healed. Many features come from that. Second, you make a unified platform because testing touches many areas. It's not just the tests that you use developer right, but it's your CI CD pipeline, your source control management, your test tracking tools, your bugs and JIRA. All of that can be brought together in one voice, all its touch points with Mabel. And when you run those tests, you get tons of output. Honestly, Mabel, when we run one of your tests, we actually instrument everything and get almost gigabytes of data. Well, I won't get into the details, but just check it out. You'll see it's all there. And finally, we want you to create customer centric tests. We're not just loading a silly little react button and clicking on it in a test harness. We're actually running the entire application, hosted on a server, going through your auth, clicking on it in a real browser. That means that the button is occluded or invisible or some crazy CSS is happening, you get the real world results. That's what you can do with Mabel. Now, you mentioned the second piece. You need a place to host that code, the code under test. We believe the solution for this at scale of preview environments. What is a preview environment? It's a on demand, short-lived environment. That means we're going to make it, use it, discard it. That's why we got this no recycling icon here. Because you might have worked at some original organizations that I have in the past, where the QA environment was sacrosanct. And if you broke it, 10 people told you broke it. Well, with on demand preview environments, everybody gets their own environment. Thousands of environments, doesn't really matter how many you have. Finally, I would just note, if you haven't heard of preview environments, you might have heard them as your view apps, ephemeral apps, preview apps, there are many names, but they all mean the same thing, that place that you're testing your code. Let's look at an example of a real world preview environment in MergeQuest and GitLab. Here we can see that our famous 10x developer, she's been busy, and she's got MergeQuest we need to review really quickly. So let's say it's really important, and it's for a major customer, need to do an ASAP. So we can see here that it's only CSS. So what could go wrong? I personally, like you, I'm a DevOps professional. I don't really know that much about CSS3. So I think I would be inclined, given how important this customer is to just go down here and click approve. But what if there was a better way? What if I could really understand what these few lines of CSS really meant? What if I could show, rather than be told what it is? Well, you can do that in GitLab using review apps. So here, it's the same MergeQuest, but there is a review app here. And we've been told it's been deployed to review docs, this URL. And we can actually click the button right here to go and look at that app. So let's do that. Oh, my goodness. That, that is not right. Now, you might not be familiar with the Mabel homepage, but we are not a candy store. We don't sell candy canes. So this, this is totally unexpected. And we might not have caught this with a typical Selenium test. And if somebody didn't go in and actually look at it, we probably wouldn't find out until our customers were tweeting pictures of it to our detriment. So if we went back, and we gave everybody this button, we would democratize and enable testing across our organization, because you would not have to understand CSS 3. You could actually just go in and experience the application as a user would. And that's something that almost everybody can do. So with review apps, and these sorts of buttons, we really empower everybody in your organization to contribute to swashing bucks. So since you're DevOps experts, you're probably thinking, there are some challenges here. How are we actually going to operationalize all this? For example, you might be saying, all these servers, they're going to cost me a ton of money. And all the networking, the VPC, the firewall rules, and all the DNS entries and domain changes I need to make, this is going to cost me a ton of money and time. However, if we use your now post DevOps era cloud skills, we can use serverless to get rid of the servers. There isn't a server problem. And the networking problem, let's just outsource that as serverless solution, it can take care of that for us. When we push that lambda out, Amazon will route everything for you. What about all those domains and how do we set up DNS? Don't worry, we can use path routing or a wildcard. Now you don't need to make those, you know, dangerous changes to your DNS entries. And finally, cost isn't really a concern, because these new serverless solutions, they all scale to zero. They cost micro cents per request. It's a new way of thinking of things. But basically, it means that cost is no longer a challenge. So how do we go about making such a serverless preview environment? Well, I want to show you a simple case for the typical single page application we see today, or SPA, those are the sorts of applications you probably have in your organization, things like React, or Angular, extension JS. All you need to do is take the static content, mostly HTML and JavaScript, and drop it in a bucket. As we see here with these logos, it could be in any one of these cloud vendors, it could be in the buckets that you have an S3 and Google Cloud storage, the blob storage that you get at Azure. And finally, you might have some dynamic pieces of your application, we can drop those in a container, and we can run them on, for example, AWS Lambda, or in cloud functions, Google, or in Azure functions. Now the dynamic parts and the static parts are hosted, but they're all on highly scalable solutions. For example, at Mabel, we've gone in and looked and seen places where Google Cloud function might have scaled up to 25,000 parallel instances. But it's not a problem. It doesn't cost much and just scales. It's not something that you and your DevOps team has to worry about. So let me give you a detailed example of preview environments that we use here at Mabel. So every time we build one of our apps at Mabel, we take those HTML and JavaScript outputs that React compiles for us, we go and put them in a Google Cloud storage bucket. And that sits behind a CDN, a vanity domain name. So all that routing and networking is taken care of for us. We take the dynamic pieces, maybe some path rewriting that has to happen. We place that in Cloud Run, which is a managed container runtime, similar to AWS Fargate. And now we just keep scaling. Every commit gets deployed. In the last quarter, so we've got 1000 preview environments, for about $1, really is that easy. And you might be wondering the details of how do we route all of those different application versions? Well, very simply, we can take our base URL, we can use path routing. So we give it a name, the pet store app, and then the hash from the commit you made. And now that will route to that specific version. And that domain can have many thousands of different versions on it. Similarly, you can use more complex mechanisms like wildcard domains, we've got an article here for you. We actually go into the deep details. We have code examples of doing this in Amazon and doing this in Google Cloud. Really is that easy. So now you've got preview environments, you've made them scalable, but we still need to make them work for you. As we mentioned earlier, we need to take those tests, those complex end end tests you've got that really test the entire swath of your application. We need to point them at your preview environment, each of those commits. So we can get that instant feedback back to the developer who's making that change. So she can know that she needs to pause and fix that problem before it gets any further. Further, we need to have that in your merge request, as we showed in that example, to democratize that testing across your organization so that anybody in your company can now go in and see what the change looks like. They can be shown, not told what it looks like. And that really enables people across the board to contribute. And finally, you can use some tools you've got in GitLab Pipelines, for example, merge trains to further take this to the limit. You can take main before you merge in, and you can deploy that to another environment. So you can be very sure when you merge to main or master that your tests, your code are all functional. Here, for example, we have what Mabelbot does for us. And every time we make that new commit, bam, we get a link that says, here's the preview environment, you can come and check things out. So you've gone through what this looks like and how you build it. Let me show you brass tacks of building preview environments and scaling them out in GitLab. So in GitLab, you're probably using CI CD pipelines. And there's really two components you need to use here. One, you need a review app. And the review app is that placeholder that tells the pipeline that you have deployed something somewhere. It gives it a name. It's the placeholder for that application that you just hosted. And two, we need to use the dynamic environments. Once you've deployed that out there, that concept of the dynamic environment is going to let downstream consumers in your pipeline use that application you just made. It's going to let them test against it, view it, do anything else they might need to. And this is by passing different environments in the URLs that are passed into the environment variables. Excuse me. So let me show you the YAML because as you all know, we are these days YAML programmers. I went to school for programming, but YAML is where it's at these days. So let's break this down. Here we have a stage in the pipeline. And we're just using Node.js because this is going to be a react app. And so we use the base node image. This is deployment stage. And now we're just going to run the build command. We're going to make the artifact that we would deploy for a single page application. Now we're just going to use gsutil because in this case we're going to Google Cloud Storage. And we're going to say make these files public from our build directory and go and drop them into our bucket. And we're going to call it app name, whatever your application's name is, and we're going to use the commit shot as the unique identifier board. And here's the magic. We're going to define an environment. We're going to give it a name. So now our environment is preview slash our shot. And we're going to tell it where that URL goes to. And now everywhere downstream from here can use the environment variable to get the environment. That's a mouthful. And they can test whatever they need to by hitting this URL. And finally, what do we have here? Well, we only want to do this for merge requests and for branches because that's where we're giving people that instant feedback. We're going to exclude me because you're probably going to use that in a different way. You're probably going to have other tools to take care of that. So that in a nutshell is the YAML. Now, one thing we're announcing today is our Gateway app pipeline integration. We want to make it even easier for Mabel customers on Gateway app to use us as that SAS provider to scale out the testing phase. And what we're going to want to do is run all of our intelligent tests on the Mabel cloud, which is going to help with scaling because maybe you've got one test you recorded in Chrome, but now you want to run it in Chrome and Firefox and Safari and Internet Explorer. And we want to parameterize these. So maybe there are 10 different permutations. So 10 times four, that's 40 tests. So we're going to instantly parallelize and launch all those containerized browsers right when you made that commit. We're going to get all the answers and then we're going to tell you if this passed or didn't pass right there, right away for you. So you don't have to worry about all the operational headaches that cause. And that test status update is going to pop right into your merge request as we see here, including links to the rich test outputs. So you can rapidly drill down, say, why did it break? Okay, here's the trace from Chrome's memory. I see the exact problem. So I don't have to go and make my own test harness. I can now change that line of code. I barely broke my workflow. I stayed in my flow state. Now, when I mentioned the live results, you can go view. This is an example of what you'd get in the Mabel app. If you're using that as your SAS provider, where we actually see all the deployments, how well they're running, and you can actually click on any of these runs, drill deeper, get the screenshots. As I said, Chrome traces, you can dump the HTML from the page. You can look at the actual network traffic that went back and forth. Many, many different outputs in there. So you don't have to go and try to find a copy of Internet Explorer 11 and figure out what went wrong. Let me show you again in the YAML how this would work. Here we've got a build stage. It's the Mabel test running stage, and we're just pulling in the pre-made Mabel Docker image. And in our test image here, we're going to pass this argument line. And all we're saying is A for the application ID. So this would be the ID of the application you want to test. And then we're passing in the URL. And here's that tongue twister environment, environment URL I mentioned earlier. And this is just being pulled in because we created a dynamic environment for our view app. So this would be passed down through our pipeline subsequent stages. And finally, we're using the built-in line to run the script execute Mabel tests, which is actually going to call Mabel, go out to our API, launch, as I mentioned, those 40 parallel browsers, get all the results and it's going to emit a good or bad zero over non-zero exit code. And it's going to do this for all of your merge requests. So what does this let you do? Well, it's going to let you not worry about how you scale those tests. Now, since you're all DevOps experts, we've got even more treats for you. If you like COIs, we've got the Mabel COI. Once you roll up your sleeves and dive right in, anything you could do on our UI, excuse me, most things you could do on our UI, creating, listing, editing a lot of our entities, you can do that right here with a single line of bash so you can stay in your most productive state, the command line, and tie this into all of your bash scripts so you can easily insert it into your other pipeline workflows. And using the COIs, you can see here, you can actually get the real-time test outputs from those tests you're running. Of course, you might be even more comfortable with the curl, some people, so we actually have an API. If you don't want to use the COI and you don't want to use pre-built integration, you can just hit the API directly for test results and creating deployments, all those sorts of things. So let's zoom out for a moment. Our core takeaway here is that you, as DevOps professionals, are critical enabler in your organization to bring continuous testing to the forefront. You can use your skills with serverless and with cloud to shift as far left as possible, finding those bugs in your organization and empowering many people that couldn't contribute before to contribute to the quality and the pace at which you are shipping product. If you've got any more questions, you can ask them below. I'll be listening during this talk or you can see our references we've got here. We've got some blog posts that give you the details of how you can set up serverless environments on other vendors like Amazon or GCP. We've got this slide deck and if you'd like to learn more about our integration, it's right here. And finally, Mabel is actually free as in no credit card. Go kick the tires, run a bunch of tests. You can do a free trial there. If you'd like to keep up with us, you can check Mabel HQ or West Coder on Twitter. Thanks a lot.