 We've been talking a lot about testing on Visual Studio Toolbox. In today's episode, we're going to invite Windows desktop apps to that party. Hi, welcome to Visual Studio Toolbox. I'm your host, Robert Green, and joining me today is Timo Tias Margo. Hey, Timonius, how are you? I'm good. Thanks for coming on the show. Thank you. Thanks for having me. So we've done some episodes on testing code, unit testing of code. We had an episode where we looked at some of the unit testing tools in Visual Studio. We're doing a series coming up very shortly on unit testing of code and that's all good, your code. But there's also another part of the application which is your UI. I know that there are frameworks out there for testing mobile UI. I know there's frameworks out there for testing web UI. But what about testing desktop applications? WinForms, WPF, UWP. That's what we're going to talk about today. That's right. We're going to use a tool called Windows Application Driver or WinApp Driver. So you're going to explain to us what that is, and then you're going to decipher that name because Windows Application Driver, it doesn't say to me, oh, this is about testing your UI. It sounds like a driver. That's a good point. Well, let me start with what WinApp Driver is. It's a UI automation tools, tailored for Windows application. As you mentioned, there's UWP, there's WPF, Win32, or WinForms, you name it. Now, talking about the name, why do we adapt the name driver? It's because this tool is based on the web driver, the Selenium web driver that's out there. You actually just mentioned for mobile UI and for web UI, there is actually an open source web standard out there from W3C called Selenium Web Driver. Now, this is actually adapting the same framework. So you have a consistent and uniform way to test your application whether it's a web app or a Windows apps that we're talking about today. Okay. Cool. Yes. Now, one of the things that comes with the benefit is you actually get to get all the benefit from the Selenium web driver framework itself, such as you can pick any test ordering language you want. Okay. You can pick any test runner you want, whether it's JUnit, VS Test, and that should fit well with your CI setup or your build system, and you get to also target different platforms. For example, if you are targeting iOS and Android down the line or now, now you can actually target them all with the same framework. Cool. So that's what it is. Today, I can show you what a typical test ordering flow or experience. Now, as I said, user will have a lot of choice in some of what they want to use, whether the programming language or test ordering language they want to use. But today, we're going to use C-Sharp using Fissile Studio. I believe this is one of the good or the best experience that you can get. Now, we're starting off with an alarm clock test. What it is, it's a test project, assuming if we are a developer of the alarms clock. This is a UI application. You have four views, there's alarm, work clock, there's a timer, and there's a stopwatch. Typically, when you have an application like that, there's going to be some UI scenarios that you want to be able to test. Now, let's just quickly see what we already have it in the test project. For example, if you have this setup, this script looks just like any other MS test project. The only difference would be that this project will have a NuGet package, that this is the only extra thing that we add into a default Fissile Studio test template. Once you actually pick up the Appium.WebDriver NuGet package, it will bring in the dependency with the right version. And does that also bring in Selenium WebDriver? Yes. So Appium is a layer on top of Selenium. It just gives you a little bit extra capability to deal with application-specific, more than web-specific operation. Now, in action, we can actually see if you run this, you can see one thing that I totally forget to bring up is, you need to actually start WinAppDriver.EXE. And where we can get it is, in this GitHub, you will have a release, and this is where you can actually get the MSI. Okay. I pre-installed the MSI, so you should be able to actually find it under. Now, once you run it, then it's good to go. Now back to where we started, there's a little bit set up that the reason why this project is here, I just want to be able to show that all you need to provide is just the URL into the server or driver. So you can actually install it in your VM. Okay. You can install it in any remote machine. And within one test session, you can actually target more than one machine. So this will help you in scenarios. For example, you're writing an instant message application, and you need two devices to talk to each other, you can pretty much drive it this way. And in this sample, you see you need to provide the alarm clock app ID, that's pretty much the executable or the package name, that once you provide it, it will actually launch it and find it for you. So back to, okay, now we're going to start testing it, the scenario. So that's actually launched the app. Yes. There it goes. And it will do some set up, for example, now we're testing the stopwatch. As you can see, it's trying to set up some lap time. Then once you set the lap time, it will gonna start to scroll up and down. We should mention that you are not touching your computer, so this is all happening automatically. Yes. And as you can see, there has been some test that's not going, that's failing, and you can see on the VS test it's kind of showing. And that's also one that's passing. And in this case, if you can see, we not only use click and select, but then we can also give keyboard input. And you can interact with the toggle, and right-clicking and context menu so that you can actually delete the alarm you created too. So this is kind of like what a typical test project that somebody that owns that application would write. Now today, why don't we add one scenario. Can you talk a little bit before we do that about how you set this up? How did you, is there a UI recording capability? Do you have to code all this by hand? So we do actually have a UI recorder. It's currently in beta. We are working on making like more improvement to it, but then if you see this under, there's one release that's a release candidate. It's currently on beta still. It's pre-release. We can have a look at, you guys should actually check it out. Now at the moment, typically, this will help prototyping something really quick. But then once you actually have, if you own the application, you would normally construct the test yourself. Some people even approach it from writing the test first, before actually writing the code. Now that's, I think that's the main scenario that we're going to actually focus today. And I think that's a typical or what our customer is actually doing out there. But I also want to be able to show how easy it is to actually create a test scenario. So one of the tool that we actually use, say we have this pattern, I find that it's a good practice if you have one view per class. So this simple class pretty much makes sure that the application is going to be on the timer view. But we're going to add one scenario if we're in alarm clock, we open it manually. When you're in timer view, why don't we do some scenario like creating a new timer? And for that we need to press on this button. I will use an inspect tool that comes with Windows 10 SDK to tell me, let me close this really quick, to tell me what that element is. Because you want to be able to specify, hey, now let's click on that element. And you can see on the inspect tool, it shows me the automation ID is at timer button. And that works for any desktop app? Yes, whether it's WPF, Win32 and Winforms, provided that you have the UIA, your compliant or your application is accessible. So now that's the at timer button. Once we actually click through, let's say we're going to set countdown timer for five hours. We're going to want to be able to select five there from the out looping selector. And we also want to change the timer name. So let's go back to the code and see how we can actually do that. So first thing that you want to do, you create your test method. We can say at timer. And just like any other MS test, you want to decorate it. You can see it's easy to choose when you have intelligence on. And this is one of the latest thing that Visual Studio team did. It showed up in the test explorer as a new thing. So this supports MS test only, not end-unit, ex-unit or new unit. It does support end-unit and j-unit and et cetera. Okay, got it. So this is just like today with this, that's kind of the thing that we get for free when we adapt any different tool. Cool. With this project, you have an alarm clock session already set up for you. It will actually take care of the launching and closing. You're going to see it along our samples in GitHub, having this kind of skeleton to help you create pretty much a test that you can run individually all together. Now, you can simply say, okay, my session, find me the accessibility ID, because we are trying to actually press the add alarm button back then. So, or add timer button. So once you actually do this, you can actually select, the assumption is we find the element and what do you want to do with it and we want to click it. So, so further along, let me see, within this timer, we also want to pick the, what we call the, is that our looper or something? Yes. Let me try to refresh it really quick. I think that will be, okay, ours, our looping selector, okay. Now, what's interesting is, once you actually find an element, you can actually find an element within it. So if you want to find, within the looping selector, we actually have the item that we actually want to pick. So you can actually do a nested search, like find within me, the one with say five, and click that. Yeah. Well, to save us from the pain from watching me typing, let me show thing that I would actually put them. Okay. Like that. I was actually enjoying watching you typing. Well, you're welcome to come to my office and spend more time. Cause it's gonna be more painful for me watching myself doing typos. But as you can see here, we're gonna try to find a text box too. And unlike the one before, we actually can now say clear and send keys instead of click. What they will do is actually empty the text box and actually type in sample test timer as we see on top. Now, okay. So now we have that. Why don't we actually give it a go? Let me right click. And we can see it in action. So the alarm clock is currently open, but then it doesn't matter because it will actually reuse it if it's open. And if it is not yet, it can actually launch it from the beginning too. But what I'm showing you, like hands off here. Oh, let me actually just, let me try that again. So let me quickly, I'll try that again. What happened? What's happening? Yeah. What happened is it's within a page. Sometimes when you write a UI test, you need to make sure that you actually start off from the right page. That's why you need to have a cleanup code. Oh, got it. It's not smart enough to navigate to the right page. Unless you write a code to do it. Right. Because you said click a particular button which wasn't on that page. So since the app was already up and running, it didn't know what to do. That's correct. Got it. Then it will just fail it. Now, if we try to run this, the step that we just put together, it will actually click the alarm button. There we go. Well, that's good to know. Yeah. Now, you can also see we actually type. The alarm and actually accept it. So now we created the alarm. Cool. Now that's half of the story because now you send all the actions you want to. Yeah, it's half of the story because all you've automated is the thing going and you have to go look at it. Which so you probably want a way of testing the test. That's right. Of figuring out what happened, right? That's correct, yes. Now, there's a few things that you can actually have to look once. Okay, now you're in this view. You have a timer. Now you have a list of timer. And you can see, okay, that's my timer. It will have, the reset button will be enabled because it's going. Otherwise it will be disabled. And say this text, say the sample test timer because we name it so. So why don't we check for those two UI condition because you want to make sure, okay, if I name it this way, I want to make sure they enter that I created half the state. Yeah. Now. And you could potentially, if you could grab the value of that counting down time, you could check to make sure it's not the same as where you started. Yes. Assuming it takes the test. Less than five hours. Yeah. Well, also assuming that the test has run in less than a second or whatever that thing's counting down by. Yes. So depending on the latency. Because you could also put in your test delay. That's correct. Delay, yeah. That's correct. Yes. Okay. No, absolutely. So you could probably do that. Okay. Now, let's have a look into, say you want to be able to get those texts. That's what you meant. So that will show up. Say, under this text. So depending on how you write the application, you want to be, say, timer value text. Okay. That's what we want to see. Let's do, at this point, let's find the, to save us again from typing. Let me just grab a little bit of. Yeah, at this point we can just use the ready to go code. Yeah. But we will do what you just mentioned. Okay. Now, we have, what this code simply is, just take me the list and then give me the last one and that must be the one that I want it. Okay, so verify that we actually had a timer entered. Yep. Some of the thing that we can do, for example, typically when you create a test, you want, you can actually say, I want to make sure this is true. Then you can say certain condition that you expect it to be through. If you have timer entry property, you can actually see, is it displayed, is it enabled or the coordinates is certain way. But now we actually gonna attempt to get the text timer. So it's gonna be in the entry. And I need to see again, what do you call it? Timer value text. Timer value text, here we go. Now, sometimes at the end of it, I still don't know, okay, if I get the element, how can I actually get the value? Is it coming in the text? Is it coming in the name? Now, why don't we start with assert? It's not now the timer text. So, and let's just break there. So you will redo, that's why it's called the test automation, it creates a new timer with the same name. Okay. Now, an element couldn't find, okay, find element by accessibility, the timer value text. This is automation. Is it because it belongs to the sample test timer? Sample test timer. So timer entry, find element. So it actually didn't find it, interesting. So sample test timer is actually one of the entry, the, now there should be two, right? Timer value text. It should actually find, I'm actually surprised. Okay. All right. But for now, actually let me try again really quickly. Would it be session? It will exist in the session, yes. So when you specify something else as a parent, like we use timer entry, they just to narrow down where we actually wanna start again. Typically if you do this, like chances you will find it, it's just like you can't be sure which one. Oh. Yeah. Especially if you have multiple. Okay. And that's just the trade-off. Now we actually have like three different timers, right? But that's okay. And one of the process of actually creating test ordering is like really trial and error. An element couldn't be, okay. So it really doesn't find it. Okay. That's what happens when we go off script. Well, yep, untested scenarios, but that's okay. Now, one of the thing that we can actually test, maybe why don't we have to look at some of the other elements, like the timer entry text and the timer entry set button. Okay. Actually, let me do it again. Now, once we find the, once we added the code before, now we can actually start inspecting what actually we care, what we care about in this scenario. For example, if you have the entry reset button, that's pretty much like if you have like many of this guy, this reset button should be enabled. And we want to make sure that sample test timer should be named sample test timer because when it's set it so. So, why don't we try again and set that? So we want to make sure that it's true, that it is enabled. And the second part of it is like, we want to make sure it's R equal to new timer name, which is sample test timer. Okay, we go. Are there good guidance for how much coverage you want in your UI test? I suppose it's absolutely everything. It's a little bit harder than understanding you have coverage of your code. That's correct. Because in your UI, people can do things that you would never think about. That's right, yes. So that seems to me the tricky part to this. Yes. Well, one good thing about it too is some UI does come the logic, for example, updating those state of the button and renaming, there's actually code that covers it. So by exercising this scenario, it will actually exercise those code and increase the code coverage. Now, that was a really good question because sometimes when we go overboard and we make everything is verified like down to the details, one day when we decided, oh, we want to change the UI, it actually create more work. So sometimes it's the finding the right balance of knowing, hey, this is like the best kind of coverage. We don't have to go all the way through. But the nice thing about the tools is it enables you to do, you can go overboard with it, but for the most part, you want to make sure that, this is the one that matters. And if you test it once, chances, you don't have to test it in different kind of scenario. For example, you rename the timer, something else and you give it like 10 hours now and then you don't want to redo every single UI changes other than the value that you change. Okay. So before we ended, as you can see the test when it's completed, it kind of closed the application too. That's part of the cleanup. And I think it's always a good practice when you actually write a test to clean up after yourself. So when you create a timer, sometimes you do want to make sure that you also clean up after yourself. So the nice thing is there's actually a different way to do it and anything that the user can do on the application, you should be able to do it through WinAppDriver. So in this case, let's just give it a shortcut. We're just gonna give it a delete and enter key sequence because that's the quickest way to actually delete the entry. So keys delete, yep. And then you can also, so besides sending strings and other numbers and things into a text box, you can actually send it some key presses too. And this is how you would be able to do it. So altogether I think if you run it, now it's gonna open it because now it's just being closed. The alarm clock will be open, launch. We don't have any timer, but then we're creating one now. We're selecting five hours, name it. And then now we, once we verify it, we actually delete it. So that's a simple scenario. That's so cool. Yeah. Thank you. That's awesome. Yes. So quick question you had mentioned that you can get the names of things if your accessibility enabled. What is involved with that? That extra work? If I have an app I wrote three or four years ago, how do I know? So typically depending on the UI framework you use, a lot of them... WPF. Yes. So if you use some XAML elements and some WPF standard components, you will very likely end up with some UIA accessibility property on it. Okay. Now it gives you some. For example, we get to see the name and the class name for sure. But I think what's more important is sometimes when you want to write an UI, you want to actually be able to take your element in a certain way. Because if you create, if you rely on a class name, for example, like list item and buttons, it's not unique. So you may end up with a lot and you may actually miss. Now if you actually tag it with Automation ID, you can do your own diligence to make it unique. And at the same time, if you, having this practice, making your application accessible is good for your user too. Right. Because now the narrator can actually traverse through it well. And for the most part, a lot of the application that I've seen are UI compliant. Okay. So if you're talking about like really, really long time ago, there was an application that was supporting an older version of UIA. It was MSAA. They do actually get support a little bit, but then it's not, we don't officially support it. It's just like the limited support that comes for free will be there, but then it's not fully covered and there's no guarantee on that one. Cool. Cool. The other thing is the inspector tool. Yes. You said is a Windows 10 thing? It is. Is there a Windows 7 equivalent? No, there is not. Actually, let me take that back. The inspect tools, I'm not sure. Okay. But then at the moment, this Windows application driver we're targeting Windows 10 and up. Okay. Yes. So the application that we try to enable is the one that actually runs on Windows 10. Okay. Yes. Which of course your WinForms and WPF apps automatically do, but you require the tester to be running Windows 10. In the end, don't care what the apps running on as far as the user is concerned, because you're just running the app on the tester's machine. Yes. So it's more like on the tester's machine, whether it's on the VM or devices, it needs to have WinAppDriver.exe installed. Okay. And that machine at the moment should only be Windows 10. Right. I just want to make clear that if all of the users are on Windows 7, that's fine. You just need the person doing the UI testing to be on Windows 10. That's correct. Yes. Okay. Good. Yeah, precisely. Okay. Excellent. Yes. All right. So, in the end, we can actually showcase the latest thing that we had in place. Okay. Yeah, we've got a couple of minutes. We can go long. Now, one of the exciting thing that we add into this WinAppDriver is to make the strong point of Windows, which is like on desktop, now we have multi-touch, and we also have pen with a lot of capabilities. For example, you can do like with pressure, you can apply pressure with tilt angle and twist. Now, the question comes to us becomes what if you actually want to make an application that has that nice experience, right? Yeah. So, that's the latest thing that we add on the 1.1. Maybe to showcase it, let me just open different samples. All of the samples you can actually find on GitHub. Okay. Including the alarm clock? Yes. Cool. Except the scenario that we just did. That one is not there. Yes. We can possibly add it. So, like this is Paint Session, and we still have the WinAppDriver running, as you can see. So, you run it. Paint is a Win32 classic application. Classic. Yes. So, the fact that we can actually automate it is a testament of like, okay, what we can do on it. Now, it's also kind of nice that Paint supports, oh, let me, didn't get the focus. Okay. So, yes, you can see it's trying to actually put all those different touch points. Yeah. But the application didn't get the focus. Yeah. So, let me quickly, if I actually do it again. So, let's see some scenarios that we can do with Paint and this is a little interesting because what if we can actually draw with multiple touch points together? The payload of sending these actions with the multi-touch and pen comes in a new package that's part of the new standard of W3C actions, WebDriver actions. Okay. And that thing comes really fresh. Some support is not across the different language binding yet, but you can see how you can actually specify different things. Yes. Now, this is just one of the things that you can actually, obviously manually doing that wouldn't be possible. But you can also see different tests that we can set different velocity and different interpolation. So, this is how you can actually automate a lot of the actions on the screen. We do have more samples, especially on the pen and multi-touch too, showcasing some of the new things that doesn't exist out there yet. For example, with pen, I can actually, maybe if I can stop that now, maybe if I can actually open one more sample. Sticky notes. It's a relatively new application, but this sticky notes doesn't take mouse input or touch input to sketch. So, it's only take pen for inking. So, we can't really fake it with any other device, but what I'm trying to showcase, let's just say, if I have draw basic square. Okay. Let's just have this guy. Now, let's just run this one. The idea is being able to apply pressure. And this is part of the new thing that we try to push to the standard too. Like, we understand that more and more in Android, in iOS, the pen experience becomes a lot more mainstream. And now, the standard hasn't specified much about how we can actually specify it. And this is something that we contribute as Microsoft into the standard. Doesn't look like it ran. Oops, you're right. Oh, I see. Okay. So, this is just a basic without any pressure on it. We'll try to create a new sticky notes and then try to perform something else. Now, you can see the thickness of the box below showing like, okay, now we apply more pressure to it. And the last one just for fun, our pen has an eraser and this showcase how you can actually erase the stroke on top. Cool. So, that's what I have so far today. Awesome. That is so cool. All right. So, what we've seen is how you can now add UI testing to your existing desktop apps. WinForms, C++, Win32. I left that out the first time, not intentionally. WPF, UWP. This is amazingly cool stuff. So, people can go get it from the GitHub. Yes. It's open source. They are invited to make contributions if they have something to add, right? So, some part of our code is open source. Okay. Especially the samples and the test and the UI recorder tools. Okay. Not everything is open source yet. Fair enough. But do put in your feedback and issues we look at this regularly and we want to hear what your experience is and how you would use it. All right. Cool. Thanks so much. Thank you. Hope you enjoyed that and we will see you next time on Visual Studio Toolbox.