 Okay, so it's time to say thanks everybody for coming. I'm Andy Farmer and today I'm going to be talking to you about a way we can make it easier for our therapists to interact with the application by the application giving a helping hand to the best. When we're working with our tests, quite often we'll find that certain things have to change fairly regularly, so it might be that UI changes or it might be that something we have to do changes to the actual method, the task that we have to perform and change. And what that can lead to is a change happening in the application and all the tests. That's one of the most common complaints about testing information is that the tests are brittle. But it's not really that the tests are brittle, it's more that there's not enough communication between the people who are doing the testing and the people who are writing the application. How many of you here are working as testers? And how many of you here are working as developers? And how many of you are in a separate test team? So that's pretty good, not as many of you in a separate test team. Are the rest of you in an agile team or a team where you have the developers and testers working together? Is that the case? Yeah? Okay. So when you're working together in this team, do you all work together in the three amigos? So you have an analyst and developer and a test worker together? All of you? Excellent. Okay. And when you're doing that, are you helping to write the application code and are the developers helping to write the test code? No? Okay. So then where it's called the three amigos, are you all sharing the same thing or do you have different things? Because the three amigos happen. You have to stand a little bit farther away from the speaker so that you won't get a feedback. Okay. I have to take the back of the screen. You can use the podium, that should be good enough. Yeah. Okay. But then I'll stand where I'm at. So how many people here have heard of mob programming? A few of you. So the next question is going to be how many of you are doing it, but if only a few of you have heard of it, then I'm guessing not many of you. One of the most effective ways you've been working, and this is my first experience I'm not saying everyone's going to be, is that I can have three people with the analyst, the old product owner and the developer and the test worker working around the same thing, and you're only working on one aspect at a time. You can take it through step by step. Obviously a lot of organizations are able to do that. So some of the things that I take for granted might not apply to your situation. One of the common reasons for, as I said, for test code changing is because the application codes change. So the tests appear to be brittle, but it's really just that there's not communication. Somebody's made a change. They haven't updated it in the tests, and then the tests fail. And sometimes the first thing they know about that is in the consumer integration framework. The question should be why should the code change? So single responsibility says that a class should only have one reason to change. At the moment we've got two reasons to change. One is that the behavior is changed and we want to do something different. And another is that the application is changed just to UI. We really only want to come and change our test code when what we're actually doing, the tasks that we're performing are changing. If the UI is changing, we really want the isolation from that. Now there's some things we can do around that, and I'm sure a lot of you have experienced this. How many people have had experience with using X path to select their elements? And how many of you have had problems with X path selecting your elements? It's about the same number of hands. So the common practice is to do something different. Most people would say to use ID, which is great. But we still have to be fairly intimate with how our application is doing something. So if we have a select box that we reference by ID and we want to change that to a radio button or vice versa, we still have to go and change the test code because the way we're interacting with that has changed as well. So let's look at some code and some examples. So I've got a very, very simple application here. Do you see that? I'll zoom in. So we're working for a company that does dog grooming, dog walking, dog sitting, and you've got these three services, and at the moment you can choose by a tech box what service you want to select. So that was the easiest thing to do. This was the easiest thing to do. So we've got some code to interact with that. This will look familiar. How many people here are using page objects? Okay. And is anyone using Selenium directly? Yes, one. So if we were using a page object, we'd have a page object something like this. For this example, we're just going to talk about how we interact with it directly. So can everyone see that? Is the text big enough? But this is the version we're looking at here, the UI version one. So we hit the link and we're looking for a CSR selector with the ID of services text and we're looking for the input filter in that. Does that look familiar? Is that how many people here would work with that? And then we send in the key of the service that we want. And that should run. Yeah, that works. That's straightforward. But now somebody's made a change to the UI because that's what they do. So we're now at version two of the UI and that's the radio button. Because it's only three choices, that's what we want. So we have to change the test code, don't we? Okay. So now we've got a new ID. It's a radio and now we have to select by the input value that we're looking for the radio button with the value that we're after and then click on it. So that's all relatively straightforward. Yeah. And if we were at version three of the UI and the user experience person said, no, we don't like that. It looks like we want to go for a select box. So there's our select box. Do we have to change our code again? Yes, we do. Why do we have to change our code again? This time it's because we know it's a select box and we have to do select and then once we've got the select we can do select by the input value. What's changed about what we're actually performing in this test? Yeah, the locator, but the actual, what we're trying to change, what's changed? The user actions are changed. So that's in terms of how we do what we're doing, but what's changed in terms of what we're doing? So if I'm a user and I'm phoning up the dog company and I want to book in a service, what's changed? So the person on me and the telephone, is it any different to them? No. So the actual thing that we're trying to do hasn't changed. So we can take a step towards that. We can take a step towards making that more explicit by adding in some metadata to our application. So we have a look at the code. So we actually, I'm not sure if this is going to work. You see here, I've got this data question here, it's a service acquired. So this is sort of the application explaining what the information it wants from the user is. So if we think of it as a conversation between the application and the person using it, the application is saying, I need to know what service you want and how you implement it is up to you. In this case, we're doing it with the input element. Does that make sense? So if we then look at the same three examples again, version one, version two and version three, the CSS selector now looks very similar between the three of them. We've asked for the question. Instead of looking for it by ID, we've looked for a question for the thing that we're interested in. Obviously, we've still got the sub selector, the input and select. But it's essentially the same question three times over. It's just infinitely three different ways. All three of these work again. If I change the URL here, will this CSS selector work? Will this test work? If I use a different version of UI? Why not? That's not fair. It's trying to do the same thing, isn't it? Why can't I just change it and use a different version of UI? So if you look back here, I've also added in some extra metadata, which is the application telling the user. In this case, I've used the metaphor robot handle and I'll come back to that in a second. But if we imagine that our user is a robot and the application is having a conversation with the robot, the robot has some answers and the application has some questions. And what we're doing is we're facilitating a conversation where the application says, this is a question I want answered and this is how I would like you to answer it. The robot can then say, okay, well I know how to answer that type of question and I have the answer for that type of question. Does that make sense? So here's a version where I've got the question. The question is the service required and the answer for what I want answered is the group. I want the group. So now this one's using version three. I can use the select version. And I've made it much more specifically using the question and select by the new text answer. Does that make sense? They're all following that. It still won't work if I change the URL. We're getting closer. We're getting closer to something where I can work in more free life. So these selectors here, these are the kind of things that you end up with in your page objects and you'll find this kind of thing over and over again. It's always the same pattern. And one of the things in object oriented design is if you see the same thing happening all over the place if there's lots and lots of duplication then probably that's something that should be extracted into its own and called in a different way. So what I really like to do is just say once, this is how I want to interact with text fields. This is how I want to interact with radio buttons and this is how I want to interact with select fields and then have the application and the robot work out amongst themselves. Does that make sense? So what I'd really like to say, what I'd really like to say is something like this, what I'd like to say. I'd like to say the application start at version one of the HTML, create a robot and then tell the application that the service applied is free. Does that make sense? Yeah? Does this work? Do you think this coding works? Why not? I'm not telling it how to interact with the element. Okay. Okay. I'll show you again. It's not working. Oh, there we go. It's Java and it's running on a fairly old MacBook Air. So it's not the fastest. Sometimes it's quicker. So that worked. Yay. Let's have a look at how it works. So we've got this. So in this case, I've renamed the web driver to application. This is just the web driver. This is just proof of concept code. It's just to show what can be done. If we were going to do this as a framework, we will be doing this as a framework that people will be able to just download and install and then we'll start abstracting things away. But at the moment, we're just acting purely on the web driver just to show what's possible. So this is just web driver. There's nothing special here. But what we do have is we have this robot. And in the robot, we have a method called tell, which takes the application, a question, and an answer that it wants. It knows the answer to. It has a question that knows the answer to. So we then find the element by the CSS selector. So you remember I added that metadata and the data question. So all types of, no matter what the implementation of UI is, they all have this data question metadata on them. So first of all, it finds the element using web driver. And then it uses the other metadata that I added, the data question type to look up how I interact with that type of element. So if you remember, we said that we had the data HTML text. Yeah. So this is, right, but the evil shortcuts for some reason are messed up onto the moment. So I have to probably not have a presentation moment. So we've got, for example, HTML radio. So we declare at the start that this is a robot handle for HTML radio. And that's the thing that comes in the data question type. So how you interact with an HTML radio is you find the element underneath the element that's marked with the question that has format input value before the answer and then you click on it. That makes sense. So that means if we have like three values, walk, groom, and sit, and I've asked it to look for groom, it will look up the value that has groom in it and then click on it. We can do the same for the HTML select. Yes, that works, okay. There's our text. So again, we declare it as being a robot handle for text. And the way we interact with that is we find the element that has the input and then we send the key to the answer. The same again for HTML select. So in all cases, we find the appropriate question and then we have a little adapter that tells us how to interact with that question type. Does that make sense? Okay. So if we go back to the examples, I put this into presentation. So what will happen now if I change the version of the UI? Should work. Fingers crossed. And it's Java, so it's slow. It's not the Java slow, it's the JDM right now. Here we go. Three options. One of the reasons it's so slow at the moment is it's actually doing some reflection in the background to find the robot handles. So it actually looks through, looks in many, many places and it takes four to six seconds on the initial start up for it to find those. So it can be quite slow. But once it gets going, it's quite quick. So there is a way to make that faster, but it involves narrowing down the search range. So at the moment, we have to search the entire Java namespace and looking for robot handles. What I don't want to do is force people to put it in a certain package. So at the moment, my robot handle is livingcom.rivil.robothandle. But I don't want to make everyone put them in com.rivil.robothandle. So that's why it's slow. If we were going to do this without proper implementation, we'd probably do it in a way where you can say this is where you can look for my robot handles so that they'll be the default ones in the rim drive package namespace and then you can specify com.vvil if you were to do a role or however you want to organize the code. But what we didn't want to do is restrict people at this point. So it did work. Did everyone see it flash up on the screen? So that's a little bit of an improvement, isn't it? It's like now we've got a version where the tests don't have to change when the UI changes. Let me just fill off this bit. Yes? Yes. So I'll go back to that. I'll show you. So here, can you see here, this one says data question typing for the HTML text. There might actually be a way of inferring that without having to write it into the code explicitly. And also, for the same reasons, there might be something that we can do with ARIA around the question type as well. So it might be possible to use ARIA attributes. So is everyone familiar with ARIA? The accessibility attributes? Yeah, so for screen readers and things. So one of the things that's quite valuable to take away from this is that the things that make it easier for the robots to interact with it are also things that would make it easier for humans to interact with it and what we're doing is we're adding useful information about what we're trying to achieve, the questions that we're asking into the page. And if we've got that collaboration with the developers, we can actually get that added in relatively easily. If the developers are using templates for doing their HTML or they're using something like Polymer or Angular to write widgets, you can practically get that for free because you just draw a template, there's the data there, and then you just instantiate that. So at the moment we're hard coding it and we're making it obvious and that's the demonstration purpose but there might be a better way of doing that. So this isn't the be all and end all, this is just proof of concept demonstration. So if I go on to version two and you can see that the HTML type there is HTML radio. Yeah, so if there was a mistake in the HTML yes, the robot would fail, but then that's the same situation we already have with page objects where if they change the type and don't update it correctly, so if they didn't update the ID correctly or if they didn't update some other metadata that you're using to your selectors, that would equal the test to fail. Yeah, so the question was does that mean that we can make the change just in one place in UI and yes, that's what we can do with it, we can make the change in one place because the tests don't have to fail. So let me just, yes so it's not something that I'm familiar enough with be able to talk about with any authority. It's a W3C spec called accessibility, it's called ARIA, A-R-I-A and it's a way of adding extra metadata into the applications that screen readers can understand what the purpose of the elements on the page before so one of the ways to do it, who's familiar with using unordered lists as navigation has anyone seen that? You know we have a tab list at the top of the page for navigation quite often that's implemented as a list but the semantics for a list don't really lend themselves to navigation so what ARIA lets you do is you say that this list here is actually a set of navigation attributes and each of these things is a place to be navigated to so it makes it easier when the screen reader is reading it can explain like a screen reader for someone who's blind they can't actually see what the screen looks like, they can't see what order the tab is in, they can't see where the fields are on the page the screen reader needs to be able to infer from that and if the way that you've written the page makes that difficult for the screen reader the ARIA attribute for a way that you can give it explicit information to help it out so that makes sense so it's quite possible that the same mechanism that ARIA uses to inform screen readers we could use to inform the robot and that would give us a double bonus in that by making our application ARIA compliant we'd also make it robot compliant whoo win so I'm just going to run this again which this one is now running against the select version I don't think that's one we ran last time and this one should just work and what we've done is we've abstracted how we do something from what we're doing so we've abstracted the question that we're answering from how we answer that question does that make sense? so the metaphor of robot handles it's going to be something that we're changing but the concept of robot handles was a conversation I was having with a few colleagues and I was trying to explain why the existing kind of automation tests tend to be quite brittle and if you think about a robot that's moving something delicate like eggs from one place to another if all of the eggs are exactly the same size then our robot can be fairly simple if they're always the same size same orientation and they always end up in the same place then the programming for the robot can rip this much move over here, release but as soon as we start adding variation into that, that robot would fail so if we have a different size egg or the eggs coming different orientations we stop doing eggs and we start doing cartons of milk or something that robot will then fail, it's brittle so what can we do to make it easier? well we could make it so that the robot is incredibly intelligent it can have sensors, touch sensors, sight sensors so it can go over it can identify the orientation it can identify how much pressure to apply it can weigh it and know how much grip to apply but that makes it a very complex robot and what we find in tests is that sometimes to get around the problem of a change in UI we do start to do a lot of analysis of the page before we start trying to interact with it and that makes our test code complicated and if it's complicated and we don't have the support we either don't have the support of the developers or we don't have enough skills to manage that complexity that can become very unwieldy very quickly so those two options that it's always in the same place option is like X path but it's very easy to break because we just change where something is in the page the very intelligent robot is let's write an entirely separate application purely for testing this other application we've just added a whole bunch of complexity another option is we can create a set of attachments for our robot and we can create a set of boxes that the delicate objects go into and then all we need to do is apply the right robot handle so that's where the robot handle comes from it makes our job easier by giving us an abstraction around the thing that we're trying to take but now all the robot has to do is identify which handle it needs and then it can pick it up and move it to the second place so there we go I left it up on the screen that time it actually selects the right type there one of the next steps we can go from that is actually we don't really need to explicitly tell the application every time what we want it to set what we could say is our user has a set of knowledge it has some things that it knows about and then if the application asks for those things we can then add those in we can say oh I know how to answer this because now we don't actually have to know in advance what the application is going to ask us so our better example is here and we can say that the user knows that the service required is going to be a green and then it exchanges that knowledge with the application does that make sense? that's not too much relief so this one again is against all the versions of the UI so this one is currently working against the text problem version 1 I think I'm going to have to find myself a more powerful that problem just for demonstrating purposes there's absolutely no other reason so I'll continue to it and while that's being explained it's reframed when I was setting up I was trying to connect the projector in and as soon as the projector was connected I couldn't even log in because everything slowed right down so I don't know if that's on the theme or if the projector is stealing the energy from my computer or something let's just go back so so far what we've done is we've abstracted what we're doing from how we're doing it so that means that we're doing something clever with JavaScript so something like select two which replaces select boxes with much nicer JavaScript select boxes all we have to do to make that work with code is create a new robot handle that works with data select two and then that will work with all instances of select two throughout our application as long as we have that communication and this is why I was asking earlier whether when we're working as the three amigos are we working on the same computer or are we working separately because obviously if we're working together then we can do these things together let's make the change to the application let's add the robot handle which is required to make the test work or let's add what we think is the robot handle to make the test work and we can do it together whereas if there's that separation if it's a case of testing that makes that much more difficult so obviously this is something that works better when you're working together with a team at the same time rather than separately and going your own way so I'll just this exchange knowledge with the thing here we have the ability to learn from the application and the ability to inform the application so what this does learn from the application what I'm going to do is I'm going to look through the web driver for any elements that have data knowledge attached to them this is metadata that flags up things that we think the user is going to be interested in so for example if we were doing a flight booking system it might be the booking reference for the the booking when it's completed these are the sort of things that the user needs to know and we know what those are going to be because we're doing this for somebody at the end of the day aren't we we're trying to facilitate the conversation if I'm an actual user using a flight booking system then there's a bunch of things that I know which is where I'm flying from where I'm flying to and the dates that I'm flying and then there's some things that the booking system will tell me such as how much it's going to cost what my booking number is so this is a way for us to say these are things that I want to know and these are things that I do know does that make sense so the first thing we do is we look through the application and we learn from it so anything that the application knows we will then take that into our knowledge so we now know that as well and then we inform the application of the things that we know about so informing is a case of for every question that the application has if we know what that answer is we will fill it in for you and another benefit of this is it means that we don't have to be overly specific on the amount of questions that we have because as people we already know much more than the application is asking of us by abstracting away what we know from how we into that into the application we now don't get errors if that question isn't being asked on the page so if I know what my age is but the application isn't asking for my age I won't get no element exception when I'm trying to enter my age because I'm not going to do that what I'm doing is saying what do you want to know about me and the page that says I want to know your name so I just tell it my name so I've got that one has crashed I don't know what's going on there this is inside the robot yeah so go back to the test there we go so in this case the user knows the service required I've got a slightly more complicated example here I'll bring that up on the screen first this one is a much much more complicated form but it's still using the same three elements like you don't want to get into too complicated if we were doing this in a real application it's obviously way too simple but for a real application we could find more complicated robot handles so we've got the name of the event it's Selenic Conf 2016 it asks for my name, ask for my gender and it asks for the country and it also asks for my opinion on the union so we tell the robot what it knows so it knows that my name is Randy Farmer it knows that my gender is male it knows that I'm currently in India because we are and it knows that we need you with a very big estate and and then afterwards we want to know we want to assert that having visited this page that the user now knows that the name of the event is Selenic Conf 2016 does that make sense that's not will it work it's will it run certainly thinking about it there we go it's done it yep I just said what I knew and brought it together and it all just worked so there's been no changes so for all of these robot examples all I've done is defined how I interact with text boxes how I interact with select boxes and how I interact with radio buttons and then I just tell the application what it knows and because the application tells me the type of questions that all just works so that's the demonstration that's what we've got so far any questions hmm so if I take the question was will that lead to any negative consequences such as if I take out the gender field then it's just fine that's actually a really interesting question if I took out gender from the application it would run just fine because it didn't ask for gender it wouldn't add it so that's it really depends on what we're trying to test if the application can't continue without the gender field in there then we'd expect there to be some other validation that's something later in the some later event so it's not necessarily a problem it might be a problem it's one of those things but then that's another place for it to break if it isn't a problem so if we were explicitly testing for gender and they removed gender because it was no longer necessary then the test would break as well but then it would be breaking for the wrong reason so it's a it's a balance isn't it you have to sort of whether not having a field that you think you should have is more important than having a field you don't need you could do that so Anthony was asking is it possible to add an assertion against the application that the application knows what the gender is I would I would actually be tempted to say to do that in a different way and ask whether the user, the robot knows whether it's been asked that question so it would be more a case of here's the questions that I would ask and gave answers to and here's the questions that I wasn't asked are those important so you could then do have you come across the term approval testing no anyone approval testing approval testing is something would do for example like with the look and feel of a website so what we'll do is we'll take a screenshot of a website as it was and this is what we expect the website to look like and then in the next week if there's a significant difference for it the test will fail it's not failing because it's wrong it's failing because it's different so what it's saying is there's been a change I wasn't expecting a change have a look at it and see if those things are okay and if you say yes that's okay then the new one becomes a new baseline so what we could do for your question is we could say okay here's a list of questions that we work we know and weren't asked about and then if it's suddenly stopped asking us about that question we can say ooh it's stopped asking about that question is that okay do we mind and that then lets a human sort of decide if that's the right thing or not but obviously that's a slightly different topic does that answer the question yes but the purpose of this call is to say we don't always want to test a fail so that's the first one saying you might want to fail if there's a change but you want it to fail for the right reason so that would be more of an approval test than an application failure so the question was can you use the ID as a selector rather than adding the metadata and yes you can one of the reasons that I didn't do that for this example is because you can actually ask the same question multiple times in different ways and IDs have to be unique so what I want to know is if I'm asking sorry if I'm asking what my name is normally I'd expect that to be a text field but later on I might be saying say on Facebook where you're tagging people in a photo I might be selecting from a drop down list and I might be asking the same question multiple times and using ID won't work because it's already asked my name at an earlier point so that would then mean I'd have to add the same knowledge I'd have to add my name then tag1, tag2, tag3 which isn't necessarily what I want to know or not necessarily what I want to do what I really want to do is say this is my name and I'm in this picture so when it asks my name it automatically works so you can all of this is proof of concept all of this is the demonstrating page it's all available on github so you can check it out it's at riverglide slash robot handle example and so you can have a look at how it works and then if you like this way of doing things you can adapt it yourself this is just showing what can be done any other questions? the question is how do you handle dynamic elements the example was dragging and dropping files onto the page and then having the multiple files come up with different ideas I would imagine that that would be a the placeholder would be the questions being asked and then the files would be the answers that appear and imagine it would work in much the same way as what we've got already except that you now have multiple elements so in the same way like facebook tagging pictures you might have multiple different tagging pictures so you'd have to write a separate robot handle for that and it would have to be able to deal with multiple types but yes I can't see even why it wouldn't work but I haven't tried it so I can't do the same thing with girls if you can send me an example and see what I can do oh yep on the back yes yeah okay so the question is sometimes the business logic is depending on what questions are asked so for example if the gender is male it might then ask for age whereas if gender is female it might not ask for age and if that's part of what you're trying to test is that the behaviour of the application shouldn't ask for age when gender is female then that's an extension so that's similar to the question from the front earlier which is should we check to see when questions aren't asked and so in that instance it doesn't do it at the moment but it'd be a relatively straightforward change to say these are the questions that haven't been asked and so we could then set up a scenario that says given that my user is female when I interact with the application then the questions that I weren't asked include age but it doesn't do that at the moment but it's relatively straightforward to add that okay no worries okay so I think that's time so thank you very oh sorry yes they're up on github they're at riverglide so github.com slash riverglide slash robot handles examples I don't actually have that on the screen sorry but I did in the so this is riverglide so if you look for our github repository you can find it on there or just email me or tweet me and I'll sort that out okay alright thanks very much