 Hi, everyone. This is really exciting. Appium Comp is super inspiring and energizing for me. So, yeah, Dan already introduced me. I look like that. I'm even wearing the same hat, and I have a different shirt on today, so I'm a little bit off-brand for CloudGrey, but, you know, it works. So, yeah, I worked on Appium for a long time, mostly while I was at Sauce Labs, and then more recently started my own consulting firm around mobile automation strategy, and obviously do a lot with Appium, and happy to say that I've even worked with some of the people in this room in that capacity before I spent a lot of fun. So, here we are in India. I notice it seems like Google Maps actually puts real weather on their satellite view now. At least that looks like real weather. I hear that there's some kind of cyclone happening that some of us will have to fly back through. So, I'm not looking forward to that. But I'm really excited to be in India. I've only been in India once before, and I was almost a decade ago. I've never been to Bangalore. I'm really grateful to be here, and I just want to also take this moment while we're here in India together to recognize a few different people and a few different groups for their contributions to Appium. And this is something that I think is really important because obviously you get a lot of attention for standing up and talking at things like this. So, there's sort of an inappropriate level of focus that gets put on people like me and Dan. But it's an open-source project, and the real drive for Appium is not even the maintainers, but it's the people that use Appium and the people that contribute to many little ways every day. So, if it were just for me and Dan, there would be no Appium conference, it's really because of all of you. So, I just wanted to ask, if you've ever contributed code to Appium before, would you raise your hand? Awesome, yeah, good handful of people. And if you, so just keep your hands up, and if you have ever contributed some documentation to Appium, raise your hand too, okay? And if you've ever submitted a bug to Appium, yeah, a lot more people, that's what I expected. And if you've ever gone onto the Appium forums and either asked for help or helped someone else out on the Appium discuss forums, so that's like a lot of people. So, I just wanted to take this moment to recognize that this is the kind of contribution that really makes the project work and makes it awesome. So, thank you all for the part that you play in moving Appium forward. So, kind of theme for this talk, such as it is, it's really a thinly veiled disguise to do some kind of experimental thing later on, but we have to have a talk first. So, what we're going to talk about is the changes that are underway in our industry. I don't know about you, but it seems like as I look at the type of work that QA engineers and software testing professionals are responsible for, it keeps getting more and more complex. The cost of quality failures keep getting higher as the importance of well-functioning and well-designed applications gets higher. And it gets hard to keep up with the additional requirements. So, 10 years ago or 20 years ago, if you were in software testing, you would be expected to make sure that the app worked on a functional level, and that was pretty much it. But we're very much in a new world right now. We've moved pretty far beyond functional testing as the basic set of requirements for jobs in our industry, and now we have responsibility for things like visual testing. We have to make sure that our app looks good and doesn't have some kind of visual regression or bug. We're responsible for performance testing because it's not just whether our app functions or not, it's important, but it's how quickly it functions and the type of experience that the users get. There's more broadly user experience testing, which encompasses a whole host of other kinds of testing, but really puts the focus on user experience quite broadly and wants to make sure not just, again, that the app functions as intended, but that it does so in a way which is actually meeting the needs of your users. If you're some kind of video or audio streaming app, you obviously care about the quality of the content that you're delivering and not just whether it's delivered at one frame per second versus 60 frames per second or whatever. There's a lot of really important aspects to audio visual or audio video testing that we didn't really have to care too much about, especially before mobile devices came along and we started consuming media all the time everywhere. And obviously more. I'm sure there are people in this room who are doing kinds of testing that I haven't even considered. And that's kind of the point that I'm making. The kinds of testing we're responsible for is expanding, and it's expanding very quickly. So to dig into a few of these a little bit more specifically, we're probably familiar with visual testing. This has been around for a while now, and our AppianCons sponsor, the Apple Tools, really pioneered this as a testing discipline in a really important way. But if you've never done visual testing before, haven't gotten into it yet, it's basically designed to answer the question of whether your app's visual appearance has changed in a way which is not intended, which could indicate any number of different problems. And just to talk about a few ways that bugs could enter in to the app on a visual level that don't really map to the functional level, you could have intrusive or absent elements. Things that shouldn't be there that are, or things that should be there that aren't, that don't necessarily affect kind of the functional level of the application but really affect the user experience. There could obviously be style or design bugs. Your app could look ugly, and that would be really bad. But that's the kind of thing that would never be caught by, you know, one of the standard type of apium test that's just looking for elements and interacting with them and so on. There could be bugs due to, you know, form factors or device layouts that weren't considered in app development. And all of a sudden you're running on a different kind of device and the layout is totally broken. And, you know, again that's the kind of thing that a standard apium test wouldn't tend to catch as long as all elements are there. There's also stuff like internationalization bugs where developers assume that strings are a certain length and that's what makes the layout work, and then you translate your app into other languages which have, you know, different writing directions or different lengths of words and phrases and you can get some pretty nasty visual bugs there. So this is the kind of stuff that we're now responsible for that, again, we weren't so responsible for the state of the art in app development improved and before the importance of getting all these things right really moved forward in a big way. There's also the category of performance testing, which is in a way really trying to answer the question of whether our apps are responsive enough, you know, making sure that they're not stalling, making sure that they're not too slow and frustrating users, and, you know, there's all these studies that have been done that show that users are actually, you know, a quite easily frustrated group of people, much more easily frustrated than, you know, developers of applications when they're using their own apps and, you know, if you have a delay of even fractions of a second, some scenarios, it can really turn a user off of an application. So it's super important to have a very responsive app and people are beginning to test this as a discipline and to test it in automated ways. So some of the things that you can, that can lead to performance problems in app development or, you know, network design and things like that are, you know, developers often make the mistake of writing code that somehow, you know, does something bad with the UI thread and your app becomes not responsive. Or you can have situations where the network is slower than you anticipated and that leads to a bad user experience. Or your backend team might have a poor network stack design where they've, you know, I was talking with some folks at Headspin who focus a lot on performance testing and learning a bit about the types of problems that they see and, you know, one problem that they see is network connection objects on the app in the app code not being reused well. And so, you know, all these network requests are continually recreated and you have all the overhead of new TCP connections and stuff that really increases the amount of time that it takes for content or images to load and, you know, a simple fix in the network stack or the use of the network APIs for iOS or Android could fix this. But often people don't uncover these things during app development and so it comes to testers to uncover these bugs and ideally know a little bit about what could be some of the problems leading to them. Of course, there's also kind of typical performance issues that plague any software application when in certain scenarios you overuse system resources and this is especially problematic on mobile because obviously mobile devices are pretty resource constrained compared to, you know, our laptop computers. They're quite powerful, remarkably powerful in fact but, you know, especially on Android, it was problematic in the earlier years when one app could take up too many system resources and really negatively impact the performance of the whole device. And so, you know, it's really important to have good app development hygiene to make sure that you're not using too many of these resources and tools like the data analysis of performance data, flame graphs and all that kind of stuff help the testers and developers figure out where these problems are breaking in. Also, sometimes when we do testing, we're not totally aware of all of the contexts in which our app will be used. So, you know, people that are developing for one market that tends to use a certain type of device might not realize that their app is being downloaded and used by a completely different market that tends to use devices which are, you know, much lower power than they were developing on. And so, they're kind of leaving a lot of money on the table, so to speak, by not testing for these other markets where their application could be much more popular and useful if they only, you know, coded to the particular hardware that those people use. So, we also have, you know, just to take one more example, audio and video testing, which is particularly important for applications that deal with streaming, video and audio, which are a lot of applications nowadays, and we probably, each of us have our handful of apps on our devices that we use to watch TV and movies and listen to music and so on. So, there can be a lot of issues here, right, that wouldn't necessarily be caught by an Appium test because Appium isn't really designed for, you know, high frame rate capture and verification of video and stuff like that, so you could have, of course, drop frames, which really impact the user experience, even though the video itself is still playing. Videos can also fail to play, but if you're just checking for whether an element exists in your application, you might not realize that, you know, the most important element, the video element, isn't actually playing, and you might happily go on your way to the test without verifying this, and obviously there's a number of ways to verify that a video is actually playing. You could have, you know, a high frame rate video that's playing, but it could, you know, have a lot of chunks and blocks and weird compression artifacts in it, and the same is true for audio, of course, that would make these types of media completely unconsumable and very unpleasant experiences, but, of course, you have to have some way of telling whether your video is actually playing in quality or not. And then, of course, there just as with the performance testing case, there can be issues with the back end in all of these applications because, obviously, for media content apps, the delivery system for the content is very important, and there can be bugs there as well. So this is just kind of a little exploration into some of the kinds of testing that you've likely been, you know, hitting your head against, depending on the app that you're responsible for automating. So the basic point here is that, you know, it's probably the case for you that you feel your job is getting more complex, that the requirements are stacking up, that you're responsible for more and newer kinds of testing, and you may or may not feel that you have the appropriate tools to deal with all these cases. I certainly don't think that there's a perfect set of tools for all of these kinds of test cases. And so, you know, we're in a situation where we've been cobbling together different systems to deal with different types of testing and some of it's automated and some of it's not, but there's certainly no one right way to do all this and an awful lot of complexity that can get pretty overwhelming. Of course, there are tools that we try and deal with these things, but as with a lot of other areas of software development, as we get more and more complex in the kinds of stuff we can build, we get more and more complex in the ways we can test it, and we wind up with lots and lots of tools that compete with each other and it feels that, you know, the second we learn one tool to help us do our job, there's now another tool which is, you know, potentially better, but we don't really know and should we learn that one. So there's an awful lot of tools out there which leads to a sense of fatigue, as has often been noted and, you know, and made fun of in the web development community with JavaScript frameworks and things like that, but I would argue that the same is happening in pretty much every area of software development. So how do we, you know, how do we deal with this proliferation of tools? I think it's a really important open question. How do we structure our teams? Do we specialize? Do we have individuals who specialize in one type of testing? Do we just, you know, train ourselves more in all these other kinds of tools as well? We hire more people for our teams. Do we scale back our expectations of what we can test? What are the realistic expectations in this kind of new world? And since it's, you know, 2019, it would be an omission, you know, not to mention one of the really big themes that's out there in the industry right now and is trying to deal with some of this tool fatigue and some of the fatigue that comes from having all of these new responsibilities and that's sort of the rise of AI and machine learning for testing. So obviously AI and machine learning is a big topic again in every area of the software industry and has been put to use to good application in a lot of different fields. Let's say good application, I should say successful applications. It's often not that good or necessarily positive for the world but it can be very effective at solving certain kinds of problems. And so there's a lot of people out there who are trying to apply AI and machine learning approaches to testing. So there's sort of more lightweight and more, you know, serious ways that people are trying to do this and it kind of spans the spectrum from image recognition tools which, you know, you might call AI, you might call machine learning. You might not, maybe they're just kind of specialized algorithms for image work but of course the terms AI and machine learning get applied to a lot of things nowadays that maybe technically aren't, technically don't fit that definition but there's also a spectrum of solution to these problems, some of which do kind of move into the real fields of artificial intelligence and machine learning at least. So, you know, again, there's image analysis. APM has tools for this now. There are companies, again, like APA Tools that have formed whole businesses around making this part really work well and putting a lot of smart people to work on, you know, figuring out what the right models are for doing this kind of thing. There's machine learning that attempts to help us with analyzing our user experience data. So taking some of that performance data, for example, and giving it to us in ways that make it easy for us to figure out what's going on. So obviously, you know, I showed that image before of the performance, you know, data and it was a little bit too, you know, complex for me to really understand what's going on. So now there are tools that take the raw data and try and use it to surface the important bits to us so that we have a relatively good idea of, oh, you know, this combination of network bottleneck and something else means that it's likely that your backend server is misconfigured. You know, so tools that try and help us go more directly to an answer about what could be wrong versus forcing us to slog through reams and reams of data to figure that out on our own. The same is true for our tests themselves because our tests aren't just verification or application. They're also bits of data and as we run lots and lots of tests and we move into a world of continuous integration and continuous development where we're running thousands of tests multiple times a day, all of a sudden we have a lot of test data and we want ways of kind of sifting through that and figuring out what could be the root cause of a flaky test or something like that without having to go in and again dig through all the data ourselves. So there are machine learning approaches which are offered by different folks that try and help us wrangle our tests themselves into submission in a way which helps them to serve us rather than getting overwhelmed by going through all of our test results. There are test authoring tools, some of which claim to be totally using an artificial intelligence approach, some of which are more simple. So there's tools, very simple tools like Appium Desktop which you probably use that has a very simple recording system that can generate some code for you and it doesn't work particularly well but it can help you if you're learning the Appium API all the way to folks like Testim or Test Project which are trying to offer whole environments for authoring your tests without having to code at all. And again, we can debate the merits of these approaches but it's an example of different folks that are out there trying to apply these techniques to solving the problem of test authoring which is a real problem for sure. And then even one step beyond that there are tools and companies that are trying to make the problem of testing sort of go away entirely by having tests automatically created and automatically discovered from your application. And so a company I've worked with like TestAI is trying to do this by you just giving them their app and then they generate tests and they generate reports for you without you having to even take the step of authoring a test. So again, there are people trying to use AI and machine learning to solve all these different problems and I think it's worth following this trend and seeing how successful it is in solving these different types of problems while of course taking all of the marketing hype with the grain of salt and waiting to see how truly useful these things are day to day because quite honestly some of the more radical approaches I don't find are more effective today than some of the traditional approaches of just writing test code but it's worth paying attention and seeing how these things evolve and improve and whether they can indeed fulfill the promise that they have been giving us. So where does Appium fit in this new landscape of different kinds of testing and different tools and different approaches for solving all these new problems? Well, obviously it's not going to do everything but I like to think of it as the common interface for platform automation. So I would be very surprised if Appium ever became opinionated about how you should run your tests or even to know about tests in the first place. Appium like Selenium is an automation tool that lets you remote control a mobile device but I think the vision for Appium as we've been saying for years is obviously beyond mobile devices, beyond browsers, viewing it as a common interface for automating any type of platform and obviously the Appium team could never create automation capabilities for every platform that's out there which is why we have to have a big vision and share it with the world to get all kinds of people coming and contributing automation for different types of platforms. So I think there are a lot of benefits to this way of approaching Appium's philosophy. One for all of us is to help reduce the tool fatigue is that there's less cognitive overload when we're dealing with one API. We call it the WebDriver API today. Maybe in 10 years it'll be called the AppDriver API or maybe we'll call it StarDriver like we've always wanted but the basic idea is that the more we can reuse the same APIs the less new things we have to learn and that makes our lives a lot easier helps us adopt new tools and new technologies without feeling like we have to throw away everything we've learned and start again from scratch. Another benefit is extensibility of the Appium platform. The way we've designed and architected it it's really easy to build new Appium WebDriver compatible drivers. Basically all of the WebDriver stuff is abstracted away and you don't even have to think about it when you create a new driver. You can just worry about how to automate the particular platform that you're creating a driver for. Coming with Appium 2.0 which we're still kind of working through the right design for this but I think work will begin on it soon. We've introduced the concept of plugins that could enable stuff like the thing that I built that you might have heard about which enables you to use an image classification model to find elements using Appium. So this would be an example of a plugin that's used for a single purpose of finding elements that you could download and some people might use this plugin and some people might not care about it but you can have a whole ecosystem of plugins that help extend Appium in different ways that don't need to be part of the main server. Of course you can mix and match drivers and plugins as part of this vision. And finally we always want to kind of keep the philosophy that we've had of being this big umbrella and having a big community and not just focusing on one type of platform but really giving people tools to bring new platforms into the fold and I think that's been really successful. We've had kind of new surprising drivers show up like the UI TV group created a third-party driver or Samsung created a third-party driver for Tizen that we had no idea they were even working on and then one day it just showed up in Appium and it's like great. So we want that kind of thing to keep happening. Of course as I already said Appium is not going to try to do everything so I think we need to be clear about kind of the place that Appium has and I like to think of it in this way where there's kind of a really high quality open source core which is Appium and the world doesn't need to compete on the kind of stuff that Appium does and then there's a whole ecosystem of open source tools and paid services that kind of surround this and make it better and people can choose which things they like and which things they don't. Java. I just knew that Java was going to ruin my talk somehow. I'm blaming Angie. She just released a Java course. That's probably why. So anyway I think you get the point but I like to think also that Appium will keep up with the future so it's not that we're going to say no because AI and machine learning and stuff and say well someone else will do that. I mean we'll get in there and we'll get in the mess of it and try and figure things out too which we already have been. We've added image recognition features. We've been working with Test AI to add this experimental element of finding AI to Appium. Right now we're working on ways to group commands together to send to cloud services when you're running Appium tests so that you don't have to deal with the latency of commands going back and forth. Obviously we're always committed to keeping up with the Android and iOS platform updates and general maintenance and stuff like that. Then there's always new and useful techniques that we learn about often from the community saying oh did you know you could use ADB to do this and we say oh we didn't know that but now we can make it an Appium feature and we generally do so we keep adding things that seem useful. So in my remaining time I want to just share a little fun example of something I've been working on to try and illustrate this point of Appium kind of expanding continuing to expand its platform umbrella. You might have seen a talk I did in 2016 in London at the Selenium conference called App to the Future which was maybe my favorite talk that I've ever given because it was all based around movies, movies, puns like this one which I don't know. I am a dad now so I suppose dad jokes are totally appropriate. I wasn't at the time I guess I was just practicing. But one of the points I was making in this talk where I was sort of imagining what would it be like to have Appium for an X-Wing? I imagine that you could use the web driver API to find and shoot some proton torpedoes and what that would look like. And it was sort of a tongue-in-cheek way of talking about automating physical things and automating the internet of things. So that's sort of what I've been trying to experiment. I sort of put it out there in the universe like what would Appium for IOT look like trying to do the Jason Huggins thing where you put it out there and then someone does it but nobody did it in the last three years so I was like fine I guess I'll do something with it. I don't know how useful this kind of IOT automation will be but let me show you what I've done. So this is something put out by a company called Adafruit. It's over here and it's a little it's a little computer really that has lots of sensors and you can hook wires up to it and get signals in and get signals out and you can put code on it and have it do whatever you want basically. So I got this thing and then I thought what can I do with this and I decided to find a yogurt container from the refrigerator and empty it out and put some buttons in it and I clipped some wires from these buttons together with this circuit playground device and then wrote some code actually in Python it's really easy to do this kind of thing that whenever I press a button one of these LEDs will turn from red to green but so the whole thing looks kind of like this there's a video of it you press a button and one of the LEDs turns from red to green but that's not all that happens because I was also able to connect up a little homemade audio cable and have it so that every time you press one of these buttons a sound is produced so this is actually a little drum machine which I'll show you what it sounds like in a minute so kind of from a schematic point of view I don't know anything about electronics really I was trying to get hugs to explain some stuff to me but I don't know how to do electronic schematics this is my my own version of it but you have your buttons they're connected with some wires and they need to be grounded to the board and when you press one of the buttons the light turns on and some sound is sent to your headphones pretty straightforward this is my first project like this that I did it was super fun, highly recommend playing around with this kind of stuff so this is basically the app so I have this drum machine it's a little device that I created now I'm thinking about how can we test this app and you could think of different ways to test it we could test it just by writing more software to put on this device that would trigger the audio output but then I'm not really testing the inputs to this device I'm not really testing what happens when an electrical signal comes in I'm just testing from the software on the way out I could create a robot that would hit the buttons that's probably what Jason would have done but I don't know robotics and that seemed a little overkill to me, no offense Jason but what I thought and this actually was an idea from my partner at CloudGrade Jonah is what if we had another device that was designed to send electrical signals into this device so in other words instead of buttons being connected here we would have this other device actually just sending electrical signals in so it's still an end to end test but it's not it doesn't require me to manually press buttons so I got something called a Raspberry Pi which is basically a full-fledged computer it can run versions of Linux and stuff like that and it has these things at the top you can see a big row of little metal pins and it's called the General Purpose Input Output Header and so it's just a bunch of pins that can send and receive electrical signals so what I was able to do is get the Raspberry Pi, get some wires connect them from the General Purpose Input Output Pins and hook them up to the same spots on the the circuit playground where the buttons connect so it looks something like this when they're coming out of the Raspberry Pi and then when I connect them to the circuit playground I'm basically just using alligator clips to connect them to the exact same spot where the buttons connect and I'm not even disconnecting the buttons I'm leaving them connected so I'm not modifying the Applender test so this is in true Appium fashion we're connecting these alligator clips so that we can send electrical signals and trigger the behavior of my app so then what I was able to do is actually write an Appium driver for the Raspberry Pi GPIO so that I can call so I can write a test script on my laptop that creates a session on the Appium Raspberry Pi driver and then basically tells it what to do with pins, you know, should a particular pin send an electrical signal should that signal be high or low should it be an input or output pin all those kinds of electrical things that I don't really understand but was able to model my way through and get something working so what does this driver look like well it's pretty simple I'm not showing you the implementation here obviously but it just has a few methods in it it only supports four web driver commands create and delete session so we can start a session and stop it we can find an element which in this case is just one of the pins that we want to automate and we can set a value on one of those pins and the values that are supported are either 1 or 0 because the pin can be either high or low and then I had to basically think about well how do we map pins on the Raspberry Pi to the actual inputs on the circuit playground and so I decided that that would be what the app is so we define an app as just a set of pins mapped to IDs which are just strings that define certain place on the circuit playground and we talk about what their mode is and what their initial state should be when our test begins so that's how we create a session and now I'm able to find an element by its ID which because I've set this up in my app mapping I know maps to pin 7 on the Raspberry Pi and I'm able to either set that pin to low or high using send keys so that's how the app in Raspberry Pi driver works let's actually try it if I could get your help Dan I know I'm running out of time I'll try and do this as quickly as I can so here we have my phone which I'm going to use here so you can see what's happening and then I have my Raspberry Pi which I'm going to turn on and I haven't plugged it into anything yet so here's my test test device and then here is my app I'm going to plug so the app should be working so I have I don't know so there's my little drum machine I'm not really a drummer but super fun so the app is working so now all I need to do to test it obviously is connect the Raspberry Pi stuff here to the right spots so this one will go here hopefully it's not just touching the rubber and this one will go here and I'm just remembering where they go hopefully I'm getting it right this one will go here ok things still work I can still play the drums but now hopefully I'll also be able to control it for my Raspberry Pi so I'm going to go to my terminal here and connect to the Raspberry Pi over local Ethernet so now I'm actually on my Raspberry Pi I'll try to make this a bit bigger and I'm going to start up I'm just going to make sure all the pins are in a good state first and then I'm going to go into my Raspberry Pi driver so this is actually running on the Raspberry Pi using Node.js and I'm going to start the Appium server making sure that I can connect to it from my laptop so while that's starting show you our test code it's basically what I showed on the slides here it is it has some extra functions that have to do with playing drum notes and stuff but here's the function that will hit a drum basically it sends a certain pin state and then waits for a certain amount of time and then resets it it's basically the equivalent of hitting one of these buttons and then here's where I've defined the song that I want to be played just a bunch of drum hits basically now everything is connected I should be able to run my Appium script here and we'll see now obviously for thanks Dan, I appreciate that obviously for this to be a real test we would need to capture that audio and perform some kind of audio analysis to make sure that it actually played the right song, that the drums weren't hooked up incorrectly or things like that I didn't do that but I did recently write an Appium Pro article on how you can perform audio verification using audio fingerprinting so that is an exercise left to the reader so that's basically it just as a way of summing up everything I've been saying the WebDriver API is pretty flexible you can use it for websites or input output pins on a microcontroller yes and you can run it in a lot of places I was actually pleasantly surprised that I could actually run the Appium server on the Raspberry Pi I didn't have to do anything special for that that's just because the technology involved in the Raspberry Pi is awesome already and it's just Linux so I can install Node.js and actually did all the coding on the Raspberry Pi over H2 which is pretty fun it was super easy to create this driver it's less than 250 lines of code there's an open question of whether this kind of thing is truly useful for business applications I have no idea but it's super fun and it's the kind of thing I think we should all be doing because some day some application will come around and then we'll have the technology to deal with it and again like I said last year it's important to get out there and do creative stuff with Appium even if there's not some pressing business need for it a couple of people asked me Jonathan did you bring your guitar or ukulele and are you going to play a song this year and no I did play a little bit of really bad drums for you but I'm sure that was not quite as pleasant but there is something kind of cool that's happening right now which is that all of the songs that I've done involving Appium in the last five or six years including the one where I did last year at Appium Comp 2018 they've all been released or are being released on an album that my band is putting out it's going to be released next Tuesday so if you're interested in hearing studio versions of the songs that I wrote specifically for Appium and involving Appium Automation you can check out our album that is released everywhere okay thank you everyone