 Welcome you to this amazing keynote our robots the future of testing. That's kind of a big question. A lot of us have and who better than Jason Huggins himself to present it again Jason doesn't really need an introduction, but Jason was the original creator of Selenium He's been deeply involved with Appium and many other projects And we are glad that he could join us today from Chicago, you know to be with us Excellent so so it's just us and then a thousand of our friends here Yeah, I see a bunch of them going up. So that's okay. Great. Yeah, so if you're connected So I went through the training here The other day. Yes. Yes. Okay. If you can hear me and you're connected. Yeah thumbs up Okay, actually, I'm curious just for for audience participation. Okay. Now. I want to see that side of the room Everyone thumbs down. I just want to see that Does that is that fun? There's a thumbs or is there a thumbs down button? They don't have a thumbs I can't do some very impromptu voting and we'll have to do it in the discussion. Okay I think that's enough enough thumbs up though. So we can start with the show here So I should just go for it, huh? Absolutely Great. All right, so I'm gonna start my clock here. So I don't use up all your time. I Can do five minute talks I can do an hour long talk or I can talk for 10 hours about all this stuff But I'll respect your time here. So I'm officially started. Um, I will this is my first virtual conference. So hello, everybody I'm Jason Huggins I'm your opening speaker today and It's gonna be a little bit chaotic to make sure I can kind of track the discussion and my slides and stay on time So I will open the discussion tab I can't promise that I'll be able to read that and answer questions and be able to Get done in now 16 minutes, but I will try to look at that Also, just to kind of maybe break the ice here If we don't can't can't do thumbs up or thumbs down maybe you just answer in the chat So the question for the next 15 minutes are robots the future of testing What do you think you can just answer yes or no in the discussion while I bring up my slides here And I hope you can see my first slide So no, yes, definitely not. Hello, Jason is yes Yes, and no, okay At least I know there's a whole bunch of people watching not fully right so um So we'll see if we if we sway some of the votes At the end of this talk. Okay, so I'll just get into it So our robots the future of testing This is uh me Huggs on twitter. Uh, this is me in front of my Office in just west of the city of chicago in a town called oak park This is me in a january And this is me uh these days in uh in august 2020, right? So hopefully this isn't us for um The next several years, but anyway, this is just me walking back and forth from the office Um, I needed to update the slide for the presentation. Um Very very quickly. I'm going to just rush through here even though each one of these is a very long story I started the project the slunning project in 2004. So I can't believe here. We are 16 years later And very much the project is is all of you who are attending Including simon is the leader of the project right now. Um, I'm the keeper of the grand fire Keeper of the campfire stories But uh Anyway, so I'm all glad I'm very glad you're here But it takes it takes a lot of people to make that keep this project going and it's all of you now Um, so that was 2004 2008. Uh, I founded sauce labs Again, I'm just going to keep going here in 2012. Uh, we started the apium project And in 2015 I Took my side project kind of hobby and formally incorporated as a sea corporation taps your robotics And anyway, so that's my brief bio All over those years the way I would describe selenium is that it was it's like a robot And this is actually the slide that I would use when I would talk to the press or investors or whoever It's hard to vision. It's hard. It's hard unless you actually see selenium Automating a browser in real time. It's hard to kind of describe what it is. So I always say it's it's like a robot Um, and after a while after several years, I kind of wanted to test that metaphor Uh, and I call this the reverse tron if you've seen the movie tron, you know These people got kind of trapped inside a computer reverse tron is basically taking selenium out of the computer Into the real world and giving it arms and legs. Um, and hopefully it doesn't you know become skynet and take over the world, but um You know, hopefully you'll just stay True to just tapping buttons on phones. Um, this is the most recent robots that um that we've Release, this is the tapster three. So several generations of different robots. Um, I'm hoping that these Embedded videos work in the presentation did a run through first before this and looks like look like it did Don't know at the I don't know sure if you can hear the audio. It's not so important Uh, but this little demo is just sending hello world as a tweet Did not go to the next step and have it click the actual tweet button, but testing out the keyboard functionality So that's that's kind of like the latest and greatest of the robots Um, I can go into all the different kind of styles of the robots, but it's there's not enough time um But I will say this was the first robot I made this is the bit beam bot a lot of things are different Here the big thing is way back when my robots were made out of laser cut wood um It's also um, you can see I think there's even some I laser cut these holes in a particular style to be a lego compatible. So there's a couple of lego parts around here um And I'll show you a video of this first robot The design goal really wasn't to you know start a robotics company at the beginning It was it was just a you know, almost like an art project at the time in 2011 as you can see when I when I made it um angry birds was really popular um And so here's a quick video. I apologize for the Blurriness of the video. I I recorded this as like a backup to the live demo I was going to give to the the conference presentation that I was showing But I think this was recorded maybe 30 minutes before um way back when anyway, so um, that's the very you know Very brief demo of my first robot um one of uh A friend saw this video and actually said like hey if you want to speed things up you should try different robotics designs And so over the years, I've specifically tried different kinds of formats of robotics specifically with the idea of like speeding things up making it kind of just a better robot more specifically a better angry birds robot Uh thing. Anyway, so this is the teabot And there's a bunch of stuff going on here, but there's two circuit boards Now there's one that's controlling this robot if you're really into the the nerdy details The geometry for this is a core x y as opposed to cartesian. I was the one before um And then there's another circuit board here one of the things that I've added over time um is these little servos To press the buttons also just kind of a quick um Just to check in just make sure i'm not like speaking into the void here um Can you hit our thumb can I get some thumbs up just to make sure that the video is playing correctly for everybody? All right, great I imagine this is as awkward as you know anybody in tv or a youtube streamer or you know, it's just like Don't know uh, if i'm uh if my connection has dropped or anything, but so far so good um So to the to the heart of it though our robots the future of testing um I would say yes, um, but then I always have to kind of clarify it. Um Rope the way I understand robotics is that it's at the intersection of three Circles software electronics and mechanics and so now all the way in Uh, you know seven minutes into this talk I wonder if some people joined this talk because they thought I was going to be talking about robotic process automation And um, I'm not talking about that today. In fact actually one I hope there actually are rpa robotic process automation talks at future selenium conferences if not this one I I apologize. I did not check the schedule. There also should be rpa vendors in the booths I think actually there's there's plenty of overlap between selenium and rpa uh to the point where um, I've googled like What is the difference between rpa and selenium and the more stuff I read from rpa vendors saying no We're totally different than selenium the more I'm convinced there's absolutely no difference between selenium and robotic process automation um However, I would argue that um, you know, they kind of blurred the word I would always emphasize that selenium is like a robot but robots are robots like you need the mechanical and electronics Um, so I'm just focused on actual robots today not the rpa stuff Um, so I'm just going to show you a couple of examples of what I mean by actual robots in testing and and it does relate to um Most of us in the set I'm assuming coming from a software background and software testing You're not necessarily electronics or mechanical engineers on this talk And so here is is where effectively this these robots are an extension of your software testing job and a couple of examples so this one was um Long time ago. This is probably chapter one. Uh, still made out of wood. Um Very quickly a lot of stuff happened just right there But um, the robot was intended to be have the phone there But what these people did they were working on testing the nike fuel band So I'll just freeze frame it here and when they went back to the office. This is after a workshop that we had Um, they were trying to figure out how to do an end-to-end test for the nike fuel bands Basically it's like a fit bit. They can kind of measure your it's like a step counter And so they had the app they're trying to figure out how to do the whole end-to-end thing and they put they decided to put the Risk band in the robot and shake the risk band and that's how they can do the end-to-end test And so you can kind of see at the beginning of the video. There's 891 Steps and then it goes up to 892 by the end of the test. It's up to 894 So that that was really creative. That was one of the things I did not anticipate I thought they had the robot to test the phone. Anyway, but that speaks to an example of when you have Uh a test scenario that requires the phone talking to something local to it Not a database in the cloud but something of connected device like a bluetooth device or over wi-fi So it could be a tv or your step counter your watch Your toaster or something like that those kind of testing interactions You might have to there's actually specific examples where This actually happens a lot where That device needs to get paired over bluetooth to your phone and testing that scenario specifically The engineers I talked to they'll need to push new firmware to that fuel band or the watch or whatever and The test scenario that they'll need to do is nothing sometimes it's they have to power cycle the device So they have to literally hold down a button And they need something for that or they want to test the interaction where they also are turning on and off Bluetooth and they want to test that whole like first registration process And the conversations we have is like the only thing they have to for that is Uh manual testing because they have to physically hit these buttons that are hoping to bring in robots to Hit those buttons And also there's another aspect of this is you could argue that Robots are not the future testing from the point of view. You can do a lot of stuff in simulators But even if you have a simulator for the phone You wouldn't necessarily have a simulator for your phone and your bluetooth device and kind of a simulated world Where both of those things can interact together, right? So at that point you have to be in the real world and occasionally you have to have a little thing that presses these buttons um There's some other scenarios in this particular case This is a robot that we made The use case specifically was for getting to the boot loader screen of an android phone This robot's a little bit over engineered for that task because it can do anything on the touchscreen And it can do the side buttons. It was called sidekick was the name of this one But a more refined version of this robot was this We call this a pbr a push button robot all it it's not really a robot all it is is has these little servos here for pressing The buttons on the side And so it's really subtle, but if you look right here um You can see like the actuator. I've got a very simple api long press and release I'm gonna mute the audio here, uh, and this really gets you to it starts the phone occasionally if you have a device lab if you're a You know a real device vendor And you have all these devices all racked up. Occasionally you literally just have to press the button All these phones that we have are not really designed to be servers. So if you had a server in a rack, you can turn these things on Remotely, but phones weren't necessarily designed To be automated to be turned on from the network. So occasionally the conversations are like we literally need a little servo in there to Press the button. Anyway, so that was one of the things that we created This particular case you could argue. It's not a robot. This is something I showed off At the apium conference last year in vangalore. Um, I've got two things going on here one little, um Arduino circuit board is talking to the mac. That's my laptop. The other one is acting like a virtual mouse So it's not a robot, but is some it's more than just purely software running in a simulator Uh, and so this is a new a new feature that was that came out in ios 13 last year that ios supported Uh usb keyboards and so this little circuit board can act like a virtual mouse or keyboard Which means you can have access to maybe automate iPhones in a different way So there's a certain argument of like if you can expand your your toolkit of skills beyond Purely software skills and maybe bring up, you know a little bit of electronics here You can you have access to kind of automate things you wouldn't be necessarily limited by what you can do through the usb cable or through a simulator This is my favorite example of You can maybe argue. This is a robot. Maybe not people in robotics love to argue whether about some something is a robot or not um I there's a question here. I hope to get it back. I haven't been watching but What will you use to assert button actions cameras? Yeah, it's an excellent question Um, I'll come back to that so the demos that I've shown you so far No, the camera's a not been involved, but that's kind of a future thing to have a camera Or maybe I'll answer now You have a camera there to do computer vision and then close that loop to do validation But sometimes you can do validation by just getting the screens off the through the usb cable Anyway, that's a longer conversation. Uh, this is my favorite. This is one of my sorry going back to the presentation here This is one of my favorite test automation scenarios from a company Design agency in in Australia, I believe and you have three point of sales systems here and then a charge card And they're testing various interactions one again If you're setting up a point of sales system, sometimes you actually have to set up the discount rules literally on the device Also, I think the way if you read the blog post about this They're testing the charge card with different scenarios with sometimes the charge has a negative balance or positive balance Or it's frozen or something like that as they um, there's enough time The train is going slowly enough so the software behind the scenes can reset before it tries the next test interaction I thought this was like ridiculously creative So you don't necessarily have to have a robot that looks like c3po or r2d2 to do to do kind of robotic testing I thought this was interesting. Um This is very much an abbreviated version of of my other talks i've given but you know Those are a couple of examples But i've been talking about this for like nine years now And so there's a certain argument of like yes, uh, it is debatable on whether the robots are the future of testing I would argue that if you're in those specific niches Um that need robots then yes, the robots the future testing, but there's so much like, um, just if you look at the example of um Forget robots, but web and selenium and then there's the mobile platform, which is huge and would you necessarily need to learn All everything about mobile and apium um To do kinds of testing like there's so much still left to do in just web testing That you could spend your whole career only on web testing and not have to do anything with mobile same thing with even if you did web and mobile You might not find yourself, you know dealing with these connected devices Or doing things point of sale. So I think the better question then really is our robots the future of your testing to you as a person and I the thing I want to really kind of um For people to take away from this is the answer is yes if you want it to be right so if if you um I felt this way early in my career I I wasn't exactly kind of working on the things that I wanted to um And for lots of different reasons robots are just kind of fun. And so I made that true But else but I also didn't really know I always wanted to get in robots But I didn't really know that there was a really good fit I'm here to tell you that that if you did want to play around with robots and kind of leverage all of your Software testing skills you can do that. There are companies. I guess one way to look at it is every product that is made Like physically made they all all those companies. They all have secret robot labs And they're employing robots and people with testing skills to kind of do those things It's kind of hard to find them the kind of the way I actually only found them as I made this robot And then they kind of came found found me but they are out there um And I've only have like 20 seconds left here. So I'm just going to briefly tell you how you could get started You don't necessarily I this is not meant to be an infomercial for my stuff So you can kind of get started just you have to kind of believe that is possible and then kind of You know, there's a couple of next steps so one You can pick up I mentioned it very briefly, but it wasn't able to get into the details And Arduino is a this great platform that that can be your bridge between your your laptop and Controlling the environment around you. There's an upgraded version called the Raspberry Pi And beyond that there's an integrated system like you know if you I'm not going to jump back to it But that slide of robotics where it's the electronics software and electronics And mechanical sorry The lego mindstorms it kind of has all that put together and there's even a it's not officially lego, but it's it's a It's a open-source project called ev3 and Sorry, I'm jumping around here. Uh, this is the robots the printer printer bots by the ev3 project This is a particular robot that you can build And this particular example It's just drawing a dinosaur but you could swap that out with a little stylus put an ipad or a phone there And you can kind of you can build this yourself. Uh, so for a couple hundred bucks Um, you can put this together and start kind of getting into Uh, testing with robotics. So really You know, is it the future of all of our testing? Probably not But is the future of your testing it can be if you want it to be And I just want to plant that idea in your head so Two slides that I just didn't show one other thing that what it does help is having a little 3d printer This one's really cheap 150 dollars and going back to this video These little two parts right in here those were 3d printed to hold the stylus So if you have the lego mindstorms kit and a cheap 3d printer, you can pretty much do anything So that was as quick as I could do it in 20 minutes. So thank you. Um There is my uh, how you can kind of find me. Um, I'll see if I can answer some questions and Arash told me to If we can't get the question answers right now, I'll also be available this morning to hang out in the vip booth to have More question and answers So I I hope that was uh useful useful Use of your time for the last 20 minutes. So thank you All right, that was awesome I think we are little running short of time Just want to keep the schedule. We got a little last late start. So Do want to really thank you Jason for sharing your experience. I know it's a stretch to ask you to Compress all the stuff that you've been doing over nine years into a 20 minute session But you know, this is a great glimpse of some of the potential that is there So thanks again for sharing that