 So our talk today is going to be the robots are taking our jobs, but in a good way So as you heard in the keynote today, you shouldn't really be afraid of robots They're gonna help us test our products and make sure everything's better So we're gonna start with the introduction I'm Riaz. I'm a iOS engineer working on our product. Hi, and I'm Lulua. I work as a test engineer at Sharp Keep and mainly testing iOS native apps So this is shop keep we are a iPad point-of-sale system as you could see That's our app running on the iPad over there And we also have a card reader a cash register and a printer right there So those are different type of hardware integrations that you might see on our system So we have our point-of-sale. We take payments and we have a business intelligence side of our product Primarily we focus on the iPad and integrating new hardware into our system So this is a picture of our screen that our merchants see as you can see they could have different items I think a ring up for a sale When they bring stuff up, they could be any number of items There can be any number of names and then we have to make sure everything works properly So let's go into the problem Sure So as my friend here said Shop keep what we make is an iPad register application and it is a point-of-sale application And the main thing that our customers really look forward to in our app and what our app is really used for is taking payments That's literally the main heart of a point-of-sale sale app so clearly we touch the money and that is how the need arose to test the credit card payments and Making sure that we take the payments correctly We take credit cards correctly the way they will be payment is processed the right way and we're charging customers the way They're supposed to be charged and as part of this testing effort We were spending hours just doing laborious effort, which was just doing swipes And we were like we're wasting so much time just doing manual swipe. There has to be a better way to test this Our automated test suite initially before we added the solution we're doing test cases with It was simulator based and we were actually missing the hardware integration part of doing the actual swipe and This limitation was a big limitation for us because like I said being in the point-of-sale business We had to make sure the money that we were touching was always Taken correctly and that's why we were like we need to find a solution to test credit card readers So possible solutions we came up with was Stubbing or mocking the card readers in code like okay This is safe You just mark the card reader part and then you go ahead and finish off your transaction But clearly that was not the right approach if you guys heard the keynotes Speak today. They also said sometimes laws require you to even do a test process The way an actual human would go through the process and that would stubbing would not be and the way a normal Flow or for normal credit flow goes through the whole process So that would not be an end-to-end test the other option was hiring a bunch of work abyss that runs a bunch of tests which were people like me where we would sit there and just continuously be swiping cards and That was the make-do solution we were using at that time But that was not really the solution that we wanted to go ahead with So that one was out the next option was accessing the card reader's API directly and making calls to the API to make sure the card transactions were swiped in and like that and in an ideal scenario in an ideal word that would be perfect But card readers unfortunately have not reached that point where they could interact with our application and provide that API for us to Finish that transaction Reliably so again that didn't work for us Unit testing that was another approach we took and that for the most part in our code did work using unit tests But unit tests like they say are just testing units of code It was not an end-to-end test and with being being in testing you sometimes do need integration test You need system level test you need test that actually touch each part of your hardware the way we want to so that was not an approach We were gonna go ahead with so what we were left with was automating swipes with a robot We're like this sounds like a viable option. We don't know how much time and effort it needs, but let's look into this So that's what we went ahead with All right So now we'll go into the process and to start you off We will have a little video of showing how we built our robot and then we'll go into it further It was just some techie music going on in the background, so you're not really missing out on much That is the card reader that we used for automation This is called a stepper motor, so it's something that rotates around This video was actually shot in like the the workshop of a basement in One of our colleagues actually built this whole set up in house So it was pretty cool the first time we thought and we were pretty psyched about recording the whole process So as you can see Our robot really doesn't require a lot of like Crazy equipment or anything to get started with these are things you go to your I guess home depot in the States I don't know what you guys have here, but you just buy equipment and then you could get started making your own robots So as you saw earlier in the keynote, this is Arduino and it lets us control Our stepper motor from our computers. So this is just assembling like the platform. We're gonna build our robot We actually designed it so that they could be stackable So we plan for in the future. We have multiple robots doing different tests all at once So we just made sure that we'll be able to stack them on top of each other This video is basically just listing all the different parts that we use in our Robot creation. We also have a YouTube video that would show you step by step as to all the different parts We use and how we Assemble the whole whole robot together. So you guys do need to read code rerun through this video or Visit this again to see all the other parts that we've used We will share the link to the YouTube video And we're wondering how much everything cost. I think the total cost was under I think around 80 US dollars So it's not too expensive to get everything set up I actually think the most expensive parts were probably the stepper motor and driver, but I'm sure if you need something a little bit weaker, which is a little bit cheaper. You can do that as well And also a lot of the starter kits that you come with arduinos actually come with servos and things like that So you can actually get started with just a starter kit, which runs about 30 dollars Yeah, so this is just going through the wiring the different parts together just a brief overview We used a breadboard for a lot of prototyping because we actually didn't know what would work and what didn't work So breadboards are really great for that you're able to experiment with different different setups and As you can see we hot glued it on once we knew it worked probably not the best setup, but it's it worked for us Okay So why a robot so we wanted something to give us and end-to-end system level testing Which we didn't have with our old code We also needed something to be very reliable because we needed to pass nightly regressions we also wanted to be really budget friendly because knowing our organization Really knew what we were getting into and we wanted to prove that we can do something and it wouldn't blow our budget And we wanted to be really easy to use and set up because We had different offices and if this really took off we wanted to be able to replicate it in each place So every qa team would have their own robot So where this idea first came from was that our company has hackathons and a person started an arm-based robot and Although it really lacked the durability to run through a lot of swipes It also couldn't really the hold the card properly through the credit card machines But then that got our gears turning and we needed We knew we needed a robot So we also we needed something that could run every night without fail To be sure to make sure our test ran properly We also need something that could run multiple swipes in succession Because as our merchants know credit cards don't always work the first time you swipe Sometimes it takes multiple swipes and we needed to replicate that in a testing environment And this was a huge unknown for us. So we had to ensure our prototype was very low cost So building the robot you saw in our music list video We used off-the-shelf parts and supplies that could be fine in your local hardware stores There's nothing really extravagant that you need to do at least for our robot So you saw it just had plywood and then we had different parts to it So the stepper motor the motor driver a power supply and Arduino We had our credit card reader and we had a drawer slide and we used the if you saw the circular crank plate So it turned the stepper motor circular motion into a linear motion to swipe the card So this is the setup like you saw we have the mac mini on the top right the Arduino right under it There's a breadboard to the left of the Arduino. Then we have the stepper driver Thanks to the breadboard the the stepper motor in the bottom left and the power supply on the top left So this is just how a stepper motor works just a little mechanics for you guys as you can see it uh It activates magnets that are paralleling each other in very quick succession So that's how it generates the circular motion If you see the shaft in the middle is turning according to the magnets This happens many many many times a minute. So it it's actually pretty fast. This is pretty slow um, and as you can see this is the card reader going on and on and on and swiping the credit card By far, this is literally my favorite gif and everybody in the company just feels so satisfied looking at this gif right here All right, so running the server. So at first we wrote Arduino scripts to prototype just to see how fast things go if it could actually work and Then we had to see how we could actually integrate it with apium So we chose to use johnny 5 so johnny 5 is a javascript robot robotics framework We can control the Arduino and through extension the stepper motor through running javascript code and being a javascript framework We could write both robot logic and the server logic in one code base Although from the talk earlier today, it looks like apium has robotic robotic integrations now So we'll see what happens in the future. But all it takes to swipe is You just have to start the server and send a post request So this is some code. It's a very simple app. So it's a express app. So you create express app We're looking on port 3000. We also create johnny 5 and we have So we start to create a new board. We also instantiate a stepper when the board is ready And the stepper type is driver. We say there's 200 steps per revolution And then we're telling it that the pins to look for are 6 and 7 for the direction and the amount of times it should turn And then these are the rest commands So the one that we care about is the post to the swipe endpoint And so our function just basically goes through and checks to make sure everything is set up properly to make sure we have a board To make sure the stepper is instantiated properly And then we have a timeout inside the function to make sure that If the swipe didn't return to success in the in the amount of time that we expected to then we make sure we tell it that It it failed And then you can see the stepper we tell it the rotations permitted to 600. That's how fast it goes We tell it clockwise counterclockwise can also be clockwise and we tell it to take 2000 steps, which is uh half a rotation Cool, and then we go on to the solution All right, so where does appium fit into all this? Why are we talking about creating a robot at an appium conference? uh First and foremost, we use ios simulators We use android emulators also as somebody had asked me earlier But this project we started off using ios and that's why we'd be explaining the ios Set of things in more detail. So using ios simulators We connected to these network card readers the ingenico ipb 320 that we showed earlier that is a network card reader that just plugs into your land and You see it's available online once your ios simulators on the same subnet as as the card reader So you use that to make a connection We use xcu i table elements We connect the simulator to the device under test in this case the card reader The next step is starting a node server And once you start the node server is as simple as using just a ruby gem We are a ruby shop So all our appium code is written in ruby And all you need is the rest client ruby gem and you just send a post request to the web server once it's up Um, there were a few things that came up while we were setting up once we had the prototype and we actually did the appium integration We realized that we needed to adjust the speed of the swipes because as far as the swipe was being uh pushed by by the By the node server our our card reader was not recognizing it as a proper human swipe It was too fast So we had to make sure we had to make sure we adjust that speed which was done using the step rotation and the step rpm There were also swipe retries that we had to add into code because sometimes like We are set you swipe once it doesn't go through and you realize that oh, it did not go through you swipe again So we had to add in handling and functionality like that um These are basically all the technologies that we used in addition Which integrated with appium to make this robot uh come to a like an end-to-end test case Uh, these are the appium capabilities as most of us who already know appium These are literally just the basic level capabilities. You're just creating a device. Um, give it any name Um, our ios is the platform name is ios Platform version is whatever ios version you want to run it on our automation name would be xe or a test This is where our our register application resides. So you just give the path of the application and a timeout It's literally as simple as that to set it up in appium So appium accessibility id I am a strong believer that That any automation that's done in appium can sustain only if your accessibility ids are are coded correctly and You need to make sure as a tester that you you put that push across and make sure Developers know that this is what we rely on and this is what we need as an organization to have strong reliable testing And luckily for us we didn't have to push through that much But every element you see in your had an accessibility id which made coding so much easier Connecting to the card reader from the screen. This is basically a table element screen Connecting the card reader. We had ids Recognizing what seal number it is you find the card number with the seal number connect to it Checking if it is connected even the check mark you had an accessibility id And we wait for the check mark to come on which shows us that okay We are now connected to the card reader. So each step each verification step became so much easier because we had accessibility ids for each and every element in the app that is a sample Code for how we used our sorry our Our credit card payment screen in here we in for this one We use the swipe card on ipp for this particular example when we started we only had swipe cards We've now gone forward and we actually have another robot for doing chip inserts as well Which i'll probably speak about in our conclusion But for now the swipe card all it literally needed it was wait for the accessibility id to see whether we're ready to swipe card on our screen Give it the url since our Server was hosted on the same mac mini as the as where the script was running We literally just had to do local host give it both 3000 that was hard-coded and just save swipe And that was it that was literally all that you had to pass in with rest line and do a swipe post command And that that's how we took care of swipe We added retries saying that if it didn't go through you until we go through next screen you keep trying to retry your swipes Going forward as i said we have used raspberry pi now instead of the rdu no server and that has insert card remove card and the whole Functionality which does an insert card waits for it to be done and then removes the card, but we'll speak about that in detail later This is an end-to-end test of how the application works. I'll run you through it real quick Literally just use the command line. We normally use Jenkins for automation There would be a Jenkins call, but just for this case This is a this was a live video of this card reader and i'll let you guys know when When this simulator is ready it will go in and post the swipe at the same time. So it's just Installing the application right here the web driver agent Once it installs the app it'll activate the register with whatever test store we pass it in put in the password Once the register activated you sign into your to your test store you open your shift There is another setup for receipt settings. Oh, sorry. No before that We connected the rat we pass in the card Reader serial number you wait for it to be connected once it is connected You go back and now we're doing some other receipt settings in the background Which is why there will be a little bit of a delay. You're getting updates for those receipt settings And just choose an item from the screen in this case. We're getting a slice of pizza Um, you hit credit as soon as this is you'll see that the swipe happens right there Once the swipe happened it's taking a little bit to Read the swipe once that happened it asked you for a tip saying the the payment did go through And then once you are on the tip screen You just select you whether you want to whatever tip you want to put in I guess somebody's taking longer to decide how much tip to put in No tip. All right, and then you're just authorizing that transaction and signing and Getting a receipt. So that is what an end-to-end test for shop keep looks like And you saw how little of a part the swipe was but that was literally limiting our entire test case to go from one end to the other and Just adding this swipe automation robot made it so much easier for us to complete our testing So conclusion So The extension is to automate the app. Um, so we have our chip card automation and contactless card automation As luah mentioned and you can see below. This is our chip card automation that has been completed since making these slides So this is one is actually using a raspberry pi instead of the arduino and we have a Python flash server that can talk directly to the That what's called a linear actuator right over there and all it does is inserts the card and removes the card It's a pretty simple robot, but it also Extends our testing and then we also want to do a contactless card automation Before we had a really low tech solution. I'm just taping a card onto a card reader But we want to do something where we can lower a card and then raise a card when we need to We also Had easy repeatability to cover different credit card processors and readers So now we have multiple multiple setups with different readers and processors A lot of different integrations and now we can easily cover all of them with different robots And then we have reduced manual testing effort to boost intelligent testing effort So there's less of us going like this and there's more people focusing on how to write better tests and how to write better Appium tests in in particular Um, so the major gains we had so as I said before we have easy tests reproducibility um Every test can be run multiple times without us expanding more effort Um, we also actually have data collection for the first time with our tests before as maybe you guys know when you do manual tests I don't know if you write down every single conclusion But now we actually have data coming in and we can tell Where things are failing more or less over time. Um, we also you can see launch times We have the we have charts that show it over time We have card reader connection times and then credit processing time So how long do external factors weigh into our systems? And then we also have a something called a you all call a stool So we have this little card that you can program to be different cards So with just one setup you're able to test multiple card brands and card processors and even different ways of card payments This really came in handy because we actually had production bugs at one time Where some uh card readers would not take some particular brand type cards And you wouldn't know this because when you're doing testing in house, you're not testing every single card type on every single reader So the only time you would know of a bug like this is when it's already out in production and customers complain like Oh, we can't use this test card and you don't when you're taking payments You cannot afford to take a risk that big and that's why this reprogram of card types letting us run on different card readers was a big gain for us So it is open source. You could take a look at it. It's a github.com slash obv slash mumblebee And we want you guys to think how you can build your own robot and integrate it with apium So As you guys saw in the keynote before He has a pretty sweet setup But i'm sure that costs thousands and thousands of dollars hours costs under a hundred dollars And we're able to get hardware level integrations with apium Um, so you should guys should think about what limits your software based automation So for us it was actually in using a card reader and Seeing what a merchant actually feels when they swipe a card So we actually didn't know what our users were doing like all the time We weren't testing like our users would we were Thinking that we were but we actually weren't but now we have the robot We can actually test like we are a user And i want you to think about like what robot you can create to actually integrate with apium another Side note in our conclusion what we gained out of this project Which we had not initially thought of but now we have remote access to all these card readers and credit card readers So anytime i want to do a test, but i'm not in the office I'm like oh, I need to do a quick signing check. We have a new build out and i'm either working remotely Or i'm not physically in the office You can literally just log in to the mac mini use a simulator and do an actual credit card swipe and This was a plus that we we got we didn't really think of it Initially when we were making the robot as to this would be something we would gain out of it But now a lot of engineers we use the same hardware everybody doesn't need their own individual hardware You just log in remotely and do a swipe and we we gain that additional Reusability factor as well Thank you guys so much for listening any questions Right Well, so for this particular card type when we when we do swipe transactions We don't need to enter a cvv that usually the the pin code that we need to answer is usually on chip transactions but um, we have not gotten to looking at at Jason's keynote speech today I feel like I might borrow his his robot to press on keys But as of right now the swipes do not do not take in any any numbers Pressing like they we don't enter a cvv or pin code or For that Yeah, like the way we take cards In um in the states if you do a swipe transaction, you don't need to enter a cvv It's only for chip insert that we need to enter up in code. Okay. Uh for for the chip insert, uh, uh, I've just done that to uh I've guys done uh automation to enter the cvv as well So it's not really on the card reader But what happens in the application? We also have a text box that comes in sometimes when you have to enter a code We can enter that through uh the application So that is hardcoded through aptm as well Where it just gives your text box to enter the cvv and then the card is accepted But we don't have that capability or we haven't automated that on the card reader Okay, uh, of course, uh, my thought was uh, uh Me asking a cvv of you, uh, it's not allowed Oh, it's not allowed here. Okay. Yeah, we do not have that limitation fortunately for us That's why we haven't tested that scenario. But now that you say that Um, that that could be something additional that we add into it in India. Oh, we In India, we can't even ask the number of the card Plus uh, plus cvv. Okay. All right. Yeah, cool Yeah, we've not had that limitation with our automation, but going forward. That's a good point you bring up Actually, I don't have a doubt basically I saw your product at uh, and It is quite amazing things you have implemented But I was thinking like if you go to a pub or somewhere. So there you have They have audio box. There will be multiple cassettes will be there And if you will select one of the Cassettes then it will select particular cassette then after it will start playing that song, right? So, uh, right now You are swiping only one type of card So I think You guys can move ahead with that option also like multiple types of card changing the card should be there and whenever The one card is swiped already So it will go inside the queue and next card will come up with the next line So that can that way one machine will be used for all the type of the cards and we can test the multiple type of cards is getting accepted with the Card reader I think that that is a pretty good idea. So I'm not sure whether how that will work, but I think that will work properly Yeah Another fun fact for you guys So none of us We were three of us that worked on this project and neither of us had any mechanical background Electrical background. It was literally just looking at prototypes looking at Block force and coming up with ideas. So ideas like that are great like that helps us Broaden our thinking and come up with new innovative ideas of of testing So one solution for that was what riyaz mentioned in in his conclusion, we have this uh We have a programmable card called We use this tool called the ui callus tool and with that the same physical card that you use it's connected to Uh, a software and you can program that card to be a different card type each type like one one time It's a visa type card one time. It's a master card one time. It's a discover So it does that right now, but yeah, that's a that's a great future enhancement for us as well anyone else Have you done any security testing on this like, uh, so there are like chances of hacking The cards. So how do you ensure that like while entering? So you told like there is no pin based automation so far So when you just swipe the money gets detected from the customer or it's like, uh, how do you ensure that kind of scenarios? So in the testing environment is totally separate from our production environment In the test environment everything's on staging. Um, all our processors and gateways knows that it's a testing environment So no real money is exchanged. Okay, so we're not really too worried about People That is set up for in-house testing for point of sale application or that is a staging uh environment So it's it's different from what a production environment is. So like we have said that it's it's not real money that we're charging Okay, okay, thanks All right guys pump will be as we call it in our in-house project is an open source Uh tool so please go ahead check out and if you have any questions our information is on the slides as well Yeah, it's really simple. I hope you guys go out there and build some robots. It's pretty fun Thank you