 So today we're going to talk about camera testing from Java developers perspective, so post-tron I are Java developers at PayPal, and sometimes the feature we created for PayPal will affect the terms of flows, and we need to write hundreds of end-to-end test cases to cover these flows to make sure nothing is broken. So there is a Java automation framework called C-Lion, and also we are Java developers. We don't really like that framework because if we do write the test cases in Java, and Java is a compile language, so you have to make a change, compile it, and run it, so you cannot really add a breakpoint somewhere and just add one line of code to see how it works and continue running the code. It's not so easy for debugging, and we want something that is a rebel, and another thing is Java code is not so reviewable by the protagonist. So they create these user stories for us, and we want to have something that is like English, so they can review these test cases to see if the feature is properly delivered, and nothing is Java is kind of verbose for testing purpose, so you need a proper IDE to write Java code. We don't really want to have IDE when you write test cases, so because of these reasons we decided to use Ruby with Kavibara to help us on the functional testing. Just in case if you don't know what is Kavibara, this is Kavibara. It's about this big, and they are very heavy. Yeah, I read it on Wikipedia, I think they are crap, and here is a picture of two Kavibaras hugging each other, and here is a picture of Kavibara hugging a cat. So what is Kavibara actually? So basically Kavibara is a Ruby library that you can use to simulate how an end user interacts with your web application. So here are some examples. So if you want to open up a browser and visit some URL, you can use the visit command here, and you can also click on the links by using the click command. And you can also fill in text boxes with Kavibara and interact with radio buttons, select boxes. They are very convenient, they spot all kinds of user operations, and you can query if the web page has some element. So you can add this page, has content, something at the end of the test case to verify if the test case is successful. And then the domain specific language we are using here is RSPECT. It is a behavioral driven development tool for Ruby. We also use Cucumber in some other projects, but mostly we are using RSPECT for our test cases. And here is an example of how RSPECT plus Kavibara script looks like. So first we have something called a feature. So in this example, the feature is user login flow, and inside the feature we can define multiple scenarios. So the scenarios I have here is, as a user, I should be able to login to my account. So these scenarios are like the user stories created by product owners, and inside the scenario we define the steps to verify the scenario. So first, because what we need to do is to do a login, right? So first we open up the PayPal website and click on login and fill in the email address and password, and click on login again. And after you successfully logged in, we will see a welcome message, so then the test case passes. So if you look at this script, it's just like English, and it's very clear, it's very simple. Even the product owners who doesn't really code, they can understand what this means, and they can use this as acceptance criteria for the features they want us to deliver. So I just run a demo of what this script is doing. So first it opens up the browser and visiting the PayPal webpage and clicks on login and fill in the password, and after successfully logged in, after we see the welcome message, the test case passes, just like this. Okay, so then we created this thing called Autobot. So what is Autobot? It's basically an aspect with Capybara plus PayPal capabilities, and it's also Rapples, so it's very easy to create new test cases, debugging test cases with our Autobot framework. So Autobot basically has four parts. The first one is acceptance. So basically in acceptance, you can define the flows and steps of the test cases, like how you're going to do express checkout, or how you're going to do onboarding, stuff like this. And the next part is seed files. So for seed files, the test data we have, they are things like test accounts and expected contents are stored in the seed files. And for different countries, we have different seed files. So we can use the same test script for different countries just by selecting different set of seed files. And then we add a lot of PayPal-specific helper functions, functions like to help you to log into your PayPal account to edit your PayPal account and some admin operations. And we also add support for REST, RPC and database operations. So it is very easy to call the PayPal REST APIs with Autobot, and you can insert, validate and update your database throughout the world easily. Hi. Now I'm going to show you some demos to show you why we really love Tabibara. So for the first video, here I will show you how we used to write tests in Java. So this is a code in Sela and our PayPal framework. So as you can see, this is a very complicated looking piece of code, but actually that's very simple things. It's just go to PayPal website, check for some elements and then interact with the website. Okay. So now let's run the code to see how it goes. So as you can see, similar to Tabibara, it also opens a website and then you will try to visit the URL and interact with the page. So it looks like the test failed, it doesn't do anything because I forgot to click on the login button. So now I need to get the ID of the button and then stop the test, go back to the code, add the declaration for the login button and then after that, I will need to go down and add the code that actually clicks on the button. Now that I fixed the code, I will run it again and hope that it will work. Let's see. Okay, so it opens the page, but it looks like it still doesn't do anything. It looks like I added the click on button in the wrong place. I checked for the content before I click on button. So now again, I need to stop the test, move the login button piece of code around and then actually I need to add a light way for the button to show up. Now that I hope I fixed that, I need to run it again, hope that it works this time. Is it like the compilation? Yeah, it's compile and then run. But the light between the screen and the browser? Actually, I think it waits for the browser to be ready. I'm not sure what it does. Okay, so it looks like the test passed. Okay, so now that you see how we used to do it in Java, now I will show you how we write and debug our code with Autobots. So here I will do the same thing. I have the PayPal website. Then the test will fill in the email and password. So here's the code. And yeah, it will try to do the same thing as the Java version. So now I will try to run the code. And you can see it opens a new browser. And again, I discovered that the test failed because I forgot to click on the login button. So now what I'm going to do is I go back to the code. And I will add a breakpoint to the code. I will try to fix it interactively. So now I save the code and run it again. It will stop at the light by not trying. Yeah, and it stopped here. Then now I can experiment with the code. I can try to click on the login button. Okay, so it looks good. Now I can continue with the test case. So you will see the improvements. So first of all, in Java, it's very verbose. You need to have a lot of code to defy all the page elements. And then very long methods to interact with the elements. And also another thing is in Capybara, we have the repo in which I can test and write the code interactively. So it makes me, it costs less time to fix the code. Okay, so another feature that we add to Autobot is this one which I'm going to show. So this thing is quite easy when you have a simple page, but it gets more complicated when you use a lot of Ajax and your JavaScript goes serve some web server and the server send back some data. So it's really hard to know what's going on unless you can see all the HTTP requests and response. So to deal with this, we use Firebug, which is a plug-in for Firefox. So now to enable Firebug, I just need to add this one line, which requires drivers Firefox.rb and then now when I run the code, it will show a browser and you can see Firebug is enabled, it's the orange thing in the left, in the right corner. And below you can see all the HTTP requests and response of the page. This was very helpful in one of our projects where we need to test that the JavaScript in the page actually send a request to one of our checking servers. So with Firebug, we can export all this information into a JSON file and then we can use that information to test if the JavaScript actually does the right thing. So to sum up why we love Kavira, the first thing is the repo. As you can see, it helps us to write code very easily. And the next thing is really easy and fast to write. In Kavira, there's only a few comments like visit, click on the page to have some elements. So it's really easy to write. You don't need some fancy IDE to write the code. And the last thing is very readable. So that concludes our presentation. Thank you for listening. And now it's your turn. Actually the script I have written is actually that script will set up the Firefox so it adds the Flux in to the... So it's a custom script? Yes, it's my script. Actually you can Google that, but it's not very... Sorry, I've noticed that all the tests were running against the live takeout. Do you guys also use Kati Bharat to do like three shipping automated tests? So usually in PayPal, we test in our test environment. Some machines that run a version of the PayPal website should be updated. So we test against that. We do not test against the real PayPal website. Have you guys tried using building custom functions for a category to... Like if you've got features that you regularly test so you can just plug in one line into your category test and that feature will always be tested the same way instead of you having to write out the tests every single time. Do you guys have experience with that in your setup in PayPal or have you gotten that far enough? Yes, we did that. So we have some basic page like Login which most of the test gates go through. So we have methods for that. And for something like checkout flow is very common in PayPal so we also have some helpful functions for that. Do you make use of logging very much along with your Kati Bharat? So besides the feedback with Kati Bharat you can also look at your logs. I mean coming from Java, I assume you guys are interested in looking at rings of logs for such long time. It's very helpful for helping with debugging and figuring out what we want. As an alternative to the Firefox plugin that you just demonstrated which is very unique by the way. Thank you. So in February we have the screenshot. It automatically takes the screenshot where it fails so we buried up. We mostly rely on that. Another thing is for me I don't want to read too many logs so I write my own logs. So for most of the helpful functions I write some simple steps like it goes here, it goes here, it goes here but it's not very full, too many logs. It's just basic logging to see. So we do have some product owners who are reviewing this. They didn't write this kind of test but they can review this as the case is to see if the feature they want are properly delivered. But they don't write this test. We use Kati Bharat in test. See if we can get some genuine log JavaScript because the problem with see if we can get it set is that it gets to be logged into JavaScript sometimes you need to worry about something that will happen before running. Like in the case that you should show that if you wait for the page to load you need to go to the next page but in single page distribution all you do is wait for something to do. When you click a button, a dialog opens instead of going to the next page can you do that? I think it's very, very hard to do with that. There's another code waiting for Ajax. Yeah, it says wait for Ajax and actually when you do with just most of them you just need to wait for some text to power up that's what the user sees. Basically when you test with Kavel you try to simulate how the user interacts with the page so most likely when somebody shows up then you just need to wait for that element to be present on the page so that's the basic usage of Kavel Kavel can inject JavaScript you can run some JavaScript code inside the browser so if your code has something like when you done loading some data you set some variables and in your Kavel you can check Also for a follow-up to the non-technical users really code Ruby has the advantage of being so convenient to do it nicely with business analysts maybe not the end owner of the product and the business person that suits the way up top but they agree with the other ones you might probably have business analysts that read most of our code when they're confirming that things are working because Ruby is so nice they can follow it as long as we use the right abstractions so I think it's a good thing to think about that Are you agreeing with it here? So basically you have a business analyst that is sufficiently frustrated with the app and he finds that the code is actually easy to read and he often points you in the right direction how to fix it I'm not recommending that you frustrated then that's bad I agree with the guy who lives in here like we have our business analysts that actually reads our source code sometimes a lot of times it's when someone reports a bug and she can't see the new layer like another suspect or something she'll read and find that there's an in-conditioner blocking it and you're like hey, why do you write an in-state block it's sufficiently readable enough to read if something's happening, block this I think that's the wrong business logic so she helps us a lot by reading our source code try that with your 5-3 hands I find it interesting that you test for the JavaScript to make a call to your server using your feature test I'm curious why you chose to use a feature test to test for that instead of maybe writing like a Jasmine spec to make sure that at a unit level you're making that call when that function executes so you mean why don't we write a unit test to test the JavaScript yeah, to make sure that your function is actually calling out to okay, let me rewind back so when I think of what a feature test is doing it's trying to drive interaction with the page and something like testing for an HTTP call seems like a unit level test and instead maybe you'd want a test to see like can the user interact with the page that I would expect that they could and so I'm curious as to why you're testing for the call to be made and the response to come in instead of like maybe an interaction the thing is for us we don't own that piece of code so it's hard for us to try that unit test for that but we still need to make sure that it works so we need to try that so you can't write the unit test because we don't own some team are responsible for some piece of code so we cannot do whatever we want with those pieces gotcha okay, I'm going to wrap up the discussion at this point for now but if we have more questions please approach the speakers later on afterwards would you be willing to share the slides and for the sake of those of you who couldn't text you the call okay, sure I'll talk to you guys but maybe that would be helpful for us to make the unit test thank you so much