 I'm ten years in quality, five years plus in test automation and lately as I'm working in small startup I'm doing everything so I'm like developer, bit of DevOps working with deployment and observability concepts so I'm desk dev ops whatever guy and I really like this kind of multi-heath personalities that I have and I'm a big fan of serverless in later stages and in free time I enjoy climbing which is totally sucks if you live in Amsterdam and totally flat country so I'm going to Ardennes and Alps all the time and I like Belgian beer too much sometimes delirium still have yeah some influence on my speech let's say okay move on so what's wrong about Selenium the question is that I already heard there's nothing wrong with Selenium Selenium is like 15 years of history lots of contributors lots of yeah open source different projects like night watch and yeah wave driver IO and stuff so I'm not advocating against Selenium I just want to show you guys what else is available for the choice when I joined my recent company I was selecting where to go like which way to go and we have a lot of GS code base and yes of developers so so okay let's see what kind of available on GS market for the test automation except Selenium so this is like three choices that I had back then the Puppeteer do you guys know about Puppeteer use it okay I think that is great tool but it's not test automation tool it's browser automation tools it's good for like making kind of spies and crawlers and stuff like that it's done by Google guys to test the Chrome itself so yeah very powerful but not very easy to use would say the second one is test cafe someone use test cafe okay not so much it used to be very kind of big and commercial framework they open source it also recently it's done by developers and quality automation engineers for quality automation engineers so it contains a lot of key oriented topics and key related kind of things that might narrow choices that you can have and the last one is Cypress like then it was like 0.2 version or something so it I I made really a bet to go with it and I really don't regret for now so what Cypress is is a yeah test automation framework JavaScript you can also write TypeScript of course it's an electron runner and debugger application bundled and also it's a SaaS platform for balancing and test recording so it's kind of thing in itself for later you you have to pay a bit if you want to use it on the scale but for a free open source project is free and you don't need to care worry about your a Selenium grid cluster and stuff like that so it's all balanced on the south side white Cypress for me I was like one key guy in the company and in my previous company I also want one was one key guy in 80 developers and I wanted to scale you I wanted not only be a bottleneck so whether I was choosing the framework or the tool or whatever choices I do in my professional career I think about who gonna use it and most of the cases is going to be developers so it should be developer friendly that is decision number one that I made and second one in Cypress you can go up and down and test Pyramid so you can write a unit test API integration test and end-to-end test in one tool or one framework so you don't need to maintain multiple code bases or multiple paradigms when you switch so what's endeavor in oriented kind of what I think is the oriented it contains a lot of tools already bundled already pre-packaged so Cypress tests are basically mocha tests so various familiar syntaxes for everyone this chai assertions with lots of other like libraries that very common in JavaScript landscape it contains already reporters out of the box with multiple tools supported like junit reporter mocha reporter CLI reporter Jason whatever so you can plug it into your reporting tools or portals to get information about your test runs it's extendable play plugins so I know that guys were complaining that I we don't have gherkin bindings or cucumber bindings for Cypress so someone wrote the plugins that now you can use the gherkin language for tests you can write you can add your stuff there it's very easy to sit up for your project it's CI ready so by that I mean that we have around 20 plus examples for almost all of the CI tools to integrate Cypress with it's either like you know the club drone cloud or circle CI or Travis it has extended debugging capabilities you can inspect your test really well because your test is basically running inside of your browser so you have access to everything inside of the dome inside of the browser yeah spice stops and clocks if you want to mock your function for unit tests with the spice or stops if you want to control the time and you can control the time in Cypress for for example you have like annoying for seconds spinner when user waiting for something to happen in tests if you have like more than 100 tests already 400 seconds that your test run is just delayed so with the Cypress you can control the time say okay now last four seconds and this will be carried on in the browser and in the test so can control the time can fast forward for this kind of animation purposes and it contains automatic weights so in Selenium you have to assert if element is present if not present weight weight weight so it's kind of a lot of synchronous code in Cypress weights are implicit so if you do any kind of action on elements it's bill wait for default time out with this element to be present on the page and that's why there is no like stale element exceptions as you have in Selenium so less flakiness in this way so talking about different layers unit integration and end-to-end layers go a little bit in depth there so this how the test looked like as I said mocha syntax you describe your test case before each of test execution you visit the page you visit index exchange HTML then you start a server the mocha server and you say that for any request that application will try to make for this route a post request I will return to 100% and you make lias as upload function then you prepare your test file because you want to say test up file upload and on the file upload elements you trigger change with this file and then you just validate that the route was called so in this way you kind of yeah making start backends calls for API tests or end-to-end tests you can either go through UI completely so in this case the example is user who is not authorized try to visit the dashboard page and we want to validate that unauthorized users should be redirected to other page with in this case text containing you are not logged in and the URL should can have unauthorized so this is example number one when you actually hitting the web page visiting loading everything this rendering everything and checking that you will be redirected so it's kind of slow and be fragile because it's a certain on the text on the page so you can do it's in the API level so you can use execute the request so CI request is API call without loading anything you just executing call to the dashboard page then you expect that response will be 302 so now authorized and you also expect that URL will contain our unauthorized so you have two options either go through completely UI or go through API and I do a lot of API parts in my test for seeding like user login authorization preparing some test data and stuff like that and last it's end-to-end test here I will show some demo so we organized everything that we have on our web pages in page objects so we have selectors on the page you have methods that you can do on the page like login register do something so it's bundled in page objects then we split our user flow in parts so we have user registering user login user making offer user applying for this offer and using signing in so this is sub flows that contains pages inside and the big flow is grouping everything I came as this idea because I didn't want to make code duplication and and I wanted to have tests very very like short simple instead of repeating everything like every time I login or every time I create the user I want to have it in tests like explicit implicit so I kind of over abstract sheet out of my test framework and I didn't like it now because it's grew out of proportion we were five developers when I joined and now is 30 and I will show you on example how it looked like can you see on the back so anyway this is a test so first I prepare lots of test data that will control the test execution so I say okay this is user parameters different different user parameters arguments then the before execution of the test I load all of the data and then I run everything so when this test fails and you have to debug it as a developer you're gonna come to me and say probably hate you and you will be right because it's completely not clear what happens right so the test fails and then we open it first okay you have this flow and then you go inside of flow and there is another nested things and you go there as okay again another something nested and finally you arrive at the point when there is an assertion so assertions were also outside of the test anti-pattern and think this gentleman asked me what you should not do you should not make assertions is outside of your test I think so this is not clear not good at least for me and this is how we're doing it right now so tests are quite long quite explicit so you can see on every page what you're doing so it's easier to debug if it fails it's very long it's like hundred lines and stuff so it's a lot of code application but sometimes it's okay to make it if you want to increase readability of the test so yeah that's that's kind of how we do it in the new 10 and few of my suggestions to you guys Cyprus of course have some trade-offs it's not a silver bullet whatsoever so first trade-off and number one the biggest one for most of the people it's only Chrome browser so you cannot test multi browsers cannot test Firefox or Internet Explorer or Safari with the Cyprus lately in last few years browsers go closer and closer in APIs to each other so I kind of trust that in 95% of cases it's gonna be the same so and the other 5% of cases yes will be yeah different but then you can also get it from your monitoring login or or your users so it's only Chrome it's test automation tool is not Swiss Army knife so don't use Cyprus for us a lot of people use Puppeteer for like scrolling scanning it's not a API tool as well so you cannot replace all of the API testing with Cyprus and of course it's not a performance testing tool it's runs in single browser on the single tab so you have to if you have messaging applications at like I know Facebook or whatever when you have user a user be talking with each other and inside of one test this will not be possible so you cannot make multi user tests in in Cyprus and it's works on the same super domain pages if inside of scope of one test so in one test you can use star.google.com but you can use star.google.com and star.facebook.com so you cannot visit super domains in one test so yeah strategies common strategies with Cyprus you have to define you want to stop backend or not stop backend you want to test completely from end-to-end or you can use it you can use the API tests what I do I have few smoke tests which are completely end-to-end not mocked not stopped and rest I stopped out or mocked out. Tests are deterministic means that you don't you never should have the casing or conditions in your test it means that you don't know what you're doing if you expecting in some cases element A or element B presence that means that you not to not control your application state and Cyprus allow you to control complication state completely. Tests are should be independent so you should not create a test that first login and then another test is that do something on the page so because if first fails test fails second will fail because user is not logged in so tests supposed to be completely independent from each other Cyprus doesn't have X path locators capability so for someone is bad for me it's okay I'd never like to express and I think no one likes you shouldn't test third parties as you don't control third parties so again example of login page if you log in this Facebook don't go and test how the Facebook works just mock it out or stop it up and it's fine to have multiple assertions in one test so it's fine to visit the page and check everything on the page you don't need to as an unit test have granular test per assertion it's all right it's kind of also second time so that's test strategies and as I said in past year was testing dev room now it's testing the room it's not present so I have to say it from face of all of the testers guys whatever you do whatever you choose whatever framework you choose just go and write test testing is great it gives you confidence in your code it's give you confidence when you refactor stuff so just write us that's it okay in question time well really we have only two microphones yeah so for my company I've been doing some Cyprus test we're quite new at doing Cyprus test so basically we started from nothing but what I noticed what was kind of a problem is that when you decide not to stop and you want to do end-to-end testing you run into problems when for example you need to check whether a file has been downloaded or you want to upload a file that really breaks everything because if you download something for example you go out of the DOM so Cyprus gets super confused I was wondering maybe you have some solutions for that or but you're checking that file is actually downloaded on the back-end side yeah for example if you have something in the back-end that changes something in the UI but as well download something that like breaks the code completely so I was wondering maybe you have a solution for that yeah for for this kind of cases they're trying to use a Cyprus exec so you can execute some kind of yeah npm not functions outside of Cyprus so outside of the browser so then you can do all of this like back-end checks and things like that so it might help but yeah don't trust back-end you know don't trust anyone yeah I'm key I shouldn't trust so more questions yeah hey when do you usually run your end-to-end test is it like on stage server on production while development or all the time all the time yeah so we we run it on deploys we run it on commits we're added on push requests on merge request okay all the time and you received all all the time you receive the same result and it should be the same result right like on stage server it's yeah unless it's broken okay thanks yeah how long do your tests run how long yes if you put it sequentially I think it's around 10 hours if you run it in parallel it's 8 minutes yeah yeah we have the problem really with running tests and before the Cyprus implemented this parallel on the dashboard we have to make all sorts of like our parallelization and splitting so build build our like dispatcher so we put set of tests and it's crazy like docker swarm instances and then on each instance you try to run the test and stuff but yeah it's easier with the reason Cyprus dashboard yeah yeah sure yeah it's hard to put everybody on the photo I was right yeah one two three cheers works yeah make three okay no more questions oh yeah because if questions are not in microphone the people who watch in it not here yeah do you have a generator for those tests no no no it's really man written you know okay is it plans or yeah if you want to know what we test and we want to make sure that it's not automatically checking all of the things on the page or something all right no questions yes hello so in your integration test example you showed some promise execution and basically there was some magic happening I think that it waited until the end do you know a bit more about those because in my other frameworks I still have to be very explicitly about when I'm downloading stuff and yeah so in Cyprus it's the code looks like synchronous but it's unsynchronous so they implemented their own version of promises which magic yeah sometimes it's debugging hell magic but in most of the cases like 99% is fine so yeah you don't have a sink away it's inside of your test much okay yeah thanks my question is what kind of application can you test with the Cyprus like just web page application so I know that people test web pages I know that people test Chrome proud plugins extensions that use electron or no web yeah everything that runs inside of the Chrome pretty much those cypress is kind of electron application on itself the the runner is electron application and I forgot to show it good good point so this is a Cyprus runner so it shows you all of the test sets and it shows you execution of the set test so for example I'm gonna run one test now I hope I'm not I'm not on Wi-Fi now I am so here I'm doing the API call to seed up the user and after that if Wi-Fi will be be faster I will show you the test runs but this is just rendering it's just rendering what happens inside of the driver server Cyprus driver server what I like also in this kind of application when you're debugging test on your writing test you can go back in history and you can expect elements and DOM of the past you know so you can see what happens you can see what means you can see some logs during the test run so this kind of nice thing for me can also debug test here so you but basically inside of the test you can put them between like debug and then it will stop there and you will have full access to the the browser state at this moment that's everything is run within the browser everything runs this is in the browser so when you run a test on CI it's what it does it's boot up the docker image well you run inside of the docker image let's say so it's run it then is you it's run the chromium or electron actually yeah so it's kind of heavy in this sense so the Cyprus minimalistic image is around 400 megabytes and if you record video of the test so you have options of just recording the screenshots of the failures or the whole video of test run if it record video of the test it boots off so the window manager XVVM FM impact so it's like really heavy you know so it can be a bit slow so I would not suggest to at least on CI work with videos but if you are making yourself an electron application yeah webkit can you run Cypher's within the same application yeah yeah that's all