 Today, we are going to hear from Ballerina from Albania to talk about how software developers can be part of the test automation. Is it real? Is it possible? So it's a very interesting topic and I'm also looking forward to hear from her. So let's, without any further delay, I will give it to Ballerina. Well, hello everyone. First of all, I'd like to say that I'm very happy to be part of the trial India conference and I'm very enthusiastic to be part for the first time of this amazing community. This presentation aims to give a real life story of how a product team in our bank decided to experiment a new model of doing test automation with the software developers. But before jumping into the dynamics of our work, I would like first to introduce myself since I'm here for the first time. My name is Ballerina Nasta. It's 15 years that I work in IT, specifically in software development area. I am a Web Technology Solution Lead in Rifeisenbank of Benia in Tirana and in the last two years I have taken also another role of IT delivery lead for retail lending products. Basically more than two years ago our bank started this process of transformation from the standard structure to build more agility. And in this transformation, it started to build a new culture and a new mindset of working, specifically using new principles and disciplines like agile and DevOps. So we have changed totally the way how we develop products. So from the traditional way of working with projects into creating dedicated product teams with a shared vision and with the work toward this vision. And in the context of this transformation, yes, I have taken this new role. And being in the role of IT delivery lead has given me the possibility to see things in a more wide range, so not only in the development, but also, for example, I've started to be an advocate of software quality and I have started to promote a lot continuous testing as a very critical part of this overall process of having speed and quality in our deliverables. Everyone who wants to reach me can do it via Twitter, email or LinkedIn. Well, I believe that we can all agree that when we talk about achieving agility in terms of specific product teams, but also in terms of more in a wide range, like organization level, there are no shortcuts to achieve it. So basically you can just cut specific activities of different processes in order to arrive at the final destination earlier. So it just doesn't work like that. And mostly in the agile way of working, the necessity to have all the competencies inside the team in order to do everything internally, it's really critical. So it's very necessary to do everything inside and to don't rely on any other piece of work that is done outside of the team or that is done in silo. This is a relevant and so for the overall life cycle of the software developments and specifically for activities that aims to embed quality in these deliverables. Today, there are a lot of researches around when that are done for the so-called high performing teams and besides many findings, one very important one that I'd like to to mention is that in high performing teams is seen a very strong relationship between the business success and the technical practices that these teams use in order to achieve it. When we talk about the two main pillars of development and testing in these teams is more and more is seen this blurring line between these two activities. What does this mean? This means that more and more is seen that, for example, testing is a responsibility of everyone in the team and more and more is seen that software developers also are taking a lot of responsibility when we talk about building quality into our products and they are doing this not only through developer activities like building unit tests or integration tests but also more and more they are being involved in performing end-to-end tests or automated regression tests which we must say are kind of tests that boosts even more the level of the confidence that the specific product team has at the end about the quality of its features. So this kind of let's say blurring or the necessity to have everyone involved in the activities of building quality of testing was seen even in our team. But first I would like to give you some more context to this specific product team in order to understand better later the dynamics and our problem. Our team was created at the end of 2019 as a cross-functional team with a product owner, a scrum master, and a development team. And the vision of the team was to work in order to position Rifeis and Bank Albania as the best consumer landing in one touch point to improve the customer experience on consumer landing and to work in order to achieve two tangible let's say KPIs like the time for the prover of loan for 30 minutes and also decreasing the work rate by the bank to 20%. In the moment that the team was created in this agile setup we immediately understood that in order to succeed and to accomplish our vision we needed to leave the full benefits of the job. So we needed to be an autonomous and full cycle team. We needed to own the product vision end to end. We needed to produce faster to have better quality into our product and to increase our operational effectiveness. So it was obvious for us that in our path of working we would need to really build and have this necessary competencies in order to accomplish everything without depending on others not part of the team. I will give you some more context to the system that was in the center of the developments that the team needed to do in order to achieve the vision because the overall lending process of the bank is optimized and embedded in one web application that is built with Angular as a front end with back end web API in .NET with an SQL database and it runs on Chrome. It is not a customer facing application it is an internal application of the bank but it is very critical because basically it embeds the overall process of lending of the bank for our retail customers from the moment that the customer applies for a specific lending product to the different verifications that the system automizes about the personal data, credit history, calculation of indicators in order to prove the eligibility of the customer for a specific credit loan to the approval, the decision taken automatically for the system, the preparation of the contracts and the disbursement of the loan in the core banking of our bank. Also the system embeds a lot of other after disbursement processes which are very important because they aims to work and to do automatically the maintenance of this credit product that our customer has. We must say that the lending process is one of the main sources of the revenues for the banks in general. So doing the right developments, adding the right functionalities and embedding the right quality in the developments that we needed to do in order to achieve our vision was very important. Well I remembered that when we started to work in this agile setup in the first sprint we were working in the same way that we did before that we were in this setup. So a user story was taken from the backlog, the developments were done, after the developments were finished it was released in the UAT environment. Our business colleagues, other stakeholders do acceptance tests and if everything was okay with the functionality as per acceptance criteria of the user story it was released in production. But if in the traditional way, so not agile, this kind of work at some point made us to succeed because the phases of the software development life cycle have been divided and you basically had quite some time to do even development and even testing after development. Now working with the agile setup in this short iterations of two weeks was basically evident that we were not able anymore to accomplish this kind of activities. So we were not able to finish our tests inside the sprint and sometimes our features would last two sprints or even more to be ready in order to be released in production. So if I could describe the situation in a more detailed way, basically every time that the story was completed all the previously one needed to be re-verified because we needed to be sure that new developments have not regressed the existing functionalities of our landing application. But from sprint to sprint the number of user stories that needed to be re-verified was increased from sprint to sprint and the total number of the user stories that needed to be regressed was quite big. So the team productivity was really suffering and it was taking a huge portion of our capacity. I have done an analysis of the first 10 sprints and what we saw with other team members was basically that 48% of the team capacity was being spent basically on regression tests, manual regression tests. And it was obvious that it was a huge capacity spent on manual work and it was an effort that could be spent better in other activities, more productive activities. To mention is that during this phase software developers were involved in doing developer tests like unit tests and some integration tests, mainly API tests. And even though they were taking care that in every code commit our tests were passed, the quality that was built in an isolated way was not guaranteed to be the same when all these pieces of work tested separately were integrated to form later a full functionality of the system. So it was obvious that it was a really need to have cover even this part of our work. Generally when you evidence a problem in a team and you find or identify a specific need and in our case the need was to reduce the manual work and a way to reduce the manual work of our regression testing was basically to automate them. The most legitimate scenario is to okay we have a need why not hire people that specific specialized profiles that can help us to do this kind of activities. And this is totally normal but in our case it's very difficult to find this kind of profile so test automation engineers. The only specialized profiles in the market are the ones that are built by their organizations for their specific needs. So it's very difficult basically to take them from their organization and to hire in your organization. So yes we have quite a great restriction but we were determined to don't let this restriction holding us back toward improving our work. So it was obvious that in order to do to tackle this problem we needed to do on our own. So we needed to do in our team and it was obvious that we needed to upskill our software developers with the necessary skills in order to do also activities that were related with the automation of our regression tests. I must say that when we have this idea there were quite some some reasons that let's say we're supporting you know our idea and maybe the success of our idea. First of all developers are awesome at programming so why not involve them to develop so end-to-end tests. Developers are already professional when we talk about the functionalities of the system because they are building it and they really can help to provide real tips in risk analysis. Developers it's true that they are missing you know the part of the tester mindset but why not help them to learn why not coach them in order to succeed even in that part. And talking about the Gile will of working talking about the Gile team setup it was obvious that it was a need that the test automation should be needed to be an integral part of our Gile software development so inside the team and not let's say a separated process done outside or or in silo. Well let's say that in in a general analysis that we have done we had quite a criteria let's say that we're supporting us but the most important thing was that we needed to talk with software developers because yes we needed to have them on board they were the main actors basically of this our idea and without them we couldn't succeed so I talked with them in the beginning there was this concern of how this kind of new skills would change their profile would they still be software developers or would they change into into test engineers for example we have explained this we have clarified this idea was not to change their profile unless someone at the end would like so much this kind of activities that would like itself to be a test engineer but the idea was basically to to start and to think that we can all contribute when we talk about building and embedding quality in our product the idea was to start and to think in building a more t-shaping profiles for for our developers in order to don't have only one specialized expertise but also to have other skills like for example in in testing in test automation or in yeah other like for example DevOps skills which are also very important for for the development when we talk about the process of yeah committing building etc. I must admit that we have been really lucky because our software developers have been very open-minded they accepted to be part of this process of upskilling and yeah trying experimenting this model to see what we will get at the end because this was something new but they had one request and their request was was totally legitimate they said okay let's try to do this but please let's try to create a good developer experience in test automation let's try to break a little bit this perception of activities like frustrating or boring and let's try to to have fun in in test automation and in fact their request we will see later that was taken in consideration in the moment that we have defined the how of our test strategy so choosing the tools that basically would support us to to to write the tests and to automate them so having identified the problem having discussed and agreed the solution we were now in a situation that we needed to execute it we needed to do something about it so we have in front of the real challenge and the real challenge that we had was basically these two perspectives from one side the developer perspective and from the other side the tester perspective regarding the developer perspective we were let's say quite we were feeling good because our developers you know with the skills that they have as a software developers we're able to think over the architecture of the system and to also the the necessary mechanism for launching them like continuous integration pipelines they were quite good in writing code because they had some years experience as software developers and it's important to to mention that at the end test automation is software developers so the idea is that the auto tests that we are going to develop are the same development product as the product that they are designed to so this was let's say quite a good advantage in our in our perspective from the other side the tester perspective was something new for us and here we had to work a lot because we needed to to learn as software developers at the end what is software testing what is a testing methodology we needed to learn how to know and identify you know weaknesses of the product we needed to know how to read a test case or how even to reanalyze a test case in order to automate it in the right way so this kind of tester mindset was something new that we needed to learn a lot for this reason we have started and we still have ongoing this process of upskilling in tester perspective for our software developers this is a process that is part of our employee development program also in association with let's say a much wider program that aims to t-shape our developers with the necessary skills in order to to be able to accomplish as much as possible inside the the product teams and also I think that this is a process that will be for sure and will stay for long because it helps even the organization to keep track you know of the evolving skills demand that the organization has from from year to year currently we are working with two ways or two methods of learning one we call it the self-open learning where our developers use this online skills development sites but also the say approach is like pairing with an expert or software developers that is more expert in testing and test automation and the other approach is the one which we call curated learning because they are dedicated academies that are built created from Rifeisenbank International in Vienna because Rifeisenbank Albania is part of the group of Rifeisenbank International and our developers participate in this intensive three months programs where they can learn about specific let's say areas of test automation like for web application or API or mobile so depending on the specific needs that the product teams that the product team may have well so we had a problem we identified the solution we have started a process of let's say upskilling because in the moment that we were doing some activities from profiles that were not let's say dedicated or knowledgeable about this a learning process was was necessary and it's still necessary to to to continue but from the other side was very important for us to have a test strategy we were very determinated to don't do automation just for the sake of you know having coverage or for the sake of covering every corner of the application without automation our scope was to basically solve our problem and our problem was a high percentage of effort spent on on manual work so low productivity so we we wanted to document our approach of how we are we are going to do this this kind of test automation was very important for us to to document it and to have it communicated to our stakeholders in order to have a shared understanding about it and of course in order to have support in the implementation of it from from our stakeholders from our product owner from our from our board of management but yes we were software developers none of us was let's say an expert when it comes to building you know templates of test strategy so we thought okay let's try to build a test strategy I called it in a lighter approach in the form of the checklist that was built based on a method that is called the Kipling method the Kipling method is a well-known method that uses six questions very short questions that aims to trigger ideas in order to solve specific problems it is not a method related with a specific kind of problems you can use it for everything in this in our situation we have used it for for our problem of yes defining test strategy so our test strategy is today is the result of you know answering to these five questions why should we do test automation what are we going to use automation for where is our application that we need to do test automation when are we going to write and run test automation who is going to write the automation and how are we going to implement at the end these tests the first thing that we decided to to do and to add in our test automation strategy was to define the why and the why is is very important because here the way how you respond to this question is basically or defines basically you know the level of support that you will get you know from the stakeholders and from the product owner in order to to implement it here is very important to define the why in terms of what business value what business outcomes will give us doing test automation because at the end we need to to underline that the final scope is not the automation the final scope is to do automation because at the end we will get some values that in our case was related with decreasing the percentage of effort that we we were spending on manual work and in our case this would have meant that we would be able to reduce a lot our cycle time for changes and we would be able to increase our deployment frequency so having defined the why of our test strategies in terms of business value it was important to define what are we going to automate and in our case as have described till now our pain point was the regression part that that was was taking you know a lot of efforts in the team so it was obvious that we are going to do to automate regression tests but how much of them because yeah the scope was not to automate everything the scope for us was to be able to identify which are the test cases that are most valuable that will bring us you know an immediate decrease you know of the effort spent from from from our stakeholders for this reason we have used the gram model which is a model created from Dorothy Graham a well-known and a pioneering testing this is a model to define what to automate and it uses five criteria which are the gut feel the customer risk value of test cost of efficiency and history for each test case it gave a score from one to five and this is an activity that is done for from all the team from all the team members at the end you get a final score for each test case based on this criteria and based on the three bands only those with a score upper than 65 are basically identified the most valuable the most valuable test cases that you need to focus on so with this model we were able to identify exactly what we needed to automate in order to get to get the greater value so from 65 total test cases identified from the team we have found that 35 of them was the most valuable so our focus in the test strategy where this 35 test cases another part of our test strategy was also the answer of the question where is our is our application that we need to automate here is important that we have a description of our system under test of its architecture of its main characteristics regarding hardware software and different integrating components and it's also important to have a description of two main important attributes that are related with testability with testability of the application and the automatability of the application so it's very important to have a description of how easy is for the application for example to create an initial state how observable is the application behavior how easy is to get the current state of the application how easy is to diagnose something in the application when something is different or not accepted and so it's very important to have a description of how let's say the user the user interface element are developed how they can be identified in a unique way and it's important also to have a description even of the integration components because they are very important when we talk about automating the business logic from the other side the next part of our test strategy was to define when to do our our automation I want to mention here that we were software developers our efforts or allocation that we were able to spend on test automation was a percentage of the total efforts because we have worked let's say with two scenarios one was working with a 30 percent allocation on test automation and the other part the other percentage on developing features and another scenario that we have used is to work with rotation so in one sprint for example one developer will work only as a test automation engineer and in the other sprint he would be a software developer and the other will be a test automation engineer these were both scenario that let's say we have seen to be to be good the second one is better because it gives you more possibility to have real deliverables in the sprint but was very important for us to have a constant progress when we talk about automation it was very important to have in every sprint a real work done about the automation for this reason we have put a test automation as part of the definition of done of our user stories because only doing this we would be able to you know to have discipline in this in these activities that were new and to have that that constant and incremental let's say output but from the other side we need to consider that doing automation in short iterations like two weeks it's difficult so it's difficult to do good automation in such a short time for this reason in order from the other side to have real outcomes because this is even the essence of working with sprints in agile we have worked let's say I call it we automated strategically what does this mean this means that we have done a reanalyze of the test cases that our business stakeholders has provided to us and in many situations we have done a split of these test cases into smaller parts in order to be able to to accomplish it inside the sprint and many times with these test cases we have decided and taken decisions like for example to move specific parts of the test case from the UI testing level into lower levels of the test pyramid like for example if we had a specific business logic that was tested from the end user in the application when we automated the test case we decided to move the test of the business logic in the API testing level and leave in the UI test level only you know the verification of for example the final results with the necessary assertions I believe from our experience till now this is quite a good approach because when you have a test case you have it from the perspective of a human doing the tests but then when you automate it it will be done from the computer so many parts that are done from the human can be basically distributed in other levels of the test pyramid like the integration part or even in the unit test level well after defining the first questions of our test strategy so the why the what the when the where the answer of the question how how are we going to do automation comes comes naturally in our case considering that software developers were involved in these activities was very important to for us to find a tool that was appealing for them and we wanted basically to find a tool that would make them feel the same experience when writing tests just like when developing software and regarding this our needs as software developers with the tool were were some so we wanted a tool that was you know easy to install with minimum configurations we wanted a tool completely modular with everything bundled in it because we don't want it to spend time in installing dependencies for example we wanted to have a tool that had testing capabilities in the UI but also headless mode because we wanted to run our tests with the continuous integration pipelines of course it needs to be to have quite a good documentation because we were learning and doing and the documentation is really necessary in the situations being software developers and debugging quite a lot it would have been nice to have also a tool with debugging capabilities and of course we were looking for for an open source tool and we were not looking for any any commercial one so having this criteria let's say for the how of our test strategy we have done some researches and also worked with some proof of concepts with these tools in order to see their real capacities and at the end we decided to go with Cypress Cypress was our tool that we choose to do basically an automation it fulfilled let's say our requirements and we saw that also it has quite a good non-functional requirements fulfilled like for example it had quite a rapid growth in popularity and it is quite a sustainable open source with a dedicated team that works every day for its improvement and for further development of the roadmap and new capabilities of the tool and also we saw that it was really very good rated even the ThoughtWorks technology rather has rated Cypress as the top tool to be adopted compared with others like for example Papatier or TestCafe and of course for us as web developers which have you know in the backbone for example JavaScript was quite easy to to adopt with this tool to learn it and to master it quite good in the technical perspective well I will have a quick time check you have five more minutes okay so I will not stop to to to tell anything about Cypress because it's not the presentation doesn't aim to do this but from the perspective of software developer I would like to say that this tool has quite a lot of features that developers loves like for example it is it is modular tool basically it has everything bundled and you don't have to think of any dependencies because it has everything bundled it in it is it gives you a very good experience as a developer in testing because it has a very nice and practical user interface because maybe based on its architecture it gives you the possibility to have in the same run loop in the browser the application that you are testing and from the other side in another frame the specifics on the tests with the with the details of the assertion and everything that is going under the hood let's say not related with the specific assertions that you are doing for the application it has this great capability you know to debug you can debug in it and to be honest not every tool gives you the possibility to debug your code in testing it has this time travel in order to see what is going on with the previous test run or those that are that will be run after and you have this capability also to have videos for your test run which we are now using as part of our test summary report that we are sending to our stakeholders where they can really see how the test is running and you have also screenshots about the errors that are happening in your tests a very important part of our test strategy was also to define where when are we going to execute our tests because it's not worth to have a test automated and to don't run it at the end the primary scope of automation is to basically have the possibility to run them in every part of your software development life cycle in order to get quick and fast feedback about the quality and to take immediately and fast the necessary measurements in order to fix a specific problem for this reason we have built the necessary pipelines since we are Microsoft stack we are working with Azure DevOps but it can be in any in any other technology so we have one pipeline where we are running our smoke tests and another one dedicated to run the whole suite which is let's say a nightly build for our test suite I can say that with the help of our software developers we are now you know in this mode of continuous testing so we have created this capability of continuous testing because we are now able to to to run our tests automatically for every environment and every phase of our software developments and have yes a clear view of what is going on with the quality of our application part of the process is for sure reporting and monitoring because our stakeholders need to have a transparent overview of what is going on with the quality of the application and for this reason the necessary reportings with the necessary videos of the tests run for them is sent every day and I must say that these are numbers that are very good you know evaluated from them and they really see in every day basis what is going on with the quality of the application at the end I can say that we have we have 10 months now that are working in this model and of course challenges are a lot but I think that challenges are part of our work even if we are not going to do this this specific thing of but we are going to do other activities but important to measure is that we have we have had very good results and currently we have seen very good improvement in our in our agile engineering kpis which has result has resulted in very good you know increase of our business agility for the team concurrently we have had a very good improvement of our deployment frequency from twice per month to once a week our cycle time for change has decreased from 20 days to seven days now we have a set of regression tests automated because before everything was done manually and the efforts spent of manual work in the beginning if you remember were 48 percent well now are decreased in 15 percent we aimed to decrease it yes even more but the final thoughts our experience has shown us that software developers are very good to do test automation but we have to support them to learn if you support them to learn they can really be quality advocates and you know having software developers that are quality advocates is a huge benefit for the organization in terms of speed and yeah high levels of quality for our deliverables thank you very much i hope that the presentation have served someone for maybe the same situation or or not thank you well so now since the time is out we may not be able to take questions i would say yeah yeah thank you the session was very informative and we would go and explore some of those techniques that you mentioned