 on to them and you can get prizes for asking questions and with that I would like to welcome Olga Maciashek-Sherma from Codarte talking about Accurist. Welcome. Hello. I will be talking today about a tool for defining and testing REST contracts called Accurist. So first I will make a short introduction about the tool. Then I will talk a bit about the background, why we made it, what was the situation, what made us think of such tool. And then I will talk about the solution. Okay. So Accurist is a consumer-driven contract verifier. It will take as an input very easy to write groovy DSL scripts and it will work automatically in Gradle to produce both wire mock stops and to produce tests. For now we have spoke tests and very shortly we will also have JUnit tests. So who could benefit from this talk? Definitely any developers working with REST, QA engineers working with REST, people interested in the micro-service infrastructure and the challenges it can cause. When it comes to the tech stock, it's a Gradle plug-in but some of our contributors would like to make it available for Maven. So we hope to have it available for Maven as well. Then it's written in groovy and it supports groovy fully. It definitely supports Java. We have never tried using it with C, C++ or objective C but as Gradle supports them it may be possible for you to tweak it a bit and make it work for you. And of course it's open source so you can just fork it, you can tweak it or you can just get inspired and make a similar solution for your own tech stock. So what happened was that the company that we've been making a project in was having a monolith system. It was a legacy system, it had considerable technical depth and as any monolith everything was contained in a single big application, a single logical executable. And what happened in that particular system as it was quite old, the business process code in that system was already very entangled and also the modules in that code were very entangled. And when you were fixing something you wanted to work in one module, you could easily introduce a bug in any other module. So the development was resource consuming, time consuming, then the testing of this system was also very long and the deployment process took a long time. It was not that bad really because it had a lot of tests so it felt quite secure but we thought the architects thought and they communicated to us that there would be a better solution to split it up so we could work in a more agile way. So we switched to microservices. So before maybe we had client identification, client registration, different financial operations in the same one big app. Now we would have a different app for client identification, different one for client registration, we would have them all set up together and communicating with each other using REST. It would also influence the structure the way we worked because before any team would work on that big monolith system and then after the change what would happen is that any team would just work on one or more microservices. So they would be fully responsible for this application to work. Then they would probably not communicate that much and not know that much about the exact inside ways of working of other microservices. So the new issues and challenges that appeared there, there were many of them and people working there developed many tools to make it work smoothly but now I would like to only talk about those related to creating and modifying REST contracts and also testing services that communicated using REST because those issues cost the creation of REST. So when it comes to creating and modifying REST contracts before there was one server it had limited clients and this approach was going to change. Now we would have many small applications and any given application could be a server for other applications while being a client of any other service. So we would have much more REST interactions. And we would have to define and modify the REST contracts much more often. Also we would like all the different teams working on the services to know that anything has been modified in a contract that we're using. So probably a service A was using a REST API of service B but service CD and E were using that as well so service E requested a change. And now service A has to know that this change has taken place as soon as possible. So these changes could also be simultaneous and maybe sometimes clash we would like to work with this efficiently as well. And this has brought us to consumer driven contracts because if the client is the part that defines how the REST API should look it is generally more efficient it is better designed. Because with the big server approach usually it is the server team that is taking decisions when it comes to the API because this is the core component so they are behind the main logic and everything. But with this approach we want our clients to define what the contract should look like because they know what will be the best approach for them to use this data. And then the second thing we wanted all the consumers to be notified as quickly as possible about any change that has taken place. And when it comes to testing this service before with the monolith end to end test what happened was that there were many of them they were quite comprehensive people especially Q18 felt that the borders were well verified because there was a one big test environment and stage environment and on that environment the main server deployed and any limited number of collaborators and the end to end test would fully use the communication between those applications. And they felt that it considerably improved safety. Probably they were too many of them probably the execution was too slow some code would get called again and again during those tests but all together it felt quite safe. So what could we do now as we were transferring to microservices? First idea was just to maintain it. Let's have it let's just deploy all the services at the same time. You could do that well in theory you could do that but that would really go against the whole idea because you want the services to be independent. So the development is quick the deployment is quick the testing is efficient and if you had them all deployed together only the development would improve. This wouldn't really make sense. So then we could make many many many such pipelines with all the collaborators there. One pipeline only used to deploy one service but we would test them as we used to with all the communications. But we wanted to have many microservices like 50 20 later who knows how many. So it would be extremely resource consuming time consuming and not worth it. So the idea of what the architects came with was to stop collaborators. This is a known approach used very often however what happened is that stops do get corroded. So you might have written a stop one day and it worked same as your server worked but then somebody has updated the API on the server deployed the new version you have tested your own service against your stop you don't even know that the server has changed maybe they've told you forgot to change this stub and then what can happen is that your production when you deploy to production the communication can fail even though the test passed. So there was too much human element here and the better approach seem to automate this. So the requirements when it comes to testing were that we would use stops but the stops should be validated against the real responses of the server. So while showing you accuracy I will talk a lot about client and server and of course as I've told you we can have many clients and server service in this infrastructure but now at any given time I will just refer to a client as the part that is using the rest API and server as the part that has the rest API. So how do we work with accuracy first the consumer team the team working on the client service submits a groovy the cell scripts that we will shortly see as a pull request to the team that is working on the server. So scripts are used to automatically generate wire mocks stops but they also generate server tests and these tests fail if they do not have the same implementation. So new server version cannot be deployed until the stops and the implementation of the rest API are same. So they cannot merge this code and they cannot deploy until they have implemented the change or they can just reject the pull request if they do not agree with this change or if another team has made a similar pull request on the same endpoint they want to introduce another field in the body they can merge it but they have to make it consistent. And now we'll go through this process. So first the client submits the script and this is what an accurate script looks like as you can see is quite similar to wire mocks script or any JSON but it's simpler in the syntax at least it seems so to us and it's written in groovy but as you see you don't need to know any groovy to be able to work comfortably with those scripts. So you have the request part and the response part and the request part you have of course the method, the URL, the body of course you do not need to have a body in the request it's optional you can put as many headers as you need and then in the response part you have status and also body and headers. Now what happens? The server team has this pull request so they can run clean build with Gradle and this stops will be generated from those scripts. So this is a simple wire mock stop and it exactly matches the structure of our script. So we have the request and in the request of course we have URL, method and then the body part is changed to body patterns and we are using JSON paths to check if it's the request that we want to send the response for and of course we check all of the headers and then we just passed the response. I don't know how many of you have ever worked with wire mock. Okay so wire mock allows you to run stops during your test and you just give JSONs and it will set up a kind of test server where it will put these stops and if you send the request that will match the request in this pattern here the response will be submitted. So just you don't have to have set up two token artifacts and you substitute the rest API of one of them with this and I also recommend using wire mock for any different purpose as well not necessarily with accuracy it's a very useful tool. Okay so then you have the body and the headers. Okay so the server team has this but as I've told they can't do anything really they can't deploy this thing they can't merge this branch because they also have the test and the test shall fail. So this is what the test looks like the nice thing is that it is automatically generated and it will be always in keeping with the script so you have one input and two outputs that have to be in keeping. In this test we use rest assured library it is a nice library for testing any rest interactions and all these methods like call or creating the request will be done using rest assured so it's from a different library. So we have the script and we use the body and the header parts of the script to create the request and then we use the information and method and URL to really make the call and then what we have to do we have to verify if the server is giving us the same response that we have put in our stop. So we verify all the parts we verify the status code all of the headers then accuracy uses Jason path to verify if all the body parts that were specified were available. So the only thing that you really have to do is set up a base taste class and tell accuracy this is my base test class and all the accuracy generated tests will extend this base test class and to set it up if you are we work out with spring so it supports rest assured mock MVC if you have if you're using spring you can just do something like this so you put standalone setup and you give any amount of controllers that you need to launch you launch them there or you can just have a fully running application you need to run it here we're running it using web integration test annotation from spring we also support juxta rest so that's what you have to do you have to set it up sometimes you have to add some mocks if you want to mock up the behavior if you don't want to launch the entire application or just use rest assured so now we have and now the server team has those tests they're not passing so they will implement the change in their rest API and in their functions to make them pass once that's done the clean build will end up in success and they can deploy and it's very important while using accuracy to connect the deployment of the new version of the server with the deployment of the steps you can deploy the steps to any artifact repository it's up to you where and you can use any tool that you use usually you would use for deploying things so in gradle you have for example Maven published plug-in and it's very important to do this together so now any client any other microservice can use those steps and they can be reasonably sure that if they are testing against those steps the deployed version of the other service of the server would behave in the same way as those steps so I also like to talk to you about regular expression support we have more features but this is the feature that I find extremely important when accurate source created the why was one of the first users and the first request we had okay you need to implement regular expression support without this we cannot use it because when you was when you are sending when you're making a stop a wire mock stop what you expect to have or you are just working with stops and what you want to have is that for example you will send in your request a client ID and some other value like here loan amount that you want to take and you expect a certain response but it does not really matter if your ID is one two three four or five six seven eight or whatever other ID it just has to match a certain pattern same if you're registering a client you give request with client data and you expect a response from the server and maybe in your response you expect a client ID a UID whatever and it's not really important I should always be the same it will never be the same normally but it's very important if it's supposed to be UID it should match the UID pattern so then you want to change the check or verify or send request using a pattern and not a value but as you know accuracy also uses this data to create tests so in a test you cannot have a pattern if you say create a body with this field and I would check that the body has this field and then you want to make a call to the server and passing or let's say my client is and you can I just say I am passing to you a client and you know this client matches this regular expression or you have to say in the request I'm making a call and in the body the client has a real ID so that's why it was very important to introduce a different handling for the steps and the tests so the stop will just know that you can be sending anything which matches a pattern but in the test you will submit a value so this is how we do this with accuracy we use a value method and we can give a different definition on the stop part and on the test part and then you will usually put a regex in the stop part and in the test part just a value and when we're talking about response it's generally opposite so you are expecting a response with a field of a certain type of a certain pattern not necessarily some defined value sometimes yes most of the time no so this this script that I've shown you will translate to this stop where if you see instead of the two equal signs that we're checking for equal we have now different sign that is checking for matching pattern and in the test we'll just have the value that we specified so we can make a call using that value there are some other interesting functionalities that are supported so we do have query parameters that can be useful for some of you as optional fields are supported that's maybe not that often used but some people wanted to have fields that they may or may not have in the body in the response then you can execute custom methods on the server so say in that previous example instead of just giving the value one two three four for the test for your client ID you could just write execute generate client ID and in your base test class you implemented method execute or generate client ID and also accuracy deals particularly well with nested structures so when you know that you will have bodies that have many nested elements which is very often true accuracy will work out of the box with that you don't need to worry it will just resolve its maps and it will make it work so this is very useful now I would like to tell you something about the possible issues so we have found it to work very easy easily and out of the box and with minimal setup in the real microservices that were very small applications but we were then quite enthusiastic and we tried using it in this complex legacy system as well until it has been decoupled into microservices and this has proved to be complex because and there was a lot of things we didn't want to launch this application each time because it was too heavy so we had to mock it there were many mocks and the problem was that currently accuracy only supports one base test class so you have to have all those mocks in one class or in some class that the other class was extending it had to be in the same hierarchy and we had many modules that was quite problematic this is probably a room for improvement for changing it yeah and then the greatest setup if you had multi module gradle project the setup can sometimes get very complex especially if you have a lot of things happening in the gradle itself as gradle allows big liberty in writing it and changing the ways that the builds are done and using dynamic functions so in that project that was already so complex that setting it up with accuracy was a hassle and some other possible issues it's a young solution I think it has been there for a year or so so there can still be bugs the good thing is that the accuracy team is working very much on the box so the support is constant and we try to fix any bugs that there may be and we try to introduce new functionalities that people need so that's a part bad thing part good thing yeah and I would like to show you a demo okay so here we have a very simple demo project it has only been written for the demo so please don't put that much attention to the functionality itself and it has two smaller gradle projects so these two smaller gradle project we can imagine they're just two different jars deployed somewhere in the network infrastructure so the first project is a loan application service it's very simple and it will just use make a request to a fraud detection service that we will see shortly and interpret the answer and it's very simple either the answer can be okay or the answer can be fraud so if it's okay the loans is processed and if it's fraud the loans is rejected and then you have the another service called fraud detection service this is what we are making the course it has extremely small API has one controller and currently in only only returns okay and the functionality that the client is expecting is the commented out that it should also reply that amount is too high if it's higher than five thousand and then it should reply that this client is a fraud so this is not implemented so what happens now and also there are no scripts yet and no stops and basically nothing and the loan application service has a test these are some spoke tests so basically it expects either a server to be running the fraud detection service or using the wire mock rule having stops loaded to the wire mock and also replying same as the server would reply so now what will happen if we just run clean build our loan application service test should fail because they are trying to communicate with the other service which is not there okay it failed loan application service tests have failed and if we see why they fail it is because once again because as we expected there was no response from the server so what we will do we will now add some scripts and the scripts have been already previously added so we just need to reset gate and we can see the script here so if we go to mappings we have the nearly the same scripts that I've been showing so we have the scripts the stops will get generated the test should get generated and one of those tests should fail because the implementation is commented out so if we do run it now effectively one of the test has failed and we can see the report here yeah so it's expecting to get fraud it's getting okay and let me show you the that what really was generated so in the build if we see generated sources here are tests and the first one is obviously the one that fails it expects fraud in the body there's no fraud and then here we will have the script so this is the way that this solution works and I would also like to suggest you to if you got entrusted if you think you might use it in your project to check out first of all accuracy code and see if it's useful for you if you need for some different tech stock maybe check if you can do something similar if you want to use it by you see some problems or there's something lacking make a pull request or just make a bug or new issue and then you can check out the project with the examples that I've been showing it has many branches with different stages of the setup in every branch so if you like to check it out you can play with it and if you have any issues you can approach me I will be in the conference and I will be ready to help you with that and yeah you can contact me about this and now if you have any questions that's the good time to ask them I have a question about who use that software like Accurast do use Accurast yes we have we have definitely especially we have one big users from the financial sector and they use Accurast a lot and mainly to their requests and their bugs that they they made we have improved it but now we are also getting more and more users from different companies because there's a whole movement of switching from monolith to microservices many companies going to that approach and Accurast really has little alternatives there are some alternative frameworks but they're more difficult to use normally they're more complicated so we have we are getting more enterprise users okay one one question rest is not only about HTTP protocol do you plan to support also different transport protocols other than HTTP for now we have been concentrating on HTTP but as I've told this is being developed very intensely and we are happy to make things that will users make more comfortable so if anybody you need something you can just make an issue and will be definitely considered and if there are no major reasons not to do that that will probably be implemented also if you want to implement it everybody is invited to participate it's open source okay and do you have any roadmap for for the project what what do you plan to implement in future do you have any roadmap for the project any plan what will be implemented in near and far future yeah for now the main functionality is that we need it and we are using in our work we're implemented but so for now what we want to do is JUnit we very much want to support JUnit and it's in a branch it should be supported soon and what also we have done was switching to JSON paths because sometime ago we had our own written validation of the JSONs and then we realized that maybe didn't make that much sense if there was something very good in already there so we switched to that and we are also getting many issues from users so we are just fixing it so I think for now it's JUnit when it comes to things that we want to add and also there as I've said some contributors that would like to make it available for Maven so we very much hope for it also come to Maven thanks okay so there are no more questions you can always come and ask me if you have later on questions or you just don't want to ask them now I will be here today and tomorrow all day long and feel free to come feel free to check out our code and comment about this thanks thank you thank yoga I would like to remind all of you to go sign up or at least vote for lightning talks there is a board in the main hall thank you what about this I'm sorry I forgot the big one yeah this one but the other one I do not remember I'm sorry I just forgot about them yeah I think do I have wrong watch because I got five minutes before and I thought we started in 950 and maybe okay no problem yeah for now no do you think it would be valid so hello everybody there was change in schedule next presentation it's gonna be about network manager and