 Welcome everyone to the contract-driven development session by Joyl and Hari. Just a quick introduction from my side. So, Hari Krishnan is the founder and CEO of Polarizer Technologies. He's a polyglot full-stack developer, architecture consultant, XP coach and trainer. And Joyl is an engineer by heart. He's a senior consultant. He loves building things. And apart from that, when he's not coding or not working, he's also a great musician. So, without any further delay, over to you, Joyl. Let's get started. Thanks, Kartik. And welcome to all of you to this talk. Let me ask my colleague Hari to give a quick introduction about himself first. Sure. Thanks, Joyl. So, I'm Hari Krishnan. I'm a consultant and a coach. I help both unicorn startups and large enterprises with their transformation activities. I love contributing to the community. I speak at quite a few of these conferences and I volunteer. Also, my interests include distributed systems and high performance application architecture. So that's quickly about myself. Over to you, Joyl. Thank you, Hari. And I have about 19 years in this industry. At present, I am a consultant and coach and my focus areas tend to be software development, software quality practices and contract testing. Well, let's jump into the session today with the demo. I'm going to show you some code. This is an e-commerce service. I won't go to the code for this too much. Let's quickly take a look at its open API specification. Even if you haven't seen an API specification before, I'm sure you'll all recognize an API when you see it. This is the slash products API, which takes an ID in the path. The ID has been defined as a number, as you can see here. And this essentially means that this API could be slash products, slash 10, slash products, slash 20, so on and so forth. And when you fire a get on this API, it's going to return some product details. And the product details, the exact shape of it is defined in this file. But you can see an example. There's a name, type, this gadget, inventory, ID. So this is how the API should look. Let's try to run some tests on this API now. I'm going to just quickly run these tests. Let's give it a second startup. And here we go. Tests are running. Naturally, we are starting the spring application up here so that we can hit it with the tests. And you have some tests that have run, right, 12 tests. As you can see from this name here, this is a set of contract tests. So let's see what this actually means. There's a request that has gone out, right? I was saying we just started the application up here so that we could hit it. The request has gone out. It's a get request to the application to slash products slash 10. And the application has responded with a 200 OK. And it has a payload here, right? And this payload is familiar. I just showed this to you a few moments ago in the contract. And as you can see, it's a contract valid payload. This is exactly the way the specification defines it to be and this test therefore passes. Which is good. Here we have another request. This goes to post to slash product slash 10. The payload this time is an actual, you know, product content and we get a 200. And, you know, this is about updating product details. You can see the title on the left hand side. So there are 10 more essentially. And these are contract tests. We'll talk more about them. Essentially, what we're trying to say is, is this API specific? Is this API implemented in the correct fashion according to the specification that I showed you in the YAML file earlier? What I'd like to show you next is the code for this, right? Where is the code here that generated these two tests? And the answer is this is it. There's nothing more. This code snippet is all you need. And all that we are really doing here is defining a few details about where the services, what port is it starting up on, which IP is starting up on. And we are using this very cool open source tool called specmatic. And we simply extend from this class and specmatic takes it over and then reads the contract and generates the test for you and you have nothing to do. This is all free. Essentially, specmatic generates a test for you for free. There is no other test code aside from what you see on screen. So that's contract tests. Hari will talk a lot more about this. But before I move on, I have one more thing to show you. I'll just uncomment this there and says your specmatic generative tests. So now we take this to the next level. You simply set the flag to true. Run the tests. And just from that, we have a lot more tests. And in fact, this time using failures too. In fact, if you remember, we had 12 tests passing earlier. Now we have 16. And we have 26 failing tests as well. What happened? What has changed? Generative tests basically what we're saying is not only are we looking at the contract now, we are giving it a much more taller test. We are generating even more tests. There are positive tests and we even generating negative tests. So let's let's take an example of what the negative test looks like. Let's see what, what, why is this test failed? It's negative. You see the word negative over here. Why is it? We're saying key zero is missing in the map, right? No such element. All right. There's an exception. Is that really the core of the problem? So let's scroll down past the exception. Let's scroll to the actual request and see what has happened. We've seen this before, right? In one of the cases earlier, post to slash product slash 10. We've seen this payload as well. We are trying to update a product, details of a product. This is new. So as per the contract, the ID is supposed to be a number. The contract does not allow this to be anything other than a number. It does not allow it to be null, for example. And since the contract makes this explicit, it just knows that there's, there could be a potential problem here. And so it says, let's pass a null and see what happens. And it passes a null. The application obviously tries to read that as a zero. There is no product calls, you know, with the ID zero, throws an exception there, scroll right back down. And the exception is not handled. So we throw a 500 and right there is a problem because this is a poor way to handle, you know, a bad request you should have a 400 series, a 4x series error. In fact, in this case, ideally a 422, which is uncrossable entity in HTTP. There are another 25 tests like this. And if you make all these tests pass, you have basically bulletproofed your application. So that's 26 plus 16, that's around 42 tests generated for you from the contract, completely for free. No code other than what I see on the screen. We'll show you how to use specmatic, you know, to do this and other kinds of things, very interesting things as this talk progresses. But now before we move forward, I think we need to take a step back and set context, understand what the problem is that we are trying to solve. And then we start getting deeper into how we are going to solve it. And for that, I've had it over to you, Hari. All right, so let's quickly get into the context of why this talk and what we're trying to achieve here. Before that, wasn't that a pretty awesome demo by Joel? Practically taking open API specifications and being able to generate tests for free. That's really interesting, given that we had to write no code whatsoever. Why is that all important? And why is this all the more relevant, given the widespread adoption of microservices? And that's why that's what we're going to take a look at now. So to set the context, let's imagine that we're building a mobile application which requests product details from a service and then the service responds with those details and the mobile app displays the same, fairly straightforward. Now, the application requesting the data is the consumer and the service responding with the data, let's call it the provider. With that terminology out of the way, let's think about how we'd go about building this consumer. Let's say I am the mobile application developer and I have a dependency on the actual provider. Now, one way for me to build this application or for the mobile app is to wait for the provider to become available and only after it is available, I'll use it as a reference and build it right. Now, this is the sequential style of development and not really productive in terms of pushing features out. So what is the typical, you know, widely accepted solution to this? I could stand up a mock server to emulate the provider so I can make independent progress on consumer application development, right? This looks good on paper. However, there's a fundamental issue here, right? The mock that I'm hand rolling in order to sort of emulate the provider need not be truly representative of the actual provider. Now, that is a big problem, right? How is that a big problem? Let's take a look. Now, as a consumer application developer, I may be wrongly assuming that I could send a string for the product ID while the actual service is expecting an integer. And likewise, the provider may be returning a name and an SKU while I'm wrongly assuming again that it's going to give me back the name and the price. What does that lead to? When I deploy the consumer application, the mobile app alongside the real provider, integration is going to be broken. And that's bad, right? And what's worse, such issues, we know we cannot find it on our local machine because obviously like what we saw, we are dependent on a hand rolled mock. So we are not really getting any feedback from the real provider. However, this same situation continues even in the continuous integration environment, right? Because there as well, we are using a hand rolled mock possibly, right? And for the provider, it is not a very different story. Provider also does not have an emulation of the consumer. So even the provider application engineers may be developing this application in an isolation, right? So the first instance where you realize such an issue exists, is when you deploy both these components to an environment such as integration testing and then you realize there is a compatibility issue. Now this is a double whammy of an issue, right? Because not only does it compromise your integration testing environment, it also blocks your path to production, which means you have unhappy users. Now this is not a good place to be, right? There's more bad news. So that's the heat map at the bottom kind of represents the cost of fixing issues alongside the timeline and the later in the environment you're finding them. So an issue found very much on the left in your local environment is very quick to fix, right? When you compare it to this compatibility issue which is being found out only in integration or worse in production. That's both, you know, it's gonna cost you in terms of user experience and also it's gonna cost you in terms of resolution time, which is not a good idea. So what would we like to do? Integration testing like we all know is not a panacea to any of this problem statement, right? What we'd like to do is to be able to shift left the identification of such compatibility issues and thereby try and eliminate integration testing altogether. Now, is this even possible to find out compatibility issues without integration testing? Let's find out. I'll hand it over to Joel to show us the same. Go over to you, Joel. All right. Thanks, Hari. Well, as Hari said, we are going to now learn how to shift left to make sure that we can identify these compatibility issues right on our laptops without having to get into integration testing. So stepping back, a lot of the issues that we have seen previously between consumer and provider are perhaps they are caused by communication gap. There might be, it might be because they are documenting the APIs using, you know, something like Word or Excel or even worse, you know, perhaps Word of mouth. So, would it not be much better, much nicer if we could capture every detail of the API, right, including the headers and the requests, the types, the payloads, the keys, the data types, is the key optional, is the key mandatory, is it nullable, is it a string, so on as well. All of that rich detail that we could capture that somehow in an open industry, accepted industry standard, widely understood, specification format. What if we could do that? The good news is these formats exist and for rest, the most widely used one is OpenAPI. And you can put all this down in an OpenAPI specification and if the consumer and provider actually have the specification, then they can get on the same page and about what the form of the API will be. And that should go a long way towards improving their integration and reducing integration issues. Or would it? Because an API specification is good. It's a step up, but it's still a description. It doesn't actually force you to do anything. So it doesn't really force the provider to implement the API as per the specification and it doesn't really force the consumer to have the right expectations about the shape of the API which should be as per the specification. So we need something more than that. We need to go beyond. We need an executable contract. We need a contract to enforce the API specifications. Now let's talk about that first. The good news is API specification is something that we do have. It contains all the details that need to be enforced. Everything is there. What if we can turn the API specifications themselves into executable contracts? Would that not actually solve the problem? And that's exactly what we're going to show you. How do you turn API specifications into executable contracts? Both for the provider and the consumer to ensure that the two of them continue to match the contract. And how does that work? The tool that we demonstrated at the start of the stock as I said is an open source tool called Specmatic. Specmatic will take the open API specification. And for the consumer, it generates a contract as stock or what we also call smart mocks. And using this, it is possible for the consumer to make sure that their expectations about the form of the API matches and is faithful to the contract. And thus the API, the consumer can stay in sync with the contract. Now having done this on the consumer side, we need something for the provider side as well. For that, Specmatic again takes the open API specification, the same open API specification and runs contract as test. With contract as test, it becomes possible for the provider to faithfully simulate the kinds of requests that a consumer would send as per the contract. And then when it returns a response, as I've shown you earlier, the contract test actually checks whether the response would be understood by a consumer. So this is reciprocal. If a consumer stays in sync with the contract, using contract as stock and if the provider stays in sync with the contract using contract as test, now they will integrate when they get into an environment. In fact, not only that, it becomes possible for them to work independently because there is no need now for a consumer to wait for the provider to be ready to start work. The consumer can just use the contract and use the contract as stock and start off. They have something they can invoke. The provider doesn't need for the consumer to be ready to be sure that they're going to integrate. They can just start off with the contract tests and know that that's going to be a faithful simulation of how actual consumers that are contract adhering will work with the provider. So let me actually get now deeper into the consumer side of the story. How does a consumer emulate the provider using smart mocks? And for this piece I am going to run a small exercise and I think this will be helpful because it will get you actually hands on and so you can see for yourself by actually learning by doing. You can hurry. I would also like you to post these details on the chat. You can download the open API specification sample from here. You can import this file into Postman as a collection. This is one of the nice things about having a specification format is that as it's a format, it's machine readable and suddenly it means that your tools, other tools can understand it. Then let's download and set up specmatic from here. Just put the two of them in the same directory. This will help you to make things a little more smooth. Then you CD into the download location and run this command. This is a Java tool. By any chance, if you don't have Java, don't worry about it. I will be doing all of this on screen. If you have Java and would like to follow along you can follow along from here. Okay. I guess, Joel, you can start off. Before we begin the exercise, let's take a quick look at the specification. Now I think after seeing the demo which I did at the beginning, this will be a little familiar. We see the same products API here. We see the get and this time the responses in line. So that means when you fire a get request against slash products, what we are expecting to see is this response in return and the response should contain a JSON object with name and SKU, both of them are strings. So the first thing that we're going to do is start this up as a stub. For that, we're going to say Java minus Ja, specmatic.ja, stub, products API.yaml Let's run this and see what we see. Now my output might look a little different from yours, but the most important thing is that you get this line at the end. Stub server is running on slash slash nip colon 9000. And once you have that and this command is posted as well in the chat, so I think you should be able to run it from there. Once you have this, the next thing you can do is step into postman and you can do the same thing with curl, but I'm going to do the demo using postman because I think most people should have it. We're going to import this contract. Let's start off with that. Let's import the contract, boom, postman understands it, which is awesome import. Now we have a sample products API we have get products and postman is set up. So essentially we have a let's change the space URL, I'm going to change it to keep things simple. So let's import 9000 to slash products colon ID. Column ID here is postman syntax for a variable. The variable is declared below. Path is part of variables and postman is generally something random. Let's put in a value of our own and let's hit send. Let's see what that gives us. Get a response with some random values. What's happening here? So basically this is a response slash one. It knows what that means because this is in the specification. But we have not yet told it what to do with slash product slash one. And so it helpfully goes back to the specification, looks at the format of the response, generates a response and returns it. And this is basically a randomly generated response with the right keys. Just to prove that it's possible I'll tell Specmatic what to do. Send 5 and to turn the batteries with you know. This is a preset value I told Specmatic how to respond to 5. That's something that we will see later. A little bit later. Has everyone followed with me so far? Has everyone got to this point? After this I'm going to take it on to something a little a little further basically. So I'm not going to ask if you want to follow along. But I'd just like to see that you've reached this point and you know how to set up Specmatic and invoke it and pass it a contract. Okay. I guess it's all great. Okay. Nice. Thanks for the feedback. Yeah. Thanks. So I'm going to take it further now. I'm going to do a lot more demos and in the interest of time I'm just going to show you how this works. But I think what you can always do is get your hands on an open API specification and these things are all very easy to try out. We will share with you the documentation for doing this later. But for now I'm going to kill this and I'm going to switch to another window with a slightly more involved contract where we are going to try out a few more things. So you can see this is a very similar contract. In fact this piece is exactly the same and I've added on a small API here. Here in the second API is for adding a product. So slash products you post an object to it and you add a product. So let's see what we can do. I'm going to say Specmatic stub you've seen this command before. This time I'm not saying Java minus Java because I have set up Specmatic as an alias so the command just works and we start the stub off and now we try 5 and you see the randomized response because this time I've started up a new instance of Specmatic and I haven't told Specmatic what to do with the value 5. Well let's do that. Now we'll see how we do this. First I created a directory. Inside the directory I'll create a file inside this file. So Specmatic is reacting and it knows that there's something wrong with the file so it hasn't loaded it. Inside the file I will put this content just kept in a snippet there and I will just change this to cvc123 save it I'll just give this a quick restart there you go. Specmatic has read and accepted this file. Let's fire a request here. Specmatic now returns. Batteries ABC123 alright. So first question is what if I want to stub out multiple things how do I do that? Let's say that it's not only batteries but also torches so we'll call this torches. We'll rename this to batteries in torches I guess we should have a different product ID. Make that 10 torches save it 5 reacts with batteries 10 reacts with torches. Of course we forgot to change the SKU if I send it with 5 it still says ABC123 and I think for the purpose of this test situation we really don't care what the value is so I don't want to have to think about it I'll just say string here I'm just telling Specmatic you return some random value leave me out of it and you figure it out I don't care what you return just get me a string back and let's see now what happens when I make a request to 5 Specmatic return to random value and if I do 10 Specmatic returns another random value and this way I can get a load of my head let's specmatic do the work that I don't want to do and this makes things a little easier maybe instead of torches this time we'll call it notebooks this will be 15 this will be notebooks and voila let's give that a let's see what happens when you do 15 it should just work there you go, notebooks returned as well of course now comes the next interesting point what if we tell Specmatic to stop out the whole point of doing this was we should not be able to stop out something incorrect so let's try to do that see what happens, we know SKU is supposed to be a string not a number so let's try to stop out a number give spec, yeah there you go Specmatic immediately tells you response.pory.sku so these are the breadcrumbs the breadcrumbs basically give you pinpointed feedback about where the problem is and this is exactly what the problem is contract expected string but stub contain 10 which is a number which is fair interestingly we had said this before we made this change now Specmatic no more recognizes 15 because it's rejected it Specmatic will not accept a stub that does not match the contract so enough of the response, let's take this to the request let's say we want to we had an API to create something so let's see how that could work we'll say create towel the create API was a post as I recall so there was a post to slash products the body basically goes in the request this time this time we are saying towel we are saying PQRXYZ and what should the body be here we'll have returns with a 10 or something let's save it let's see what Specmatic thinks this need to give that a restart oh this is interesting looks like we got the body wrong so we are saying response.body.sku contract expected string but stub contain 10 so essentially I stubbed this out incorrectly let me just quickly check the contract the contract basically says we return an ID so obviously that was wrong return an ID let's make that 10 let's hope that we got it right this time Specmatic will take a second restart oh sorry this was supposed to be a string oops oh the problem is somewhere else the problem is somewhere else my bad I think this is one of the other expectations which we had set up incorrectly yeah we didn't revert that and so now we have this problem here I think we should be all good now but this is interesting this is exactly the kind of problem that Specmatic is designed to solve everything is loaded now successfully I was not allowed nothing was allowed to fall nothing was allowed to fall through the cracks I missed one thing here I got it there Specmatic got this one I fixed that I missed this Specmatic got that so now we post products let's just take this body and post it I am going to create a new request here let's post it to HTTP localhost 9000 products we post the body we say raw we say JSON we post this and we send Specmatic accept it of course down to the same question we might not really care about string we send that that is accepted as well and once again let's try to answer the obvious what happens if we stop something out here we know what error we get right what happens if we try to do that here as well we accept it Specmatic won't even accept the request now this time the error is not coming from the stub this time the error is coming from the request the SKU was incorrect the SKU was the wrong type this doesn't even reach the stub Specmatic just tells us that the request was wrong okay now we've spent a good amount of time looking at Specmatic and even seeing literally on the fly how mind mistakes during the demo were caught by Specmatic which is I think pretty cool but we should now talk about workflow tests in a sort of in a real more of a real scenario what if we were to post to one API some data get back some ID and then we have to pass that to the stub right the thing is there might be you might have multiple applications you might have an application which are testing whose ID you get back and you need to pass that to another ID to another API which you need to stub out and that basically means you're getting the ID from some application on the fly during the test and if you're doing that what are you going to put in the file everything that goes into a stub file has to be known ahead of time but if I'm going to get an ID in the test from some application and the ID is coming live I can't predict what the ID is going to be but maybe I want to validate that ID maybe I want to validate that the request that goes out to the stub has the right ID which is just called generated what do I put in my file I can't obviously the answer is I can't put anything in the file because I can't predict it ahead of time this is this much is obvious and obviously I've shown you the specmatic restarts but a restart is not a valid way of running a test it's very unpredictable what we really need to do is we need to be able to tell specmatic after it has loaded here we've seen how to tell specmatic before it loads and when we make a change to the file it reloads everything but now we need to tell a running instance of specmatic what to expect and how to respond so let's see how we do that I've shown you you know let's come back to this window I've shown you how to stop this out we know how this looks in a file well it's pretty simple specmatic has an API called set expectations expectations is basically what we tell specmatic to expect and when it gets that request what does it respond with so we are setting expectations with specmatic we are saying when you receive stash products this time to something different let's call this a sponge for whatever reason and we say PQR 1, 2, 3 return a response 20 we send this out to the specmatic and specmatic returns a 200 this is important because right here when specmatic responds you know whether your expectations are correct so to start with let's actually see how well this worked right we say sponge and we say PQR 1, 2, 3 and we get a response back oh sorry I had a typo there we get a response back which is ID 20 which is great we take this take the next step here how exactly would specmatic reactive we try to set something incorrect and SKU ID which is a number you've seen this before you know what to expect but let's go the whole way here you go request.body.sku so basically specmatic never lets it go at every stage be it an expectation be it a request be it a response in the expectation specmatic will flag it and it will not miss it and that means you get the early feedback right on your laptop you don't need to wait to write a line of code before this happens um so this is how you get early feedback this is how you can stop something not dynamically and now we need to actually fit this into a test let's try to take all of this and put it together and say how does this actually work with a test before we go there and show you how that works let's review the anatomy of a component test you typically have a test you will have of course a system on the test then since we are stubbing out the API before integration we have a specmatic stub which has been given a contract every test has typically three phases arrange, act and assert in the arrange phase in this case the test will tell the specmatic what to expect and what to return dynamically in many cases specmatic will validate this with the contract and will return the necessary feedback we'll see how this works later once this is done the test will now act the act section is where the test invokes the consumer this could be a mobile client in this diagram it could be another microservice anything that consumes another API as a consumer the consumer will now hit the API which in this case is specmatic, specmatic will respond the consumer responds to the test the test now runs a set of assertions on the response to check whether everything passed or not and with that as background let's just quickly take a look at how the actual test looks this is how it is this is the arrange section I just described this is the URL you have seen before this is the request format you have seen before again we are stubbing out something like slash products etc this is the response this is the framework called karate the test is written in karate and the key thing is this line on line 36 which I have highlighted is an assertion we are asserting that specmatic returns 200 if specmatic has to reject the API it will return 400 and this assertion will break the test right here you are not even going to waste the time to run the rest of it because you know right here that what the test expects of the downstream API is incorrect but assuming that specmatic does accept this expectation and then moves ahead the test then makes the request to the system under test this API is the system being tested and this is a test that is testing a microservice this microservice basically receives a call it calls specmatic downstream specmatic response, microservice response here and now the test asserts on the response to the microservice right and this is a real world this is how to work in the real world on an actual test framework does not matter specmatic is framework and language agnostic this is karate you could do this with you know anything else selenium or whatever other framework you name right alright this has been a deep dive into how it is possible for the consumer to stay in sync with the contract by validating their expectations about how the API is going to look this is good so far now we need to take a turn to the other side and see how it works for the provider and for that I'll turn it over to Hari all yours sir thanks Joel that's pretty awesome for the deep dive into the consumer side so now let's look at the provider side of the story right so if you recollect the image earlier that Joel was sharing of specmatic trying to keep the equation balanced on both sides so for the consumer you do all the stubbing and the mocking for the provider we need to do something called contract test right and this is the initial teaser that we did with the free test we were generating but now I'd like to do something a little bit more interesting and I'd like to give it a spin in terms of trying to understand what we're trying to do here when you have a specification and you have a system under test we could generate tests and that much you have already seen right we could use specmatic to leverage the specification as tests and verify that the system is indeed adhering to the specification or not now if this is the case there is a hypothetical scenario we can think about what if the provider code does not exist at all and all you have is the specification right then there is something interesting I can do which is I have tests I have no code which means I could do test first development right which is something on the lines of test driven development so let's actually try that out would it not be a fun activity to check so I have a blank application here which is more like a you know a spring boat Kotlin based application which I just directly created by going to start.spring.io and it's an empty shell right and this is a command which is essentially going to run the test based on the products API YAML file which is the specification for this particular app it's got one path here for products and it is a fairly straightforward specification it has only one operation it does the get and then for the response you will get back the product details with the name and the SKU that's pretty much all we are trying to achieve so what I'm going to do is start off by actually running the specmatic test command and I'm going to say the app is located at this location which is localhost port 8080 let's see what happens okay obviously it's going to fail because we did expect that right there is no application to actually respond so you got a connection refused so what I'm going to do now is very quickly just start off this app here and let's run it and as soon as the app gets going I'm going to kick off that command again so the app is running on port 8080 so now I go back to my terminal and I run the same command again now this time around you have a failure but it's not like connection refused you got a 404 and that's again fairly obvious because the application is running on port 80 but then you don't have an endpoint to actually support that slash product slash product ID right so why don't we go ahead and flesh out that piece of the code so what I'm going to do is quickly paste in some snippet of code here which I have and say for this path product slash product ID like any good developer would do I'm going to return hello world to start off it why not this is a perfectly valid endpoint that I'm adding to my code I don't want to keep going back to restarting the app and then going to the command line and then running it it's getting a little repetitive right what you could also do with specmatic is essentially have the contract test here essentially so what I'm going to do here is I have this contract test which extends from specmatic J unit support for this capability to be available I have included the specmatic J unit support into my Gradle build file so I'm going to show you that why I wanted to show you this specifically is I want to highlight that the J unit support the specmatic support is being added as a test implementation capability only so which means specmatic out and out is not going to be part of your production deliverable at all it's only during your test structure right so I've added this support and on the contract test side the only pieces of information I have to do is extend this and then what I was earlier doing on the command line here is to provide the coordinates of the application which is localhost import 8080 it's very same things I'm going to do here with you know the system properties during the setup stage and I'm also starting off the spring app then in the tear down I close the spring app that's pretty much all it's very basic plumbing no other code right now instead of writing you know executing this command line command I'm going to instead execute the contract test let's see what happens so last time we saw the 404 because the URL itself was not supported by the application now this time around let's see what is the failure so you got a test failure now and this time it's not a 404 it's still a 200 so what why did the test fail again fairly obvious because I returned a hello world but the specification file says that we are expecting a object with name and SKU right so obviously I didn't do a good job I need to go back and next do the next incremental step which is I'm going to add in the data object here which is the product itself very quickly let me paste that snippet in and I'm also going to return the actual product itself which the test is expecting based off of the APS specification okay and I quickly kick it off again will it pass now yes no maybe could pull down in the chat hooray we got the green it's always a joy right when you go from red to green and doing test development sort of approach yeah so this is fun we've gotten to this point but then this is not really how the real applications would look like right because you have to think about test data management now so let's take a look at that now if you look at the log here for this request specmatic randomly generated some ID 807 and centered and because we're always hard coding and test is always going to pass so that's not a real very watertight test in what is realistic is your database might have only one product or test data you have like 2-3 rows and anything else which comes in it's not going to be able to fetch so I'm going to simulate that situation by saying if the product ID is not equal to 2 so let's say that's the only ID that I have in my test data right I'm going to throw a runtime exception so I'm not entirely fleshing out all the repository layer and the service layer and all the DDD layers here but just for the purpose of this demonstration I'm quickly trying to say that this line here line number 14 is sort of representing your repository which says I don't have any other test data now let's kick off this test again and see what happens will it pass will it fail and if it fails what will it fail for that's the big question okay there is a null which is interesting let's crawl down to see what happened oh okay oh my this is a 500 internal server error and that's not good news right ideally we have to handle it but then first order of business is go from a failing test to a passing test and then we'll get back to this 500 problem so what is the issue here specmatic is repeatedly sending some random IDs last time it sent some other number this time it is sending 650 but in my test data I only have ID 2 so how do I tell specmatic that you need to send to and just some random number so what I am going to do is leverage examples inside of open API specification so I'm going to say value 2 for this particular parameter called ID and also on the response side I'm going to say that the name and SKU are these two values I'll save that in and then I'm going to kick off the test that's the only change I've done here now will it pass I'm going to keep asking you this question so it's going to be interesting if you can post in the chat what are you guessing so we could almost try pairing on this problem right okay so this time it passed again it's a good failing we went from red to green so we have a good rhythm going here in terms of the TDD so why did this work that's the big question if you look at what I did I gave a very specific name to this example on the request side called 200 underscore okay and I use the same name down here in the response for the response schema also now this is not a capability that's coming out of open API open API has request and then multiple responses right could define 200 400, 404, 422 so on but the issue is which request will generate which sort of a response code is not necessarily possible to define within plain open API so what we've done with specmatic is use this naming convention based mechanism to stitch together examples to say hey this kind of request produces this kind of a response right and that's how a specmatic is able to send it to and glean that yeah I got back a book and this is KU so it makes sense and that's how this test passed okay now let's get back to the original issue right which is it was throwing a 500 and which is not the right thing to do which means now I need to define how the application should behave in case of a 404 if the ID is not found so I'm going to quickly paste one more example in which makes sense I'm going to give it a meaningful name such as 404 not found and by now we also know the drill because I need to say I need to have a 404 response here right I'm going to paste that in also so right after here I'm going to paste in the error schema itself I'm going to say for the error I need the timestamp the status and the path and so I put in all those good stuff and make sure this is going to work so and obviously I cannot stop here I also need to give the example for this situation so I'm going to put that in right here I'm going to say this time around I name it according to the same name in the request but I'm not really defining what I what the timestamp should be what the status should be because these values will keep changing right I cannot hard code it into the example so which is why I'm doing the same thing which Joel did earlier right which is Specmatic is able to figure out if it is just a string or it can pattern match a data type or it can like you know match a very specific value so I'm going to say I don't really care as long as these four attributes are there in the response now with that I've modified the specification so let me run the test again and see what happens now notice how my behavior itself has changed right like practically this time I'm not changing the code I'm changing the specification first so I'm almost becoming a spec first developer or a test first developer in this case so in this scenario what happened we had one test pass and one test fail and which was the test that failed obviously the one with the 404 it sent a 0 it was expecting a 404 but then it got a 500 now we need to fix that problem now this is easy in Springboard that's not a very difficult problem I just need to define an error for 404 I'm going to do that right here right I'm going to define this imported if I start a problem I guess that's a string still no problem okay should be that yeah there this can be a little bit of a problem here and then once we're done that instead of throwing a runtime exception this time around I'll throw a very specific exception that I'm doing this is just Springboard way of doing it but then if you're using any other stack you could do whatever makes sense there now with that I'm going to run the app again I mean I'm going to run the test again and will it pass now and this is a lot of fun right because you're not really trying to you know just randomly just sit and write code and figure out whether this is going to work or not now I'm practically writing just enough code to fit the bill for the specification and every time I make a change I do make the change in the specification first which means I'm thinking about the API design first look how it forced me right because I did not have a 404 in the first place right now because I did something in the app and I realized I did not have I did not design the error code so I came over here I designed my error schema then I added a 404 response and then built the app so it's a nice little feedback loop for me to make sure that I'm pretty much in line with the specification I'm also building to the specification and nowhere have I gone off the actual track that I need to be on and just to call out I have used Springboard for this demonstration but then again Specmatic is both language and platform agnostic it's just an executable like I showed you earlier I can run it from command line as well but just for the purpose of convenience and since I have it I'm using the JUnit support for contract testing now that is this is what we called a tracer bullet approach wherein we use the initial specification more like an acceptance criteria and use that to flesh out the initial pieces of the code itself right and then from there on of course you will need to further flesh out this application by putting in your API test and then actually driving the logic forward and the further layers but this locks in your API signature and that's the real important part that you need to take away from this exercise so with that I will get back to the deck and one last bit which I want to leave with on the tracer bullet approach is think about API specifications as your done criteria one of the done criteria for your API itself and if you have that then this sort of an approach to use it as a test for building out your application is not just about testing your application it's actually designing your API better so that's the important piece so contract testing can often get taken for a testing approach but think about it it's largely about API design just like TDD itself is not about testing and it's more about design okay so with that I will hand it over to Joel to talk about this very very interesting topic called contract versus contract testing over to you Joel so let's quickly review what we've seen so far before moving ahead we have seen how consumers can stay in sync with the specification by using contract as stub to validate their expectations of how the provider API is going to work we have seen how the provider API can stay in sync with the same specification by using contract tests so that they know exactly how consumers would send requests and they can make sure that their responses will be easy to consume to consumers and so as long as both are doing their due diligence we know that they will integrate and that they can also both start development independently securing the knowledge that they are staying in sync with the contract and thereby with each other so this is good so far think we made a lot of progress and seen a lot of things since the start of the stock the story isn't over yet because we haven't yet seen what happens when we change the contract is it possible that there could be some compatibility problem introduced when someone changes contract and to understand that let's briefly take a step back and understand what we mean by backward compatibility essentially let's take the example of some provider API let's say you make a change to the provider this is now an updated provider you made some changes and you deploy it into some environment the consumers have not been changed yet now the consumer makes some requests to the provider which is now updated and all of a sudden the response from the provider is no longer understood by the consumer what does this mean? it means the provider is no more compatible with the consumer and we could also say the provider is backward incompatible why backward? because the provider has changed and moved ahead the consumer has not so this is overall what we mean by backward compatibility you could say the provider is now backward incompatible or you could say the change to the provider is a backward incompatible change and now let's bring this into the world of contracts pop quiz what if we make changes to the contract in the request let's say you add a mandatory required field is this considered backward compatible? in the world of contracts what that means is what we are really asking is if I make a change to the contract and the provider implements this change will the new provider still be compatible with existing consumers that have not changed alright and if that is not going to be the case then we consider the changes to the contract to be backward incompatible changes so to repeat my question in the request if we add a mandatory or required field is this a backward compatible change I am going to open my chat window and I would like you to post your responses you can just say yes or no I will just give it about 10-15 seconds you can just post your thoughts there backward compatible I will ask others I will give it a little longer I will ask others which is to post as well Santosh says yes how do the others feel I will leave you to monitor the chat hurry and let me know when to go ahead sure Joel we have one or two responses now just waiting for the rest just take a guess there is nothing this is a very straightforward quiz we don't have any curve balls here the curve balls are yet to come okay Santosh Kumar says yes Prashant says no so we have a fair mix Joel I will tell you to address the audience on why this is or this is not compatible okay so I think now the time has come to ask for specmatics opinion as well let me just adjust this so that I can see over the Zumba so let's make this change here I have the current contract as it stands today and I am going to make the change in another file so let's take this as an example you have seen this api before I think you are all familiar with it we have two files one is with the current contract which remains unchanged so that we can compare it and then there is another file with the new contract where we are going to make the change currently the two are identical products api you have seen this before we are using this to create a product we are passing a JSON object to it name and SKU at this stage SKU is not mandatory the question was what if we add a mandatory field we will add a mandatory field see what happens we are now going to say specmatics compare we will say products api current compare it with products api new and see what the result is the answer seems to be that this is not a backward compatible change let's reason this out for a second what has happened here new contract expects key named SKU in the request but it is missing from the old contract this means that if a provider had implemented this contract a compliant provider would now expect SKU to definitely be there because it is mandatory but if you put this in an environment with existing consumers where SKU was not mandatory then an unchanged consumer may not send SKU and if it does not send SKU this would not show up in the request and this might turn into some exception when the provider looks for it maybe in Java that might become a null reference exception in JavaScript that might become undefined there are various ways people languages represent a key that does not exist this key would not exist and would break the provider and hence this is considered a backward incompatible change what I'd like to point out before moving ahead is that because I ran this command on my laptop locally I did not even have to implement a single line of provider code or a single line of consumer code I didn't have to get any errors be it contract test errors integrated system errors there were no errors at all I got this just by making a change to the contract and comparing the old with the new and I got it within seconds no other code written so let's take another example in the request if I change an optional nullable to optional non nullable this what do you think this is actually also a relatively easy one what do you think I'll ask Hari to monitor the comments and let me know sure I've pasted the same question in the chat so you can stare at it for a little longer and because there are two aspects to it optional nullable optional non nullable so there's a truth table now you have to think through just take a while guess it's okay sometimes it's also a good idea to just trust your intuition on all of these things no okay we have no it's backward incompatible okay Joel I guess you can go ahead and you know break the mystery so let us try this out we will revert this back so that it's the same as before now we remove nullable right we are going from now SKU is optional optional nullable to optional compulsory let's compare the two for those who thought it was backward incompatible that is correct the wheels turn a little bit longer for this one though because of the fact that there are multiple factors to consider and I could see that it took a little longer for the first response to come in and the reason here is that there was a string in the new contract it was nullable in the old contract and that means that there was a new provider which implements this change is only expecting a string whereas old consumers may be sending null as well and that could break a provider because they are going to expect a string and get a null right here again there was no need to write code and you got this feedback without writing any code in the consumer provider you got it for free let's take a third example what if I change the schema component that is referenced somewhere in the request and response so maybe it is two or three levels deep maybe it is there in multiple parts of the hierarchy maybe it is there in multiple files maybe there are remote references and I want to show you an example of that before we move ahead what do I mean by this something like this you have a contract here that has several different APIs I will just show you what it looks like there is cart, inventory product, storage etc. this is an e-commerce contract it is a little more involved, a little more real world there are a lot of different APIs in here and all of these are different let's say that the change we want to implement is a compliance request let's say that the state is no longer nullable now we take nullable off what is the effect what is the effect of this change to the address field if I look at the address field it is in various areas basically inside the component section but it is not to be found directly in the request or responses and so we are going to have to go one by one and figure out where it is and the first one might be inventory response, is this a problem even here at all warehouse info contains it now you think through this and so on I am going to have to think this through for request and response across multiple different APIs, maybe I have thought it through for response but somewhere if I scroll down scrolled away devilishly is something in the request now I have this one request to think about as well but it is somewhere deep I might have missed it complex contracts are like that, real world contracts are like that, it is very easy to miss stuff, what is the guarantee that you are going to get it and this contract if once you start implementation it is too late, we want to actually get this feedback even before a single line of developer code is written, application code is written, before a developer actually picks this contract up to implement and a check like this is basically so easy and cheap to run and you get the feedback right here even before you start development, why would you not do it it is super easy, super cheap in fact you can even integrate this with your CI builds and make sure that backward incompatible contracts never even make it out the door in case you miss it on your own laptop it hits CI how that works I think Hari might we will probably talk a little bit about later but this is a very powerful tool that makes sure that even the contract, so now we have saved card at the consumer, we have saved card at the provider and this way we save card the contract as well a little bit about how we do that contract versus contract what did we say, Hari made a reference to this at the start this is a patent pending technique that I am talking about which specmatic uses it turns out that if you start up version 2 of the contract as a stuff which represents the new provider and you start up version 1 of the contract and run it as tests which represent old consumers if version 1's contract tests pass it means that the contract changes in version 2 are backward compatible because all the requests from generated by version 1 contract were understood by version 2 stuff and all the responses that came back from version 2 stuff were understood by version 1's contract this is a very naive explanation to work under the hood to make this pull it proved and fast but this runs within seconds as you've seen for all contracts now with that having covered backward compatibility I'll hand it over to Hari to talk about the next topic alright so quickly going into my slides so we've seen three major concepts which is contract test test contract test tab and contract versus contract testing and there's an overarching theme that you would have noticed it's a completely no-code approach so far especially the backward compatibility just to highlight what Joel was saying which is all you have to do to experiment with an API change is practically just go about making changes to the specification and then experiment away and you pretty much get a feedback on to whether this is going to be a compatible change or not that's the overall theme which we have been keeping in mind while specmatic out as a tool now with that I would like to call your attention to this topic called central contract repo and why is this so important and this big question about are we on the same page and what is the topic on which we need to be on the same page and why do we need to treat contracts as code so okay let's take a look at the situation now despite all the hard work we've done so far it's very much likely that let's say I'm the provider application developer I made a change to the provider code and then I forget updating the contract or I miss I update the contract I do all the due diligence there but then I forget to send the contract over to someone over an email or upload it to the documentation website or something like that and on the consumer side I may not be on the most current version of the open API specification I could have potentially not have downloaded it or maybe I just missed the email from the provider application team now that means both this consumer and provider application teams may be working off of a multiple versions of the same specification in their own sources of truth which means you're back to square one we have broken integration now that's not a pretty picture what we need is a single source of truth when it comes to the open API specification because that's when we really leverage the power of having a common agreement or a contract right that's when we started thinking about where which is the right place to keep this contracts and where else would you keep it it is code right contracts are code like open API specification is a YAML file why would you keep it anywhere else the ideal place for it to be is a depository a version control system in our case we use get we call it the central contractor if your specification needs to get to the central contractor it needs to pass a path of rigor right so that could be done through pull request or merge request process the initial step could be a linter where in you just verify that the specification is in line with your organizations or you know style guide for how you are supposed to build your API specifications and whatnot there are tons of tools out there the ones which we are using right now called stop light spectral it's a pretty cool tool and once you are done with the basic linting then comes the important piece which is the backward compatibility testing which Joel just demonstrated that is really powerful because that's going to let you figure out if the changes that you are making is a compatible or backward breaking change or not and only then you can move on to the next step which could be a manual review if at all necessary and then you can merge change into the git repository itself now you may ask a question what if the second stage of this pull request process comes out as you know like it's not backward compatible what do we do then then it means it's a signal for a version bump right so what we are following as a versioning strategy for our contracts is a semantic versioning but then again it's not the only way of doing it what why we find it useful is because it makes sense in the context of what we are calculating so if it is a major version upgrade because it's a backward incompatible change right in that case we might go from 1.0.0 to 2.0.0 and if it's a backward compatible change but there's still a behavior change in that case we might just go from 1.1 1.0 to 1.1 which is just a minor version upgrade patch version is reserved for only a structural change so for example if I'm extracting a mixin map from the central open api and pulling it out for reuse purposes only the structure is changing and the behavior is remaining absolutely the same in that case we might just have a notional patch upgrades this is the strategy we've been following now once that is done and you have the central contract rapport with all the contracts living there the next immediate question is how do I pull it down to the local environment how does the consumer application engineer pull it down to his or her laptop and likewise for the provider now that's where this file called specmatic.json comes in and that's a config file that you would have seen earlier that I was using and Joel was also using for the teaser and that's the one which is going to let specmatic know where to pull the contract from and what to do with it and by which I mean I'd like to quickly showcase what this json file looks like so this is the specmatic.json config file it has at the top the coordinates to your repository itself where your specifications reside ideally should be only one right and then you have two sections you have test and you have stop now a contract like you've already seen can be leveraged both as a test or as a stop and it depends on which context you're operating in if you're operating in the contract context of a consumer the contract may be leveraged as a stop and likewise on the in the context of a provider the same contract may be leveraged as a test so let's actually take a example of this so this is central contract repo that we are maintaining on as an example and if you notice here we have like just a few contracts sitting here and the naming convention for the folder structure is more like a typical package naming convention that you would follow in Java or C sharp or any of the programming language because it just makes sense right to organize your contracts in the same hierarchy as you would any other code because contract is code and once it is in here let's say I have this UI application which is the consumer which needs to pull these contracts and need to emulate the provider so in this case I reference the contract sitting there under the stop category notice how this API order B1.yaml is used as a stop here now this is the consumer application now for the provider this is the provider for the same consumer that we saw earlier here I won't be using it as a stop because I need to emulate the consumer so in this case the same YAML file is being referenced under the test category of the specmatic JSON and that's how you are able to pull it since it is Git and specmatic has been configured with Git it is always able to pull this application be it your local laptop or be it a CI environment or any other environment it's running on there is no question of the contract not being up to date so that's the important piece now with that out of the way let's actually piece all of these things together contract as test contract as stop contract versus contract and then you have central contract repository now with these four items you can put it into your CI pipeline itself how do you embrace CDD in the CI now you know that the central contract repo has the open API files and specmatic can pull from there and it can make it available as contract as stop for a server for your local environment of the consumer and also contract as test for the local environment of the provider this part we have seen let's actually take a look at how this pans out for the CI environment for the consumer in the CI after you have run the unit test for the consumer when you're component testing you don't have to look for another tool for stubbing out the provider you could pretty much leverage the same contract as stop server that you're using with specmatic on your local machine on the CI also because it's just an executable like I already mentioned so it can run in any environment and for the provider side once you're done with the unit testing we recommend you run the contract test first before you run your component testing the reason being it is important that you verify the signature first before you verify the logic the explanation there is the signature itself is broken there's no point in really testing the logic right so that's the reason for the sequencing and since at this point you have adhered to the specification on both the consumer and the provider both in the local and the CI for each of their environment you can confidently deploy to integration testing and you know for sure it's going to be compatible with each other which means you have an unblocked path to production and from the heat map point of view which we started this whole slide deck with you are very much in the green right and you are very much in the left so we are able to successfully shift left the identification of compatibility issues to the local and the CI thereby keeping your higher environments always viable for testing such as workflow and what not instead of always being stuck with integration testing now that's how specmatic is able to help you leverage APS specifications as executable contracts so that you can make independent progress on both building your consumer and your provider applications and confidently deploy them independently with while you can be sure that they're going to be playing that they're going to play along well with each other when deployed together in a higher environment and I'd like to thank all of you for joining and being a very patient audience we'd love to hear your feedback and this is an open source project so feel free to try it out and if you find any issues or need help do file bugs and we'd be more than happy to look at it thank you okay thank you so much everyone for attending this session special thanks to you I really enjoyed the session was really very well planned the demos and everything were really really well so thank you so much for everything