 It's great to see so many people here, especially so early on a Saturday morning. My name is Aditya and I am a risk engineer at Stripe. Stripe is the payments backbone for many companies, including a number of our sponsors here today. Companies also like Lyft, Kickstarter, Slack, NPR, UNICEF, and many more. I'm on the risk team. I spend my days defending our users and customers, people like you, from online fraud. If you're interested in hearing some battle stories or tales of what it's like on the dark web defending against users like these, I'm more than happy to talk to you about that later. But today I'm here to talk about API design and testing. Let's take a trip back to 2006, 10 years ago. Facebook had just opened itself up to high schoolers and college students were freaking out about this. You may even have had a MySpace page. Actually, how many people had a MySpace 10 years ago? Okay, good. I was afraid only like 10 people would raise their hand, which would mean that most people in this room were liars because, let's admit it, we all had MySpaces back then. 10 years ago, the best practices of web development dictated that you'd do something like this. You write a web server and it would respond to HTTP requests by talking to your database, doing some computation, and then spitting back some HTML, and then the browser would render that HTML. If you wanted to allow someone to write a native client for your application, you would, the native client would talk using some raw format at the time, likely XML, and would still talk to the server. The server would talk to the database, do some more computation, and then spit back some data in that raw format. And this is kind of terrible. It's asymmetric. It's, you know, different clients are using different mediums to talk to the server to do fundamentally the same things. Fortunately, at some point, we realized that this was silly, and we consolidated it. So this is what the state looks like in 2016. Now we realize that the browser is a special type of client, but it's still just a client. And it talks to the server using the exact same format, the exact same protocol as any other client would, whether that's a desktop client and a mobile client. And now the server doesn't even need to care or even know what kind of client it's talking to. And this is beautiful. It's symmetric. So in writing symmetric APIs, there are a few points that you might want to keep in mind. First, you want to design your end points, you want to design your end points first. One moment. The timings are just auto-advancing, so just try to turn that off, thanks. So design, you want to design your API end points first. You don't have to go overboard with this, but you do want to make sure that your interfaces are clearly defined from the very beginning. Even if you're just prototyping and you have no intention to make backwards compatibility promises or anything that is still in the alpha stage, you want to define the ways that your client and server are going to be communicating very first from the very beginning. Designing your API first will force you to think about the key functional components of your application. If you've ever taken classes on UX or UI or product design, you'll learn about user stories and storyboarding. This is essentially a lightweight way of doing that. The interface defines the way that your users are actually going to be talking to your product. And finally, you always want to develop your API server and client together. Even if someone else you think is going to be providing the client for you, you always want to write at least one client for every API that you're building. If you don't do that, you're not going to know what your users are going through because you haven't actually done it yourself. As for testing, there are a few different ways that you might want to test APIs. And I polled many people, and no one can quite agree on exactly what the correct difference is between integration tests and functional tests. I think I talked about 10 people and got 20 different answers about that. But fortunately, for the purposes of this talk, it doesn't really matter too much. We're just talking about degrees of granularity versus interconnection. So for this talk, I'm going to use integration test as a catch-all to mean anything that tests more than a single application function, anything that tests how multiple application functions work together, or anything that relies on non-application state. So for example, a database call. There are a lot of great things that we get out of integration tests. First of all, integration tests, they give us the behavior that our users actually experience. It's one thing to test all the ways that our pieces should work together in isolation, which is what our unit tests give us. But we also want to test the way that our code really works in practice. Integration tests can also give us information about changes that might occur in the external environment. So for example, if you've upgraded your database and you now use a different format to talk to it, you would never actually know if you're just running your unit tests that your client or your server is suddenly broken. But unfortunately, the same things which make our integration test powerful can also make them fragile and unreliable. They're reliant on external systems, which is great because they can catch errors due to changes in our external systems. But that's bad because it means that if our tests start to fail, we don't necessarily know whether it's because of something that's outside our project scope or responsibility or something that we can control. Is it something that's due to our external environment or something that's intrinsic to the project? There's a great story that I'd like to remember about this, which was some years back, NASA was building a satellite and they were sending it up. I think it needed to dock with the ISS. And they did a ton of testing on the ground level because this is NASA. And they made sure that the probe would extend exactly the right amount from the body of the satellite to dock with the ISS and exactly the right distance. Fortunately, when they sent it up, they forgot that space is a vacuum. And the exact amount of force that you need to project something a certain distance from the body of the satellite is just slightly less when you don't have the pressure of an atmosphere. And so it all failed and everything, the project was a disaster. This is not actually a picture of that, but I imagine it looked something like this. And so it's really important. They did all of their unit testing on ground level, but they forgot about their integration test, their system test. They didn't catch the changes in their environment, or in this case, the lack of environment, of outer space. So most projects will have a mixture of unit tests and integration tests and functional tests, which requires duplicating a lot of code. It's good because we get better coverage for our overall code. But at the end of the day, it means that there's more code to maintain for our tests. And more code to maintain means more greater chance of bugs being introduced. And it's one thing you have bugs introduced into your project, your test will catch those. It's really bad if you end up with bugs being introduced into your tests. So let's take a look at an example. We can look at Anaconda, which is the Twitter client library for Go, which I wrote. Anaconda was one of the first Go client libraries to do a lot of things. Namely, it returns native structs. You can think of structs kind of like native objects. Anaconda was also the first Go library to handle rate limiting behind the scenes. So most client libraries, so if you make a lot of requests to Twitter over and over and over again within a short period of time, at some point they'll tell you, you have to slow down. Wait. And most client libraries will return this as an error and expect you to figure out when you can make that request again and then try that again yourself later manually. Anaconda will actually figure out when you're supposed to retry it. It will replay it for you and then give you the actual result, all behind the scenes. I'm using Anaconda as example client library because it's open source, so you can take a look at it later if you want. But remember that for symmetric API testing, we'll want to test the client and the server at the same time. Even though I wrote the library, I did not write Twitter myself to go along with it. And likewise, all the examples that I'm using are in Go, but these are design patterns that are pretty much language agnostic. You can do this in any language you want. So one approach to testing that's pretty common is to stub out or to mock responses. Here's the example JSON response in the Twitter documentation for getting a user's information. And here's what a struct that corresponds to that would look like. We'd un-martial the JSON into this struct. And you can see a pretty clear correspondence. So you have contributors enabled, false, that's a Boolean, this is a Boolean right here. You have created app that's a string. We'd want to parse that as a date, but there's a one-to-one correspondence there. And for now, we'll just write this out manually. We can mock these JSON responses to create the native types and then use these native types in the rest of our testing logic. But we have a problem. What happens if our response structure changes? Our tests are still gonna be using the old stub responses. They're not gonna know about the new structure that the server is giving us. So this is a real case. Twitter used to return a string to tell you if a tweet was censored in a given country, or a user was censored in a given country. And at some point they changed that. So now instead of giving you a string, it gives you an array of strings. In other words, tweets used to be only able to be censored in one country, now they can be censored in multiple countries simultaneously. And because the tweets use stub responses, or the unit tests use stub responses, this problem only occurs in production or when running your tests against the live API. Now you might think, well, I don't need to worry about this because this all has to do with static typing and I use a dynamically typed language. But actually you do need to worry about this. Gee, to continue with this example, what if you're writing JavaScript and you call dot length on a field value and you think it's a string but it's actually an array? It's gonna give you an answer that's valid semantically but it's not at all what you need. And your tests hopefully will catch that. So whether you catch that ahead of time at the point where you're un-martialing or later on when you're testing for validity, you still need to worry about it. So wouldn't it be nice to be able to get the reliability of unit tests but also have the completeness of our integration tests and wouldn't it be great to be able to do all of this without having to write any additional code whatsoever? Unfortunately we can. So we're already looking at the API client so let's start with the client side of the tests. We start by creating a local server and here's how these first lines just set up a local server for us in our testing code. And then we tell the API with this line that instead of querying, sorry, we tell the client instead of querying the actual API, query our local server. And then we can construct, oops, we can construct an endpoint here, let's say my endpoint and hard code the example response that we wanna give it back for our unit tests. And now we can enable or disable this local server with just a flag or in fact, we only need to enable or disable that one line which is telling the client whether to run in live mode or whether to run on the local test server. Now we could hard code the response for each and every single controller but there's an easier way. Instead we're gonna eavesdrop on the live responses and record them and then replay them. So here's the actual code, not our test code but our actual code in the client for making an OAuth authenticated request against the actual API. And then reading the response, un-martialing the JSON and dumping that into a variable called data. It's easy for us to intercept and record these responses and we only need one line to set this all up. That's it right there. tReader does exactly what its command line counterpart, t, would have you think it does. It reads stuff and returns exactly every single byte that it reads but it also copies those to a separate place, a file. And what we're doing here is we're just wrapping the response body in a tReader and when we read from the response body to un-martial the JSON, it will also copy that to a separate file. Notice that the teeing, it doesn't actually happen on this line, it only happens when we read from it. What that means is we don't have to read the entire response, hold that in memory, dump it to, write that to a file, keep it in memory and then pass it to the JSON un-martialer which will then do stuff with it. We can actually operate on a buffer or on a stream. Those interfaces, the reason that we can do this is because response body provides a common interface called read for, you guessed it, reading and writing data. In the standard line, we recognize this. And so goes interfaces mean that we can record and replay responses in test mode while still preserving our functionality in live mode. And again, we can enable or disable this line with a single flag. Now in Anaconda, all HTTP requests are dispatched through one source. The original reason for doing this is that it allows the client to handle rate limiting for us behind the scenes as we said across all different endpoints. But an additional benefit is that all HTTP responses are also dispatched through that same source. So it means that we don't have to add this line to every single, this line right here. We don't have to add it to every single endpoint, sorry, the functions that correspond to every single endpoint separately in order to record and replay those tests. We only need to add it in one place. This is a small but important point. It means that updating our tests or adding new ones is dead simple. It dramatically reduces the barrier to ensuring that every single API endpoint, every single call, is at the bare minimum tested for structural integrity. There's no reason to ship untested code when the barrier to creating minimum viable test is so straightforward. So instead of creating a separate handler function for each in hard coding responses, now we've recorded the files. And fortunately, RESTful APIs give us a natural way to organize those files within our project hierarchy. And for example, the endpoint for slash user slash lookup, we can store that in a directory called JSONs, JSON slash user slash lookup dot JSON. And you might be wondering, why do we call that directory JSON? Why don't we call it responses instead? Because we're reporting the responses. And the reason is that we wanna test this symmetrically. The same data types that the client reads in the responses are also used by the server. The client un-martials those JSON responses into a native type. And we already saw that a minute ago. But how is that response generated from the production server, as opposed to our testing server, which is just reading from a file? Well, the production server will take a native type that it gets somehow, it does some computation, and it marshals that into JSON and then sends that over the wire. That's our interface. And the native type is used on both sides of the network interface. So why not share it? And we do. Instead of actually writing that struct definition out manually, as I said we would a few minutes ago, in reality, Go structs are already isomorphic to valid JSON. And so the computers can do all of that bookkeeping for us manually. And I know that in some language communities there's a fear of code generation. Fortunately, we're doing stuff that's fairly straightforward here. We're not gonna get too wild and in the weeds. We can use Go JSON, which is a tool, for doing exactly this. It generates struct definitions from JSON documents. So you can just pipe it, any valid JSON, and it will generate for you all the code that's necessary. And you can use this on both sides of the interface in both the client and the server. Even in dynamically typed languages, you won't necessarily need to write out your object model like this, but you still might have to for interacting with your database, like say a database schema. Even if you're using a schema-less database, you still have an implicit schema, the schema that's implied by the ways that you're interacting with your models. So you might as well define your contract explicitly because you have that contract, whether or you have that interface, whether or not you're being explicit about it. And so we could run this on the recorded or question responses every time we change the recordings, but even that's a little bit cumbersome. And fortunately, there's an answer for that as well. Go Generate is a tool designed exactly for this, for generating code that we want to check into version control based on external schemas. You use it a lot with parser generation, but also with stuff like this. So this is one line, Go Generate, and you insert this as a comment in any file. And now when you run Go Generate in your project directory, it will read all of the recorded responses in or recorded JSON in that directory and update all of the schemas, all of the struct definitions across your entire project. So this way, we're generating our native type definitions based on the actual real values that are being exchanged over the client server interface. We've guaranteed that this interface is not gonna change without our tests and therefore also our code changing as well because a portion of our tests and a portion of the code are being generated exactly from the interface definition instead of the other way around. We're taking our symmetry to the next level. Now, Go Generate is a Go specific tool, but you could use other build tools such as Make or whatever build system you have in order to get similar effects in other languages. And I wanna take a brief moment to talk about interfaces because they're actually the secret to symmetric API testing. So if you take away nothing else from this talk, this is the part to remember. We've been using the word interface a lot. An interface is simply a shared boundary that two components use to exchange information. It's an abstract concept. It exists in every programming language, but some languages provide language level support for interfaces as a type. Go is an example of that and here's how it works in Go. So any type I can define the writer interface and I'm defining that interface by the write method. Any type that provides the write method with this exact type signature can satisfy the writer interface. So in this case, it takes in a buffer that you're writing to and it returns an integer and possibly an error value. That might be null. And the same thing goes for the reader interface. It's a very simple concept. You're saying I don't care where this data is being stored or even how it's being stored. I just care that I'm writing it somewhere and that's all you need to know. And so here's how you might use that interface. This is a pretty silly example. You probably wouldn't want to do exactly this in practice, but we've got a struct and it's got a body. That's a string. And we can define the read method on that struct. And as you can see, the read method, it's got this signature right here. That's exactly what we've got right here as well. And all we're gonna do is copy the body from the struct into the buffer for the read method. You wouldn't really wanna do this most of the time, but it's just a straightforward example. Notice that unlike languages like Java, which also support interface types, in Go, the type that implements the interface doesn't need to declare anywhere that it's trying to satisfy the interface. I haven't used the word reader anywhere even though reader is the name of the interface. And that's because the interface, the type that implements the interface doesn't even need to know that the interface exists. You can think of this like duck typing in dynamically typed languages like Ruby. It is very similar, but the key difference is that unlike duck typing, this behavior is known at compile time. You're never gonna get an undefined method error by trying to call a method that should be defined on an interface and find out that your variable doesn't, or your value doesn't actually satisfy that interface. The compiler will check that for you. It'll say, hey, you're trying to call read on this. You're trying to use this value as a reader, but this doesn't actually provide the read method. It's like duck typing, but it's on steroids. And this is why we're able to eavesdrop, record, and replay our responses so easily with just that one line of code. We have a simple common interface for reading and writing data, regardless of the underlying data source and its implementation. Another example of why that's powerful. Interfaces help our code to document itself. So here's a file, or sorry, here's a function. You can probably guess without any comments what this does. It's called parse file, and it takes in something that can be read from. I will tell you right off the bat that the file type does support the reader interface, and it returns a config and possibly an error. So if you had to guess, what would you think it does? Read the config file, yes, this is not a trick question. It reads and parses a config file. But here is a trick question, or maybe not, but slightly harder. What does this actually do to the file? If I passed in a raw file, you would have no way of knowing without looking at the implementation whether or not this function tries to change the ownership of the file, whether it tries to change the permissions, whether it tries to delete the file. You would have no idea. You'd have to look at the implementation. But inside this function, because it's now a reader, the only method that it actually has access to is are the methods defined on the reader interface. In this case, there's only one method, and that's the read. That's the only thing it can do to the file. So you don't have to look at the implementation. It self-documents, and that is all caught for you by the compiler. It's duct typing, again, but taken to the next level, just a little bit safer, and more self-documenting. This is why our interfaces will allow us to separate our functionality from the identity, the function that our values and our types are performing from the identity of the values that are performing that function. And in symmetric API testing, interfaces are our axis of symmetry. The same data validity guarantees apply on either sides of an interface. You want those to be symmetric. Now, this doesn't mean that you need to have interfaces as a language level feature in order to implement this pattern, the way Go does. It just facilitates it if you do. It makes it more explicit, and it makes it more robust. As an aside, in a lot of dynamically typed languages that support duct typing or similar concepts like Ruby, Python, JavaScript, you'll notice that in the wild, people are oftentimes a little bit afraid to make assumptions about their code, or to make assumptions about what a function will actually do. And that's partly because there isn't really a great or an easy way to guarantee, to make the same statement. I'm gonna guarantee that all I'm ever gonna do on this function is call the read method. And even if you look at the implementation today and you know that that's true, because you haven't specified that contract, it's very easy for someone in fixing a bug doesn't realize that this is the implicit contract that's being specified, and break effective uses of duct typing in other applications because that contract was never made explicit, or that contract might only have ever been made in the documentation, but isn't caught by any static analysis tools. Interface types are one very powerful way of making this contract explicit. You can use static analysis tools to make that same contract as well in other languages. However you do it, you want to make sure that your interfaces are being thought first. That you think of your interfaces first when you're designing your code, and you also think of your interfaces first when you're tooling, in your automatic tooling, just like you do with your tests and with your builds. I'm just scratching the surface here with what interface types can do, and Go's interfaces are much more powerful than this. If you're interested in learning more, I presented on more of the details of advanced uses of Go interfaces at Goforcon in India and Dubai this past February, and I encourage you to take a look at that. I posted, I tweeted that earlier yesterday. I'll post it again later today if anyone's interested. Now until now, we've been looking at testing just the client, but we can use these same techniques for testing the server logic as well. And the same technique applies, but in reverse. We record the actual client requests and we'll feed them directly to the handlers on the server. So this is what that would look like. Here's our handler. I could think of it like a controller. It's a really silly endpoint. It just prints that string no matter what you do. And we create the response writer and the request object. Now the interesting thing to note here is we're never actually making an HTTP request to example.com. We're just creating the value that represents the request as the server would see it. So the server would see this and say, okay, I'm responding to a request that was made to example.com. But we're never actually executing that HTTP request because we just feed these values directly to the handler. The handler is a function. We call this, there's no network activity going on. We call this directly. And one other thing to note, data right here, that's just an IO reader. We can pass in that data through anything that can be read from. We can be using this in our tests. We can read that from a file. Of course, this leaves us with the question of what we actually want to test on the recorded responses. The most obvious thing, you might want to test for state changes, like database state. If you're testing the sign up endpoint, you might want to test that that a new user is actually created in the database afterwards. And you also want to validate the responses that are returned by the server. So for a sign up endpoint, you might want to test that an actual user object, the JSON representation is being returned in the response. But, haven't we already done something with validating the server responses? And yes, we totally have. Because the client and the servers are both using each other's recordings. The client uses the expected, the mock responses, to verify that it does what it's supposed to do with the data. For so, for an end application, you might test that it's rendering the Twitter timeline correctly. And the client also uses the live responses to verify that its own recorded responses aren't obsolete, and it records them if they are. The unit test, the recorded responses are there in case we are not able to use those live responses, or in case there are issues there, we still have those to fall back on to test other aspects of the validity of our code. And the server uses the expected, the mocked requests to verify that it's doing what it's supposed to do with its data. So for a server test, you might be, for a server app, you might test that it's generating the user's timeline correctly. But both of them use that interface contract, the JSON, to generate the native type definitions. Because that interface, that's providing the access of our symmetry. It's defining our contract, it's defining that scene, that layer, across which the different components of our application will be communicating. Oops. Now, you might wanna, you might be asking about other tools like ProtoBuff or XML. And I'm talking about JSON, but everything that I've written here can apply to other data interchange formats. So if you wanna use ProtoBuff, which is, it's similar to JSON, but sort of like JSON turned one notch up. It provides some extra features. It's better, it's more compressible, so it's good for streaming lots of data. Like the Twitter firehose would be a good use case for it. If you really want, you can use XML as well. I would really, really advise you for your own sanity. Please, avoid using XML for your API if you can. But sometimes you have no choice. I found a health tech company before this, so that was very much what we had to do. And everything about symmetry applies regardless of the data interchange format. People sometimes also ask about Thrift, which is a tool, and there are many others like it, for specifying schemas and then generating code based on that schema. It's an option, but it solves a slightly different problem. I would say that if you, you probably know if you need it. The reason that you wanna use the JSON schemas is that we're testing that our schema matches the actual interface. The JSON that we've recorded is how our client and server actually behave, and that's what we need to test. Not the idealized model of what we think it should look like. So, whoops, in short, we have a few, we have a few, I don't know why this is going like that. We have a few principles that we can use for API testing. We already have tests that compose and degrade gazefully, so our tests, sorry, APIs that compose and degrade gazefully, so our tests should as well. You always want to write at least one client for every API that you're writing, and you want to lower the barrier to writing tests because that's the only way that you're going to get good test coverage. At least test your structural integrity, but ideally test more than that. And again, if you remember nothing else, remember that interfaces are powerful, and interfaces are the key to symmetry in your application. Your API's power comes from the strength of the interface contracts that it defines, whether implicitly or explicitly. Thank you.