 My name is Konstantin Tenhard, and I have to warn you, this is not going to be a funny talk. I'm German, we don't do that. So, but I actually don't live in Germany anymore. Last year I moved to Ottawa, Canada, and there's so many Ruby developers in Ottawa. I actually work for Shopify. And in fact, we have so many Ruby developers that we just don't know where to put them and we sent them all down to Kansas to speak at RailsCon. And so, Kat already gave a talk, and there's two of us speaking later in the afternoon about how we test, and about sprockets. To continue with the shameless self promotion, you can find me on Twitter and GitHub, my handle is T60. And in my spare time, I'm maintaining a couple of libraries that you might find interesting. One of them is Action Widgets, which is a UI component micro framework, I'd say, for Ruby on Rails. In fact, the slides you just see here on screen are powered by Action Widgets. I'm maintaining smart properties, which is supercharged Ruby attribute assessors, as well as a processing pipeline for Ruby on Rails to model complex business processes. And finally, the one I want to talk today about, Request Interceptor, which is my most recent one, and it allows you to simulate foreign APIs with Sinatra. So, at its core, this talk is all about testing. And specifically, one type of test, tests that involve HTTP connections. So, most of you might know the libraries, VCR and Webmark, which are usually used to stub out individual requests, or in case of VCR, replay requests that have previously been sent to a remote API. I want to present a different approach today and talk about how we can use Sinatra to simulate a foreign API within our test suite. And that is essentially the core idea behind Request Interceptor. So, I guess I best show you how to use the library first and then throughout the talk we dive deeper and deeper into how it actually works internally to the point where I'll show what kind of metaprogramming techniques I used to hook into the Net HTTP library to make all that magic happen. So, yeah, I just mentioned it. Request Interceptor does modify Net HTTP, just like Webmark and VCR. There is no clean way to sort of interject yourself into what Net HTTP does. So, some trickery is required to make that work. But I get back to that later. The idea is that you can use any Rack-compatible app and use it as a Request Interceptor that sort of intercepts an HTTP request sent out by your application and reroutes it to your Rack app, which will then handle the request in line. And in fact, all you need to know essentially is that Request Interceptor implements a run method, which takes a Rack application as well as a hostname pattern. So, the hostname pattern is important to know when Request Interceptor sort of starts intercepting requests. It will actually look at the HTTP request and only redirect the request to your own Rack app if it matches the hostname. Otherwise, the HTTP request will be made just as a regular remote request. And the new code example here, I define the probably most minimal Rack app you could potentially implement. It's simply a Lambda statement that returns an array with a status code, no headers and a message, hello. And then I use Request Interceptor to intercept all requests that go to anything that ends with any host that ends with example.com. I do my HTTP request and then assert on the equality of the response being hello. The problem with bare metal Rack apps is that they are very inconvenient to sort of implement something more feature complete. You wouldn't necessarily want to go with Rack directly instead you wanna pick something that has a little more, that provides you with a little more convenience. And for me, this convenience is sort of given by Sinatra which sort of combines simplicity as well as provides you with a lot of flexibility on how to simulate these API endpoints. And for those of you who don't know Sinatra, it is a Ruby micro web framework and it's based around a very simple idea. You have a Sinatra application that provides you with more or less, well the most important methods are get, post, put and delete which correspond to the HTTP methods and they allow you to define request handlers in your Sinatra application. So they take a path as the first argument and then a block and the block defines how requests are being handled. So simple Sinatra app looks something like that. You don't even need to wrap it in a class or anything. It provides you with some magic to make this work and you require the library, you define that your application is handling anything that comes into slash hello and in this case it returns hello Sinatra. So given this conciseness and this simplicity, Sinatra was an excellent choice to sort of model APIs. And therefore makes a great combination with Request Interceptor. In fact, I went further because of this great combination, it's the combination I would suggest for you to use instead of using requestinterceptor.run with just any rack app, I would recommend using Sinatra and Request Interceptor gives you a define method which allows you to define a new Sinatra application with some extra goodness. So Request Interceptor allows you to define the hostname pattern. Again, just as we've seen before where we submit the hostname pattern and the application to the run method, we now define it right on the application and then we just define it as a regular Sinatra app with all of our endpoints that we need. And the result of this define call is a class again which is a Sinatra application with the added benefits. And one of those benefits is that this application provides you with an intercept method. And the intercept method is just a convenient wrapper for you around run. So instead of having to pass in everywhere where you want to use an interceptor, remember which hostname you wanna match and which application to pass in, you can just call intercept on your interceptor, provide it with a block, and then again a fire of an HTTP request and assert that the correct message is returned. And then more importantly in order to test this, you probably want to know how many requests you made, which requests you actually made and what the request and response data was. And to make this possible, the intercept method returns a transaction lock. So it's simply an array of request interceptor transactions. And these transactions are simply structs which give you access to the request and the response that was made within the block. And these are instances of RACMoc request and RACMoc response, just as other libraries usually use for testing RAC applications. I essentially use these to carry all the data for further inspection. And then the example down below shows you how you can, for instance, assert on the path of the first transaction lock entry. And in this case I'm just asserting that my program called the path hello of example.com. You can also nest them in case you want to have, you communicate with multiple APIs. And at Shopify I was on the team that implemented the Uber integration. We did that as a separate app. So for us, Shopify was, we also treated Shopify as an API, just as you would if you develop an app for Shopify. And then we treated Uber as our other service. So our application was actually had to communicate with both of these services. And it's often necessary that you know exactly which requests were sent where. And that is why request interceptors do support nesting. So both of these interceptors write a separate transactional log. And yes, of course the innermost interceptor takes precedence. So if you can actually have two interceptors responding to the same domain or to the same host, in which case the innermost would win and intercept the request. Another important feature is that you can customize an interceptor for an individual test because the idea is that you generally outline your service that you are modeling in a single file. And then customize it to certain behavior that fits sort of the needs of your test. Let's say you wanna model an error response for one particular endpoint. You would take your interceptor, call the dot customize method on it, and then override the previously defined endpoint. And Sinatra is smart enough that if you redefine an endpoint, the new endpoint will take precedence over the old one. And in this case, we are just switching the hello endpoint from to send another message. Previously it was high resconve and now it's Bonjour resconve. So now that you have a basic understanding on how they work, I wanna talk a little about the advantages in comparison to BCR and WebMerc that I think exist when using request interceptors. For me, one of the biggest advantages is that the code isn't cluttered throughout your test suite. Instead, what we do is we have one file that defines a particular service, in our case Uber or Shopify that implements all the end points we are usually communicating with. And then we customize this interceptor to specific needs in our test suite. But if you sort of wanna see in one go what your app is actually communicating with, you would just open the file and look at the interceptor definition. Another advantage for me is that interceptors provide greater power and flexibility because we're talking about a Sinatra application. You can literally go as far as you want with that. You could have theoretically an in-memory database that sort of keeps state if you wanna simulate entire workflows or you can keep it super simple and return static responses from your end points. So it's really up to you. Then of course, since it's essentially just one file, you can also go further and package it into a Ruby gem. Let's say you build a service that other developers use and you have a public API and now you want to make it easier for people to sort of integrate your service. You could provide them with a predefined interceptor they can use in their test suite. So they don't even think about hitting your API with like requests from their test suite. And then finally, and that is personally for me super important is that the code is just very readable, which is in the nature of the Sinatra application. And I personally think it's more readable than having these webmark stubs sort of scattered around your test suite. Instead, you have this one single application that defines how your interceptor works. And then there's more. There's features that I am not sure if you could simulate them with webmark or VCR. And so I wanna talk a little bit about more advanced concepts on how to use these interceptors. A big one for me is simulating network requests. Weakest interceptors are set up in a way that they propagate errors or exceptions that are being raised in one of the endpoints. So I specifically disabled the Sinatra's functionality to handle exceptions and propagate them through the entire stack. Which allows, for instance, to simulate that a host is unreachable, simply by raising the appropriate exception, which makes it very easy to test your application or the library you're building, whether it's robust enough to handle these error cases. And then of course, Sinatra gives you a lot of tools that you can leverage to make interceptor definition even easier and make the code more readable. And one of the most important things is probably that being a standard Ruby class, you can just define private helper methods that you can use throughout your interceptor and throughout the customizations you use in your test suite. In fact, you can just apply standard object-oriented design principles to, and all that Ruby gives you to sort of make your interceptors as readable and as easy to use as possible. Then there is the possibility of using Sinatra's before and after callbacks that run before or after a particular endpoint is hit. And you could, for instance, utilize an after callback to automatically encode data into JSON. Let's say you're modeling a JSON API. It's tedious if in any endpoint you always have to remember that you as a last step have to call to JSON on whatever you're sending over the wire. So just define it once in a block and in this case, I look at the response and if it's an array or hash, I encode it into JSON. And then of course you have the ability to use rack middleware. And in this case, we modeled both Shopify and Uber interceptors as API, as JSON APIs. And so we always wanted to decode the incoming JSON so we can easily work with that in our interceptors. And Sinatra provides you with a method called use that allows you to inject rack middleware that runs before your actual endpoint is hit. Now that you have sort of an understanding on how you use interceptors and why they might provide a nice alternative to VCR or WebMock, I actually wanna dive deeper into some of the internals because I just think it's interesting to see some of the powerful features Ruby provides and just as a sort of learning exercise. So in the beginning of the talk, I showed you that a requestinterceptor.run is sort of the core of the whole idea. And in fact, this is the concrete method implementation as it exists in the library. And there's essentially six steps and I will go over all of these six steps to sort of showcase how you can mess with an existing Ruby library that doesn't provide you with the ability to sort of do this in a clean way. So the first step is because you can reuse an interceptor is to clear the transaction log. That's very easy. I just clear the array that keeps all the transaction log entries from the previous run. And then I cache the original net HTTP methods because we have to make sure that once the block finished its execution, we restore net HTTP to its default behavior. And then I override the net HTTP methods with the custom implementation just as WebMock does as well. And then I execute my test and now my test will essentially use these overridden net HTTP methods. And then finally I collect my transactions and then eventually restore net HTTP to its former glory. And the last part happens in an insure part. So it's always guaranteed to run and so that it doesn't happen that your test suite actually gets into a state where net HTTP is not in its original state. So as I said, it's easy to clear the transaction log so I just wanna skip that and talk about caching the original methods. There's three methods you need to override if you wanna do something like incepting HTTP requests to start to finish in request. So to finish sort of take care of opening the TCP connection and then request performs the actual heavy lifting. And the way caching works in request interceptor, you have now a concrete request interceptor instance at your hand that is currently handling your test case. And I just assign these methods to instance variables and what instance method gives me is an unbound method. So I essentially save the original method implementation and just put them for now in an instance variable. And then I replace these three methods with my own implementation. Start and finish are pretty boring. I just make sure that net HTTP things, it has an open TCP connection it is communicating with but in fact I don't need one because of how do you redirect to this scenario application is working and I'll show that in a second. And then I define a new request method which is a little more interesting. The interceptor instance itself that is currently handling your test case has a request method of its own and all I really do is I take the data that would usually go to net HTTP request and redirect it to my interceptor. And then I also pass in the interceptor itself. I won't show the code for request interceptor request because it's a little more complex but I at least wanna explain what is going on and you can always take a look at the source code if you're interested. So the first thing I do is I try to find an appropriate interceptor meaning I look at the HTTP request and then look at the host name of this request and now go through my list of host name patterns and stored applications and see if one matches. If I find one, I now build a mock request and mock request, the initializer of mock request takes and rack application as its first argument. Once I have that mock request initialized I can call the methods get, post, put, delete on them to simulate an actual HTTP transaction. And once that happened, I get back a mock response which I now have to transform into a net HTTP response to make net HTTP believe that it actually just talked to remote service. And then I log the transaction meaning now I'm taking the mock request and the mock response and just writing them in my transaction log so they can be further analyzed in a test suite. The interesting thing is what happens if no interceptor actually matches your host name because I wanted to implement it in an unobtrusive way. I didn't want it to block just any HTTP communication especially to be still compatible with MapMock and VCR. So what happens is my current net HTTP instance which is now in this weird state that it talks to the Sinatra application has to be restored to actually be able and perform network requests. And the way I do this is shown on the next slide but once I restored it I essentially perform the request as if there would never have been any interceptors in the way. And method restoring works by utilizing Ruby's defined method would actually can not just take a block but it can also take an unbound method. So the ones we previously stored in instance variables we can now rebind to net HTTP and we can even rebind them to concrete instances of net HTTP. And it is sort of happening when the request interceptor doesn't find the matching application it rebinds the original methods to the concrete net HTTP request that's currently going on and then just calls request again and performs the request as if nothing ever happened. So that was essentially the internals of how the request cycle works in request interceptor. And if you compare that to MapMock there's certainly similarities with the difference that you define a stub within your test and in this case I redirect to the Sinatra application. I previously mentioned that there's error propagation that you can utilize to sort of simulate network errors and I just wanted to quickly show how this works. It is very simple because Sinatra supports it by just using particular configuration statements. So all you need to do to sort of have a Sinatra application actually raise an exception and not handle it and have the calling code take care of that exception is you disable the show exceptions and you enable raise errors. And by that you sort of switch Sinatra into an aggressive mode which does not make sense if Sinatra runs in a product as your production application but it makes a lot of sense to sort of simulate these network requests errors. Well, I do have further plans for request interceptor. So one thing I wanna implement is the support of traits sort of similarly named like the factory girl mechanism where you can define what your factory is building and then give it a certain trade of how it is actually building. And I want that for interceptors as well because I was running into the issue that I was simulating the same endpoint several times and what I did so far was just having a lot of these customized request interceptors but what I actually want is just in a particular test case I wanna have a name where I can refer to an endpoint definition and say I want my interceptor to run with a faulty implementation of my hello. And the faulty implementation could either be raising a 500 or raising a network error. And I wanna support different adapters. So I don't wanna just stop at net HTTP. The next thing I wanna implement would be Faraday because Faraday would give me exposure to several other libraries because I don't really wanna do a mess around with each of these libraries individual. That is sort of the two goals I have in mind right now to bring this library forward. And that basically brings me to the end of my talk and I just wanna quickly summarize what I've been talking about. So request interceptors sort of provide a third alternative to VCR and Webmark. The thing I like most of them is that I have a concise service definition in one place instead of scattering this definition across the entire test suite. And they provide me with an easy mechanism to customize them if there is the requirement in a certain test. And then finally, Sinatra provides we with a lot of simplicity and flexibility which ultimately leads to very readable code which is just something I greatly enjoy. If you're interested to take a look at the slides again because I know it was a lot of content I was going over, they are available online. Thanks a lot for your attention.