 I have a cute title, but I really didn't think of the shtick. This is more information only than a cube, which is like totally goes against the range of most of my thoughts. Who am I? A bit about myself. Recently I joined Sheila Packard's cloud computing group. So building out some cool tools like those that you probably already use. So keep an eye out for that. By the way, we're hiring. So we're ramping up our team over the next couple of months. So if you're interested in the cloud, whatever that means these days, and just sling grouping, then let me know. Also, we have a podcast called the Change Log. So who has open source projects upon get up? Play everybody. Cool. Anybody have anything we featured on the Change Log? If you don't want to, you can have them. It's one way back there. Whoops. All right. Move closer if you want to. That's easier. All right. Purpose of this talk. API wrappers. This is kind of what I do Sunday afternoons when I go to an apartment in football. A lot of API wrappers. It started when we were consuming the Twitter API for an application called Tweet Congress. It was a directory and an aggregate of all the congressional U.S. Senate and House Tweeters and compiled all their tweets and just had a directory to find your congressman on Twitter. And we were submitting a lot of paths across the job and the maker created the Twitter gem. And I think that got to become a nuisance for John. So he said, well, he won't use Tweet Congress. He seems to be more angst than I am. So I maintain the Twitter gem now. That led to LinkedIn. A gem I wrote about octopusing, which is a part about occupying some others, but I really didn't care for how they wrapped. I didn't get any APIs that I would want to call out. Let's see. Formstack, tried to find, let's go down on, too long probably. So why create API wrappers in the first place? I mean, we've got Perl. Perl is a great way of interacting with API. It's past the URL. You get some JSON back or XML. Got to admit it. It was also made HTTP. It's the same thing in Ruby without every dropout in the system. But we're Rubyists. We don't want to deal with those low-level libraries. We want more Ruby in our API. So idiomatic access. A lot of this comes from a post that Chad Fowler wrote back in, I guess, 2007 or so. It was a big debate between our Facebook and Facebooker at the time. And approaches that each of those took. So what I mean by idiomatic access, well, let's say we have a JSON hash that's returned. This turns into a Ruby hash. I notice the camel casing keys. I mean, you can still consume it, but it's just probably in here. You don't know Ruby, right? So we're going with those Ruby 5. I'll show you some tricks to do that later. So one of the cool things about writing an API wrapper is you can give the data to the language that is consuming it and something that holds a little bit more natural. So we take those camel cases and go to Underscore. It seems to be the convention. I don't know if Ruby is set in the pace, but in the JSON world it seems to be the convention for key names. But a lot of times you'll see PHP service or Java service expose those key names in camel casing because that's how they are behind the API. I'm saying it's true for method names. So if we have an API method that's user, getInfoByEmail. I think I took this from the big API. We could expose InfoByEmail. It's just a little bit more idiomatic Ruby. There's also syntactic sugar. We're used to this in Ruby. So the error functions that are in Rails 3.0 that we can do these big chain methods to create the database. We can do the same thing. Here's a case in the Twitter gem to do a search. So we can chain methods together to build our search. In this case, we're calling for all tweets that are from Johnny and Meg are too many to hash with Ruby. And it actually doesn't compose the URLs we call fetch, and then it forms the search. So method chain is a big thing that we can do with just the dynamic nature of Ruby. Also see this in action in the remixer gem, which is a rabbit for the Best Buy gem. We have a method chain here as well. This one is a remixer in your APIs. In the world of REST, we typically think, okay, we have a REST API and it returns JSON. But that REST term is really loose. There's a lot of API styles out there that return JSON. In the case of Best Buy, they have a really crazy way that they compose URLs to do things like finding all of the stores within a given radius of a zip code and returning all the products at a price that's greater than $3,000 in this case. They even have it where you can reverse those terms and find all the products that are greater than $3,000, but then return the stores that have them. So it's cool to be able to Ruby, construct those in different methods and have method chain construct your URL a lot of the way out, regardless of how it's called. This is another technique. I haven't seen Hayes here, but Hayes Davis is a local Rubyist. He's got a Twitter client called Rackle. This is cool. He takes a different approach. He has method chain here to construct the URL. So notice the client.status.update.json that follows the Twitter API to the REST URL. But he adds the bang there for post. So if you call it without the bang as a get, I'll put the bang as a post. Very mindful of the weakness. There's also different approaches to the API wrapper. So there's simple wrapping. So in this case, Michael Blight from Intridia has Twitter off. So you can authenticate via Twitter. And also that user now is a pseudo client that you can call the Twitter API and do some things on behalf of an authenticated user. So he uses wrapping here. Notice you're passing in the URL that point status is update.json. This is the status of tweet tweets we're going to tweet from Twitter off. Now Rackle has six different approaches we just saw. We call this wrapping with styles. Buzz Lightyear would say. And then abstraction. John takes a totally different approach with the Twitter gem in that he wanted something, he originally built it so that he could tweet from the command line. So what he wanted was just an update. So it hides the actual URL behind the scenes. You don't have to know about the API in the column. So why would you simply wrap? Well, the biggest thing is you can insulate against change. Your API and your client looks just like your API on the server. So the API on the server changes for the most part your wrapper will keep up. And you don't have to recompile or refact the API to keep up with that client side and the server side change. You can also leverage API documentation. So the documentation at the server side is pretty much one-to-one on your client side. So you get to reuse that documentation and you're interested in having a separate set of documentation. So then why would we abstract? What's the use case there? Whereas Chad called it and posed writing APIs around the APIs. Well, oftentimes in the case of Facebook you might want to simplify your complex API and hide all that garbage underneath the facade so that you don't have to deal with it on a day-to-day basis. You also might want to provide a business domain so you want to cache internally and provide some local value add in a way that you're consuming the API you might want to abstract the API. Tools of the trade. This is kind of where I'll camp out because this seems to be where a lot of my research on weekends goes. Transport layers. So we all know about NHDP. It's built in Ruby. Anybody know the problem with NHDP? It's not pretty safe. There you go. Who said that? So doing serial requests with NHDP is a different quest. It's difficult because that's going to be queued up and it becomes 10x how your requests are. So Patron gets around this and so these curl, these look girls fill in the room or do you bail? Do you bail? He knows about this stuff. So Phil Tallin has a library called The No-Shirt Fill. A library called Patron that uses look girl of the hood gets around this. This is another one that is supports parallel a lot of the box. So a lot of sites, Amazon's a good example when they compose their front-end pages and make numerous ASQ requests on the back end to close that page. They didn't let serial requests take much longer than when they parallelize it. That becomes much quicker because then your request is only as long as your longest request. Another approach is an event machine, we've got this great debate going on in the community between threads and defended type of architectures. This is one of the tools we've got if you're using an event machine. Another big aspect of writing review wrappers after you decide on your transport layers is your parsing layer. So if you guys normally return an XML on Jason or if you guys return to Jason who wants to deal with the XML and the type of issues that come with it crack is probably one of the more better-known ones that comes with ASQP party. Crack will parse XML and Jason. Not sure if it does the animal or not. But basically this code was a code that John left out of MIRB and Rails proposed into a journal called crack. I think that's the party part. Jason, there's Jason Pure and there's also active support if you just want to use Pure Ruby to parse Jason. That's also an option. Although it can be slow. There's also Yatchel Ruby which drops down to C extensions to parse Jason which is much, much faster. We switched to this in the Twitter gem a while back. There's been some complaints because it doesn't play nice on Windows. So they treaty you guys to come to the recipe again with Multi-Jason, which is a gem that gives you one interface to parse Jason and it looks for the quote-unquote best library and machine. It will look for Yatchel first and then Jason Pure and then active support and you dial the line. Higher level library. So this is usually where when you're writing an API unless you're doing something extremely quick and dirty you want to have a higher level library to consume so you're not that close to the model. You should be part of everybody using this to join your maker. So this is the guts of the Twitter gem. We'll talk about that in just a second. So if you've ever installed anything that defends an ACB party you probably got this message which makes me crack up when I install the latest version. So ACB party gives you, it's a Ruby module that you can include in your class and it gives you basic wrapper to get post to put in the lead and your class basically turns into ACB client. Wraps things like basic auth and a base URI. So you don't keep passing the whole URL for your API endpoints. So base URI and then everything else is relative. It handles things like default parameters like a lot of APIs will have tokens that you can pass with every request or some sort of authentication scheme or identifier that every request needs to have so you can run it around. It uses net HTTP for transport so it does have the serial problem and it uses crack for parsing JSON and XML which is both good and bad. So a sample ACB party class, in this case a delicious wrapper, includes ACB party such as base URI and includes to initiate your class and the recent method is just the wrapper for the post-recent endpoint passing in the basic authentication we call it at the top of the stack. It's also a monster mash which is a cute name. It basically has ACB party for frutitius. You want to go for parallel support. REST client from out of wickets is another approach. This gives you a kind of a DSL to compose your wrappers. It supports active resource out of the box which is both good and bad. But if you're into that sort of thing it also supports nested resources so that you can just want it as active resource endpoint to get it kind of fully ready to go wrapper out of the other side. It has a built-in shell which is really cool. So you can crank up a shell in the command line and interact with the service directly from the console which is really nice. WERI from ART Launch. This is one of my ones to keep an eye on. It also gives you a simple DSL. What I like about it is you can specify required and optional parameters in the method level. So for folks consuming an API that may not be an expert with the API this allows you to kind of mark up your methods in a way that tells the consumer these options are required. These options are optional which can then be output into your documentation which is really nice when you try to get it passed to a method. It also includes async support, kind of a callback mechanism. So WERI looks like this of the player method foo, in this case, and specify the URL whether the verb and the volts to get it, in this case we're going to tell it to go via post. It requires parameters of ID and VAR, but also as an optional one of VWAR is really helpful in it. Authenticates false and involves redirects and I can specify some custom headers. That's another big thing you're wrapping APIs. There's a lot of times you have to supply custom headers for things like authentication. So all of that turns into client.foo. It gives you a method that you can call it in with an options hash. A different approach is rack client. So this takes rack that you're familiar with on the server, kind of brings it down to the client. So it gives you a rack API with a request and a response to make a request and those can be queued up to middleware to hack on this request when we're out and the response is when we're back in. I'm not sure if it's actively developed. It was one of the reasons why I didn't jump into rack client in this slightly documented. About the same time I discovered Fairday from TechnoE, from Bulson. It's given us so much in the community. It's also rack light that supports middleware, which is really cool. He has the notion of adapters, so you can specify your transport on either per call or per library basis. So in this case, in the URL, we're hitting the Twitter API, so we set our base URI and we break up with the Fairday connection based on that URL. So we need an adapter, the default adapter, so you can set that to netHTTP, TPS, Tron, whatever you want to use, and that is set once so you don't have to worry about it. And then, in this case, I'm using middleware for multi-json and master file, which you can get to those in just a second. But on every base on the connection level for every request, I'm going to have this middleware stack so you can compose your client just like you do if you're composing a rack application. It's really nice. So then to call it, response equals the connection.get, which is your verb, do request, you have optional overrides which you can do on a per request basis and you just call it. Easy peasy. So Fairday middleware is another add-on to this that I've created. It kind of bottled a couple of the common patterns that I was doing repeating from wrapper to wrapper. So hashing. Anybody use hashing? Check it out. You can do the sort of thing. So hashing is a method hash. We'll get into that in just a minute. But I like to do it with bracket notation. I like to do it with more of an open-struck method hash when I'm consuming data from an API. So hashing does that in multi-json. Again, I've discussed and do the multi-parsing. So it seems like every request that comes back I want to parse the json in some way turn into a method hash. So let's do that. And then based on the API you can sometimes need o-off or o-off too. So I can throw those in there as needed. So this is pretty much my current stack. So hashing. Hashing makes full use of this thing. So match, dash, drash, and clash. I think we started to dive into each of these. So match is a method hash as we mentioned. So we'll have a hash with bracket notation of a key of foo. I can then just call dot foo and it does use method missing and anything like that. I can pull that back and direct. What's cool about this is to get deep hash that comes from an API or some keys, values are hashes and cells, others are arrays and it will do that deep traversal and parse. I get object notation as far as my object goes. Dash is just like match, except instead of just being free-form that anything that's a key is now a method. If you want a more strongly typed hash to speak, you can explicitly set which properties on the top of your dash should be enabled. If you want to use some sort of encapsulation or something. So trash is a translation hash. It's built on an array. The earlier example I showed where we would redefine those keys, this is where we would use that too. You can specify patterns where it takes camel casing keys in terms of the underscore keys that that's how you want to consume. And then clash does the Errol type query building pattern that we showed before where you can dive into the hash and go to those queries and fly. Let's check it out. I put the URL in the treaty. Errol it. Who's used Errol it? Cool. You use Errol in command line. Errol is basically Errol in the web is the entry of the Rails were all last year. They were four. This is Chris and Leah's entry last year or whatever they made it. It's really cool that you can go and put in your URL for your API point and test your API points. So you can add parameters at headers and it comes back and it will give you syntax modeling for the return JSON and XML. A lot of API documentation sites have something like this built in now and they're all teams done this where the API explorer so these are really handy if you need to debug the API call and figure out problems with the API itself or in your wrapper code. HTTP scoop anybody play with this? Cool. So if you need to figure out what API call is actually being executed from your code so sometimes you're posing these URLs in the fly and you're not sure in your wrapper code with the browser on your end or on the server end and you need to see exactly what's going back and forth. HTTP scoop is just a Mac app that you can download and bind to a particular interface either Ethernet or your HTTP traffic and show you a log you can even filter it by domain wildcard domain things of that sort and you click on each of these entries and you get really nice request response formatted JSON or XML payload and headers that you can inspect and figure out exactly what the problem is it's great for debugging. Charles proxy is another it's really similar to HTTP scoop anybody use this? Charles is the same sort of it's just listening as a proxy server so when the GoWallup iPhone app came out I was dying to play with the API and I haven't gotten around to building it yet so I was able to take the iPhone app point it at my laptop using Charles proxy and sniff all the calls so then if you have an iOS app an hour and an Android app native app you pretty much have an API you can even install certificates of venison and SSL traffic so it's pretty cool to be able to sniff out hidden APIs I didn't check it from the great wall of China or anything like some other folks did testing so a big piece of creating an API wrapper is testing a big piece of my toolkit is FAKEweb FAKEweb allows you to fix with fixtures for your API calls you can use curl it or curl grab your fixtures, dump them in the fixtures folder and then stub out the request put a URL in there and say that a call makes this URL matches this URL pattern then serve up this fixture file instead and you can lock it down and approve an error to say try to make a network request outside of anything you step down so you don't have to pound your API or you're testing and running your tests it's pretty cool VCR is a project built on top of that so FAKEweb it's up to you to get your fixture from your API VCR allows you to configure a layer on top of FAKEweb that the first time you'll go out and grab the fixture, dump it to a subsequent request if you try to make a call again which is really cool to auto create this fixture for you the problem with these types of libraries is that they only allow you to test the parsing of the JSON and the XML that comes back there's a lot that goes into an API called headers, responses, things of that sort where you may need to really lock the server that you're testing against and not just pin up fixtures some tools that we've got are part of this you do that so you basically activate a Rack app to spoof your API so you can build a FAKE version of your API ShamRack does the same thing, it's a great name so I can rack up a Sinatra app and spoof my API so this allows you to test against the local version of your app and this would be a great thing for folks building multiple applications on top of a well-known API to have these shams out there a Twitter sham that everybody could test against without having to call Twitter because that's destructive, actually send out tweets while you're testing and they know that the API changes but this is one point that we can manage that change it also plays nice just as Rack Nowhere ShamRack is truly a Rack application so authentication seems to be a big issue when you're writing API wrappers one of the biggest things, one of the biggest barriers of writing an API wrapper is just getting the authentication piece working seems like there's a number of ways that the API can do this, by far the easiest is just basic it's not the most secure but it's the easiest somebody supports basic authentication you can set a credential file or something in your home folder and in your test we pull that up and use basic authentication that's built into a really straightforward OAuth is a giant PIA but it's more secure OAuth allows you to let the per body site do the authentication instead of having it's broker so instead of dealing with the user name and password of the user now we're giving a set of tokens that expire or can be revoked so it's a lot more secure for the user in that case but the flows in OAuth 1 are incredibly complex how you get these tokens, you have a set of consumer keys to your application you have to pass those and you get a pair of request keys and you get a pair of access keys it's a crazy flow especially for mobile applications so they revamped this into OAuth 2 and other than the first 5 letters they have nothing in common for the most part so don't expect an easy upgrade path from OAuth to OAuth 2 if you need to do OAuth 2 applications the Intridia OAuth 2 gem which I think dropped the same day that OAuth 2 was announced is really great one of the things you run to in OAuth 2 is that OAuth 2 is specified in different types of devices where you have web flows and mobile flows and things of that sort, user agent flows in your home problem is a lot of the providers do the web flow and they stop so if you're trying to do mobile applications they don't really have it that's pretty much it any questions on UPI wrappers? is Twitter on OAuth 1 or 2? Twitter is on OAuth 1 and I'm not sure as a service when they're going to upgrade to OAuth 2 thanks for bringing that up so we're currently in a refactor we're taking advantage of the OAuth which may have already passed the time of the last couple of weeks where they're shutting off their basic OAuth and they're going to go OAuth only muddings and mudding tomorrow we'll finish up the refactor we're switching over to Faraday and Faraday Middleware just to take the opportunity to cut half of the code the Twitter gem has really been an evolution of code so we had different paths the four of the pattern for basic flows and OAuth flows and we're using this opportunity to really simplify back to Faraday and have just one flow for the application anybody else? a couple of useful tools you can mention are Wireshark and the REST client plug-in for Firefox okay cool I'll check those out in case you didn't know that's Wireshark and the REST client plug-in for Firefox another tool I was going to suggest is a big web called ephemeral response ephemeral response works like that it will cache the request but you can specify how long so you can say well in case the API was going to break I want to re-fetch all that data once a week stores it in memory or does it store it? it'll cache it on the file system for you but then it'll reuse it on the request until it expires and it'll talk to PPI again which project is that? Sondra okay ephemeral response alright thanks everyone