 here is at the last session on the last day of RubyConf. I don't know if that means that you won or lost, but you're here, so that's great. So my name is Michael Bly, and today I'm going to be talking about a new framework that I've developed for building RESTful APIs in Ruby. If you wanna follow me on Twitter, I'm at mbligh. You can also follow Intridia, who I work for, just in case. So if you wanna ask some questions, there's a little app that I set up, just go to comfast.herocu.com, you can record questions during the talk. This is not actually any kind of a cool app, this is just a demo for this talk, but I thought I'd write something that might be useful, so go ahead, log in with Twitter, throw up some questions during the talk, and I'll get to them at the end. So this is not exactly the talk that I wanted to give, and I don't mean that I'm not interested in the subject, I'm very interested in the subject. The problem is that I practiced a development methodology, I'm not sure if you're familiar with it, it's called CDD, does anyone know what that is? Conference Driven Development. So I submitted this talk, and I've had this idea for this framework in my head for a really long time, and it's something that I've really wanted to work on, and several months ago when I submitted it, I had it all scheduled out when I was gonna work on it, when I was gonna get everything done, you know, things didn't go exactly as planned, so now I find myself in a position where I'm worried that I may be over-promising and under-delivering, but we do have stuff to look at, and it is, I think, an interesting framework, an interesting technology, so I just wanna throw a caveat out there up front that this isn't quite where I wanted it to be, so if you wanna refund, you know, hit me up on Twitter afterwards. Not for RubyConf, I can't give you that. All right, so first I'm gonna talk about the motivation here. So why would I build a new framework? You know, Ruby is a wash with web frameworks. Why do we need yet another framework? So Ruby's really good at making hard things easy. Rails makes the process of building web applications really easy. Sinatra makes building simple applications even easier than Rails does, but the problem is that APIs still aren't really easy. How many of you have built a RESTful API of some kind for an application built in Ruby? Okay, keep your hands up if you thought that the process for building it was exactly how you'd like it to be. So still a couple hands up, but for the most part, you know, people's hands went down, and I think that's because there hasn't really been something dedicated to the idea of building APIs in Ruby. APIs in Rails are too entangled. Rails 3 has responders you can serve up XML or JSON straight from within your controllers, but I don't know, that always makes me a little nervous. I don't like the API being so tightly coupled to the web application, because then you're gonna run into problems where I want my API to be versioned. I wanna make sure that my API continues working even if I change the structure of my web application. Rails doesn't really provide facilities for that. APIs in Sinatra are a little too manual. I mean Sinatra's great, Sinatra's simple, but if you wanna do things like support several different formats, if you wanna have maybe a URL that takes multiple HTTP verbs, I mean you just sort of run into these limitations where Sinatra's great for simple things and it's great for an API in theory, but I feel like in practice, I just continually get frustrated if I'm trying to build an API in Sinatra, like why isn't this thing just a little bit different so that it can work the way I want it to? So why can't APIs have their own framework? Ruby, like I said, is a wash with frameworks and that's a good thing, that's not a bad thing. That means that you have choice and you have a variety of ways to tackle a problem and so this is just a new way to tackle the problem of building APIs in Ruby. And a lot of the inspiration for this also came from new Twitter because you look at the new design of Twitter and basically it's a JavaScript client for their API which is built in Ruby, Scala and these other languages, but I mean primarily it's driven by the API and I actually really like that. I like the idea of web applications that are driven primarily by their own API because that sort of frees you up and it both makes you create a robust API for your developers and it also just sort of gives you the freedom to do so much of this on the client side and with technologies like Mustache, you can do the front end and the back end using the same rendering engine. If you then use the API everywhere, you're talking about the same stack throughout your application whether you're on the front end, whether you're on the back end, whether you're a developer or whether you're actually building the application. So that's another sort of inspiration for why I think that this sort of framework is needed. So today I am announcing the first release of Grape and it's 0.1.0, it's on rubygems.org right now. This is a very beta 0.1.0 and I'd like to tell you that Grape stands for the generalized rapid API erector or something clever like that but in reality I was just like, ah, I want something that sounds like API. Eight, nope, that's already taken. Grape, nope, somebody else that in Triti is working on something called Grape. So hey, Grape, that sounds kind of like API. So to give a high level overview of what Grape is and what it does, Grape is built for ease of development. The idea is that you can sit down and if you're building an API, you just bust it out in Grape the same way that you would easily bust out any web application in Rails or a web application in Sinatra. It's designed to remove all of that friction and all of that manual work that you have to do to think about and set up API development. It's inspired by Sinatra because, I mean really, an API fits pretty well into Sinatra. Since you're talking about resources that are accessed directly by their URL, by developers, it makes sense to have a really URL-centric view of what you're doing and it's inspired by Sinatra, but then it builds on Sinatra and gives you some easy facilities to do some things that Sinatra can. And the biggest idea is that it should just work. I want you to be able to sit down, write something out with Grape and it just works. It works the way you expect it to. It takes care of all of the defaults so that you don't have to worry about it and you don't have to think about it. So what exactly does Grape do? And this is actually really kind of two questions. There's what does it do now and there's what will it do in the future? So let's first answer the question of what does it do now? So to get started, let's go through the very basics. So this is the most basic kind of Grape application you could build. You create a class, you extend Grape API and then just like you would with Sinatra, you can say get and then a URL segment and then within that URL segment you return an object that can be serialized. And so you see if I run this code here and I run that through rack because my API is a rack application that you can run directly in a rackup file and then I put a get request to slash hello, then I would get a JSON representation of the hash that I returned from that block. So right now Grape has built in JSON serialization. It assumes that you're going to be building your API and targeting JSON and I wanted to have more formats supported for the first release but ultimately I think that especially in a crowd of Ruby developers, JSON is really kind of the way that you're gonna wanna go anyway. It's really, I mean I don't really, does anyone prefer XML to JSON here? Yeah, let's, this is actually, I'm gonna add an extension to SOAP and then drop the JSON support later. So JSON is just for now. No, but JSON is a really easy and really expressive way to build APIs because you're representing things directly as objects. And that's something that as developers we can easily understand, easily interpret, JSON can be serialized into a number of different languages, yada, yada, yada. I don't think I really need to sell you on it. So basically what happens is whenever you return from one of the blocks that you set up in a Grape API, first it looks for a serializable hash. So this uses the same setup as active model where if you define a serializable hash on your object, then it looks for that method and says, oh, okay, then this is what this object declares as a representation that can be serialized into JSON. And it should just be a hash that can be serialized into JSON, that's the only requirement there. And actually you could fudge it and it could be an array or it could be a string, anything that can be serialized into JSON. But I wanted to follow the active model pattern there. If that doesn't work, if it can't find that method, then it looks for a to JSON method because you may have written a custom JSON, you may have written some custom JSON handlers for your specific needs and so it'll look for that as well. And then if that fails, then it just tries to encode the object in JSON directly assuming that it's going to be something that your JSON interpreter of choice handles. I use multi-JSON for encoding so it'll work with the JSON gem, Yajel, active support, JSON pure. So it'll work with any of those JSON libraries. And then the idea is that this would also work with XML or this would also work with Atom or RSS and the idea would be that all you do is you return your content and Grape takes care of figuring out how to serialize it into the correct format. So we can also do prefixing really easily. So if I want to have my API prefix so that everything is slash API slash something, then all I do is I put prefix API at the top of my Grape API and then from there all requests are presumed to be prefixed. You can do versioning and this is actually kind of an interesting one. So you can specify a specific version and if you specify a prefix and a version, the version will go after the prefix. So this gives you the ability to do things where it's only accessible from a specific version. And if I want, I can actually specify multiple versions in that argument. So I could say version V1, V2, and then all of those methods would be available for both version one and version two. And if you want something to be only available for a certain version, then you can actually pass a block to the version and then that'll create an encapsulated segment of code that works only for those specified versions. So if you wanna do legacy support, I could have version V2 at the top and so by default everything supports V2, but then if I want my legacy code to still be there for V1, then I'd just say version V1 do and I keep my legacy code there. So it provides an easy facility for adding versioning to your API. Namespacing, so namespacing in its simplest terms is just basically adding another URL segment. So if I want it to be slash users slash something, that's a namespace in Grape. But namespacing also gives you the ability to do some interesting things that go beyond that and we'll talk about that in just a second. But the basic way that it works is you say namespace and this is actually alias to group resource and resources. So it just sort of gives you a vocabulary that you can use to describe your API however you want. But here I say namespace admin and then I have a metrics namespace under that and if you do just a get or a post or a put or a delete with no argument, then it will work just against whatever the current namespace is. So if I do get do, then that means that if you look below get slash admin slash metrics is going to run that block that doesn't have an argument. Or you can add additional URL segments and just like Sinatra, you can use string regular expressions where if you put a colon in front of it, it will parse that out and turn that into a parameter for you to be able to access. And that actually runs using rack mount, which is the same thing that the Rails 3 router uses to parse and handle arguments and strings. So if it works in Rails 3, it'll work in Grape and that's something that I think is nice to have. But what I mean by stackable configuration is going back here, in this namespace admin, if I wanted to, I could add for instance a version argument or I could add helpers or I could add other things and those are only going to be available to that namespace. So there is a stack and a scope where it will inherit from everything above it. So the metrics namespace will inherit all of the settings from the admin namespace, which will inherit everything from the global namespace. And so you can really easily do those kinds of scoping operations that I find come up frequently when building APIs. It also supports basic auth and this is really simple. All you have to do is add an HTTP basic call with a block that takes the username and password and then you can do whatever kind of check you want. In this case, let's say I have an admin portion of my API and that's protected by a simple password that's set in the environment variables. That's how that's set up here. So now if I try to get my admin metrics, if I try to get slash admin slash metrics and I haven't supplied the admin username and password, then I get a 401 unauthorized. And if I supply the correct password, then it passes through serial as the JSON works as you'd expect. So HTTP basic is the first form of authentication that I've added. It's definitely not gonna be the last and we're gonna talk about more of sort of the roadmap features shortly, but you can also do helpers. And helpers work pretty much how they do in Sinatra. You say helpers do and then you can define any number of methods and then those methods will be available within all of your endpoint actions. And these are scopes. So if you define helpers within a namespace, those are only going to be available within that namespace and they're going to override any helpers that were defined in a super class or in a super namespace above that. So that would let you really easily tailor things using the namespacing, using namespacing and using these inheritable helpers to make your application work better. And finally, you can also handle API errors in a consistent manner. All you have to do if you want to say, all you do is error exclamation mark and then you give it a message and you give it an HTTP status code. By default, the status code is 403 so if someone tries to create a record and there's a validation error, then you can just error out with the validation message there and it'll be handled in a consistent manner. So basically the idea is that it just gives you a quick one liner that can consistently display API errors across your system. So that's sort of a basic overview of most of the things that it can do right now. And here's an overview of some of the things that I've already started work on but aren't quite ready for production right now. First of all, documentation generation. And this is the hardest thing for me to have had to cut from this talk but I just haven't gotten it working yet. One of my, one of the most fundamental principles I have about building Grape is that I want you to be able to write your API code and then using extensions to Yard, you will actually be able to generate the public facing API documentation for your API directly based off of your code. So in the same way that you can document your code now, you can use the extensions to Yard that I'm gonna be writing to document your API and all you'd have to do then is throw that up on a public site somewhere and you've got API documentation. How many of you have had to write some form of API documentation? How many of you thought that was pretty miserable? Exactly. So I wanna do away with that completely if possible and I've already had some small success. There's a long way to go but I do believe that it is going to be possible to automatically generate your API documentations if you build an API with Grape. Content negotiation. So this is actually already mostly built. The only thing that I need to do is write the ability to actually serialize out to multiple formats. So Grape will automatically do sort of the gamut of things you would expect for formatting an API. So the first thing it'll look for is just if you specify a default format and you can do that just using default underscore format but right now it only supports JSON so there's no real reason to talk about that. But then after that it'll look for an extension added on to the end of the path. So if I say slash users dot JSON then it knows that I want my results to be delivered in JSON. But then even beyond that it also uses the proper rest idea of the accept header and it'll parse the accept header looking for known media types that it can serialize out to. And so basically I found it really frustrating for instance, Goalas API uses the accept header but most APIs use either some kind of URL extension or URL prefix. And so there's all these different ways of doing it and what I really want it to be is just simple. So Grape does all of these automatically so your API supports all of these automatically without you having to add any code to it. I wanna support OAuth 1 and 2 authorization and I've written up, Grape is actually a collection of middlewares that are sort of applied to this custom endpoint application and so they actually could be used outside of building a Grape API but I've already built an OAuth 2.0 middleware. I just need to properly hook it into the DSL and figure that out so basically I want it to be really simple for you to just pass through and it'll properly parse the OAuth headers and check that against either a class that you specify or a block that you specify to find users based on an access token. So it will support OAuth 1 and 2 in the near future. I also want, one of the things that really bugs me about Sinatra is you can't really build an application in multiple files. I mean you can but it's really messy and sort of not encouraged. So I want Grape to be able to be built in multiple files because you're not going to just have, unless you have a very small API, you're probably gonna want to separate out and have different resources in different parts of your application and I want it all to work together and I want that all to be seamless. So Grape will be supporting multi-file APIs and also a plugin system. This is gonna be very extensible. I've already got everything running pretty much as middleware so it's also going to support just sort of a public API and plugin system so that you can easily extend Grape to do even more crazy things and that's where we get to the vaporware and these are ideas that I have for Grape that I haven't started on. I'm not gonna pretend they're coming in the next couple weeks but I can see how it could be done. Streaming APIs, if you want to build a streaming API like Twitter's you could hook a Grape API up to something like Event Machine and build something that would allow you to do that. Pub, sub, hub, hub so that you can automatically persist changes to your application to subscribers. A presenter class and this would be actually really easy to implement so that you can create presenters for your various elements. So if I have a user, then I can create a presenter for my user that Grape can then use as the basis for serializing the user. Rate limiting and internal API is sort of the last interesting one that I wanna touch on. I want people to be able to use the API not only by calling to it via HTTP but also by calling to it programmatically. So I want you to be able to call directly into your API and retrieve out the serialized Ruby object without having to convert it to JSON and then back from JSON again and waste that overhead. So basically I wanna make it so that you could actually use your API to drive the data inside your Rails application or inside your Sinatra application so that basically everything's going through the API and you have just this single point where you're handling your data model. So now I'm gonna talk about CompFask which is just a really simple program that I wrote today using Grape and first, so we have, where is the GitHub repo for Grape and how can we help? That is a great question and I can answer that right now before I get on. So by the way, if the same user and the person who asked the question, I have asked three questions and it seems to have been posted the same one, the my first one seems to have been posted. So if I ask two questions because they didn't give me my form, any question from there on, it's just a piece of paper for the person that I'm asking. I think that's actually. I don't know if I was able to submit that issue. Yeah. So github.com slash intradius slash grape is the answer to that and that's actually really strange because I think I ran into that problem about half an hour before this talk and then I thought it went away but apparently it has not. So I'm not sure why that is but let's examine the code for this anyway just very quickly. So this is the API and basically the application is set up. It is a simple rack application running Sinatra and Grape. It uses Omnioth to authenticate with Twitter and basically I just run a Rack Cascade that first goes to the application and then the API and so let's look at the application. There's really not much here. I just have a callback for the Twitter authentication that will create a user persisting the users with Mongo Mapper on a free Mongo HQ instance on Heroku. So that's all there is really to the application. It just renders the ERB and actually none of the data that you see is rendered using Ruby. It's all pulled using the API in JavaScript and then rendered there. So if you look at the ERB the question list is just empty and all of the actual logic takes place in application.js and this is really hacky and not best practices at all but I thought it worked and it apparently doesn't. So then let's actually look at the API. So I gave it a prefix and a version and then the first thing I did is I defined some helpers and those are around the authentication and since I'm calling these API methods directly from JavaScript then that gives me the ability to actually inspect the same session that the Sinatra application is using. So I set a session variable of user ID in Sinatra and in order to authenticate it actually checks against the session. So whenever a JavaScript is calling against the API it can automatically authenticate using the internal session and while that's not actually ideal for an external facing API because you would want people to be able to authenticate with their account using OAuth or using HTTP basic or something like that it's actually very useful in this case because now if you're thinking about things from the perspective of like new Twitter where you want to have an API driven application you can really easily do that. So I have a few API methods. First we have get slash me which just gives you information about the current user. That's not actually used in the code anywhere but then we have the questions. So the first thing we do have is resources questions and then we just have, actually shouldn't even be there. So we have resources questions and then all we do is we take all of the questions and sort them backwards and then it will automatically serialize that to JSON. So using my HTTP client I can change this to go to Confask hero coo. So you can see that the same way that it displays here, it displays here and I'm not sure why the question keeps getting repeated but the idea is that but it's actually persisting to the database that way. So I don't actually think it's a problem with grape. I think it's a problem somewhere in my JavaScript or somewhere in between but I do apologize for that. So you can also post a question and that just takes a text parameter and the screen name of the current user and creates a new question and that is actually done just by doing a simple Ajax post whenever this form is submitted. Let's see if I can do it. Yeah, did the same thing. That's very strange. So really there's not too much to this application but it gives you an idea. I mean, I wrote the API portion of this in probably about 10 minutes and it's very simple and it doesn't do a whole lot but I could feel even when writing it that that friction is gone. The friction of getting from nowhere to writing an API is just really reduced by using a framework that's built from the ground up to handle APIs. So, since the questions app didn't work I'm gonna go ahead and take out lab questions if you have any. Yeah. Sorry, can you get into the rack? Have yourself throw me a summer from Rails app? Yep, absolutely. It's just a rack application. You can, you know, mount it using the Rails router. You can run it in rack up. You can throw it. Yeah, it's just a rack application and the class itself is a rack application Any other questions? Yeah. The previous slide you had was looking at the, I guess maybe it was the code slide, maybe I don't know which direction it was in. It was when you were talking about the namespacing with the versioning. Yeah. So, you had a prefix and a version. Yeah. And you can, so you can pass a block to the version you say, but is this when you get to the multiple file support that allow you to basically say that file's my version or API and add files on the version of API? Yeah, exactly. Same, same, same point. It's just a different prefix before that. Yep. Yeah. Yep. I'm doing all this simple operating. Do you have things that you want across available and across parts, but not all of it that you recommend, say for example, using as a mix-ins, for example, for helpers you want from like version one and then you created a tooth and you decided to bring it back in three or something? Yeah, I actually, I'm going to be stealing in very short order the ability to pass just a module into helpers so that that would automatically include that the same way that it happens in Sinatra so you can pass a module into the Sinatra helpers method and it'll automatically include that and that'll be available to all of your endpoints. And you have multiple, I like to get calls if you want multiple routes to go to the same place so you have to reduce the code or could you pass an array of routes going in perhaps? Multiple routes. Right now you would have to write it multiple times but that's something that's always bugged me about Sinatra so I'm going to add that to the GitHub issues and make sure that I support that because I definitely want you to be able to have multiple routes that run the same action. Yeah. So this should, let's see if we work with JRuby there's no reason why there's not. Yeah, this should work just fine in JRuby. It's all pure Ruby. It uses the multi JSON library for serialization so as long as you require a JRuby compatible JSON library before you load grape then it should work just fine or a JRuby compatible JSON library. Yep. So I saw that the standard, your companies GitHub thing but that has nothing to do with your companies and then you want me to load this into something commercial. I mean this is something that I built because I've had to build a number of APIs now at Intridia and I ran into the same problems again and again so this is definitely scratching our own itch from a company perspective but I mean as far as commercializing this I mean I definitely don't plan to sell like the grape framework or anything like that. I mean if it grows and a community evolves around it I'm just happy to contribute and have things contributed back. Yeah. Have you thought at all about having unified lifecycle helpers or something like that? In other words you have a lot of objects that have basically the same process for get, post, delete. Doesn't it make sense to extract the data the process for that and use it as a common helper? Yeah I have actually thought about for instance when you create a record especially if you're creating a record in Ruby most likely either the record is created or there's some kind of validation errors so I do want to create those kind of built-in lifecycle elements where if I'm doing a post then I have some kind of facility where and just throwing this out there off the top of my head it'd probably look if the object that's returned is active model compliant and then check based on that whether it's a new record or not and if it's still a new record then it'll check for validation errors and display those and otherwise it would return the newly created record. So I definitely want to build in those sort of invisible ways to help ease development. So yeah that's definitely something that I'm thinking about. Anybody else? All right well, oh yeah. Yeah, the slide deck I'm going to upload it probably just to the wiki attached to the grape repository and so the slide deck will be available and I'll also probably and I'll also be posting the code for that compVASC application although since it's not working that well I don't know how great of an example it'll be. Yeah, yeah so I mean you know I really I feel a little bad because this is a lot this is a lot more beta than I wanted this talk to be but I'm still excited about the technology and I just hope that the people here can maybe get involved and make suggestions as to what it can do and also fork it yourself add functionality. This is a brand new framework I'm very open to suggestion because I think that there's a lot of people who have a lot of good ideas about how this should be done so I just really hope that grape can keep growing and help everybody in the Ruby community build better APIs so thank you very much.