 We have a really good demonstration of this last night. Anyone that wasn't there, there's going to be a repeat performance time. Where is it, Rick? Do you know the name of the place? Well, wherever he goes, I'm going to drink here. BBC, BBC. I've made a few things, so if anyone's ever used Ice Cube, which is an idea of recurrence library before Easy Translate, there are the whole things that I've made. I'm sure you can find other things that I've made also. And I work at Patch in New York, so just a quick shout-out for my employer, who I said I would, and I came all the way from New York. If you live in New York, or want to live in New York, and you're a Ruby developer, come find me. We do really interesting things. We're able to freely contribute to open source as ourselves. Which is really cool, and that's a nice benefit, which I know a lot of you have, but a lot of you also don't. So the plan for today. First, we're going to talk about do what. So we're going to talk about what do I mean when I say splitting your app. Then we're going to talk about why you would want to do that. Then we're going to talk about how you can make it happen. And then we're going to talk about when the best time to make a shift like the one I'm about to describe should be taken to your plan. Where's that little iPad thing? There's plenty of time. Do I have time? So I'm actually talking about a few things when I talk about splitting your app. It might not be apparent, but the first thing I'm talking about is services. So I'm talking about taking your app and breaking into multiple pieces so that you can hopefully get the benefits that are tied to services that we'll talk about in a second. But the second thing that I'm also talking about is APIs. So as we're splitting the app into pieces, there's no reason that the pieces that we split off can't serve as APIs themselves. And this API might be a public-facing API or it might be just something again around internally or to other parts of your business. But there's no reason that when we break off into a service we can't have a nice API to work with. So why do you care? The correlation between scaling and splitting your app is very, very high. Just imagine trying to lift something, right? And it's very tightly coupled and it goes really big at Ross. You try to lift it up and try to make it bigger. But it's so heavy and it's so cumbersome because it's this huge piece. A lot of times it's valuable. We see this like who, everyone in this room probably, has multiple database machines. It may be a single application. Right? So splitting your app at that level, we also have to be thinking about splitting our app at the application level. So taking the model side of our application or the controller side of our application and finding ways to scale that separate of the rest. This is not something to really think about until it becomes a problem. And it recently did at Patch when we went from 30 counts to 850 counts in three months. So the amount of users that we had just blew up when we did that. So this is the age of innocence. This is the first thing that you do when you want to expose some kind of an API or have some kind of service, maybe even just JavaScript on your own application calling back. You just add response to your controller and you respond to different things. You respond to JSON, you respond to XML. You start doing this in a whole bunch of different places. And respond to is very innocent in the sense that every time you respond to you're coupling that controller action and the data that it's going to give back. And if it needs to change independently of your application, you can't do that. Also, your editing these response blocks kind of just where they're needed and if it's not maintainable. So the next thing you do is just the age of denial. You decide to make a folder inside the controllers called API. And you make other controllers. They do the same thing as your normal controllers. But they only are there to respond to API requests. So they only do response to it. Or maybe you decide to use something great. Then relief comes. When you decide that you're going to take the logic in your API and split it out to a separate application. At this point, things were getting too big. So you said, hell, if I just make a new application, that'll be super fast. Maybe your relief has collapsed. So if it's built in Sinatra, or it's built in Grape, I had to put that image in there because that's the coolest thing about Sinatra. That image was awesome. So how can we do this? How can we avoid duplicating logic, which is the problem with making that new application, is that if you have all of this logic on your model layer, all of this logic inside your controllers, you end up copying all that logic forward into the API layer. Then, if a business rule changes, you need to change it in two places instead of one. If you write less code, it does more, which should just be when it makes sense and when it's still readable, should be the goal that we strive for all the time. And how can we not all build the same thing? Meaning everyone in this room not all build the same thing. So I'm going to describe a couple of tools that I've been working on, and that we've been using in patch with a lot of success. So the first one is called Flexible API. The second one is called Flexible API Server. And the third one is called Constructor RB. So, first I'm going to talk about Flexible API. So you include Flexible API into a model. And by doing that, you get a method of two hash. So two hash does the same thing that you would get out of something like attributes. And it doesn't seem very useful. It's just like this, and it's really not because it's just attributes, right? So, what we realized really quickly was that there are multiple people that wanted the same data from us. They wanted it slightly different, in slightly different formats. And they wanted to make the requests in slightly different ways. So for example, we recently did an integration with the Huffington Post. And Huffington Post, on their page, they wanted to make a request and they wanted to basically say, give me the most popular story but at the same time, we're building a mobile app. It's going to be out at the end of March. And mobile apps are also using the exact same API. And when they call about a story that allowed long, they want a list of the publications around that allowed long also. But it doesn't make sense to have to give that back to all the users that are calling in from Huffington Post. So we invented this idea of request levels. So request levels basically say, what are the things at this request level that they care about? So in this simple case, we can just list a bunch of fields. These can either be methods or these can be actual fields that just calls to be the model instance. And when we call it to hash, we're able to pass in a symbol which represents the request level at which we want the hash back. So this hash doesn't have ID in it because the request level doesn't have ID in it. Request levels can also have notifications. So you can make notifications and pass a block that basically says, for this request level, I want to do something special. And I don't want to make a new method for it because it's something small. It's just a notification. You can use this to return the time along with things, or do you see the things like this? And then the hash contains the notification also. I think it's really interesting because you can also do includes. So you can say, this request level includes this association with optional scoping. So you can say, the most recent articles can be included inside of every publication that you give back to the user. The request level can have an include that gives the most recent articles. And you can actually do the include there at a certain request level. So when you call to hash with this nothing request level, you get a nested hash back which has a person inside of it. This is all kind of stupid. It's just a hash builder, right? It's just a really fancy way, just a nice DSL for building hashes of any given model. When it gets really interesting, it's here, so find hash and find all hash. And the reason that this is really interesting is that when the model hasn't been loaded yet, in the instance of the model hasn't been loaded yet, find hash will actually do a select when it goes to get the record. So it's only going to pull back the actual fields that are inside the request level. And find all hash will actually preload all the associations forward. So you get to stop worrying about whether or not you stop having to write all of these includes and you start just worrying about what's going to be returned to the user. So then there's a partner for it. It's flexible API server. And the idea behind flexible API servers that we want to be able to quickly roll out an API that's entirely usable and whether it works is like this. So flexible API server is a small center server that sits on top of a group of models that include flexible API and it gives you paths like this so you can get things, get things by ID or get things by ID with a certain relation. And these return in either JSON format or XFL format depending on your accept header by default it's JSON. And it will return at the request level that the user is associated with. So you also get for free things like limit offset and also things like if you prefer to this medic page you can use that instead. You also get count only equals true which will just return a count instead of returning the records. You also get all of these. So click things to update an object with all of the data and click rems to create a new object or to create a new object through a relation for as many as as many years. So let your mind wander and this also does other things like deletes. I didn't work on this slide because it's scary but I'll say it out loud. It also does deletes so you can do delete slash things and that'll actually perform destroy all and all the things. Also at the base of the entire thing since this is sitting on top of the database and it knows exactly what it's doing we're able to provide documentation that's built on the fly that just details what's inside of individual request levels on individual models. Also since it's in the database we can provide example URLs that actually point to resources that the user could access. So the way that we can do that we do slash things we do think.find whatever the scope is and then just do .first and use that inside of the example. So it's not pretty but better documentation. You can also put optical look-out files alongside it that describe what individual fields are for the user. So you'd say people.id is a certain thing or name is a certain thing. So then construct.rb is this way to get back a list of all of the things. This is something that you would use in the other half of your application to interact with the API for Ruby. And the way that it's typically used is like this. This is the way that we use it in our application. So instead of inheriting imagine your application being two pieces where one piece is kind of the front end back end and the other piece is the back end back end. So back end back end deals with the data that's where the API would be running and the front end back end deals with handling the controller actions and making calls back across the API. So this is a way to inherit inheriting from active record base and inherit from construct base and when you make calls the DSL mimics active record DSL. So when you make calls they're actually happening across the API boundary but they look like they're happening locally. So what we were able to do at this is a few models in our application that we were able to completely swap out with absolutely no changes to our code and our application is making calls across the API boundary using construct. The other cool thing about construct is that it's just an construct RV. So we also have construct.js and we also have construct.jective.c. So our application developers on the iPhone are writing calls in construct.jective.c. So when we add new API calls they don't need to write new code to parse the JSON or figure out how to deal with it and they can just use it. You also get things like reload so you can reload at a certain request level. So in patch what we ended up doing is we made a completely separate app that runs this and we use construct.rb to set our application and slowly are moving things to the services model using construct.rb in our application. We also use git sub-modules. The way that we use git sub-modules is we actually sub-module our existing app inside of the API and then what we do is we load the entire model and vendor layer so we have access to everything that was in our existing models and our API can use all the business logic and when something changes it automatically changes in our API we don't have to rewrite or maintain two applications at the same time. We just maintain the API layer. Currently, and this is actually in production right now our mobile developers are using this so we have an iPhone and Android platform that are both using this. We have external consumers external inside of AOL and also external to the world and also we have certain places where the JavaScript on our site since the API and the application are sharing a cookie the JavaScript on our site is actually making calls to the API instead of our application and this is all happening right now, but so a couple of issues that you might write into is something like this one is rate limiting so what we ended up doing with rate limiting is we used billware to handle the problem because it's just rat, right so basically if you make too many requests in a certain period of time you just throttle we also, I think our ops team actually added rate limiting and apache because we were not passenger why we need both authentication is another issue so the idea with authentication with flexible API is that users are tied to request levels and everything that they do is tied to what can this user see and what can they update and that's all contained inside of a request level so when a user views slash things and they are of a certain type they have the default request level that will be associated with them and it also enforces itself on posts and puts so if they try to update attributes that aren't inside of their request level they have access to so wrapping up this is just wrapping up I think that I mean obviously there's value in splitting your application but I think that finding ways finding ways to deal with the least pain that are still somewhat elegant is really valuable and I think that that's that's what we've done and I'm excited to hear any questions or feedback that anyone has on any of this thank you yeah so the other thing that we do the question was what if you want to limit access to certain types of record and not just the fields inside of the records so what we do for that is actually you can inherit flexible API server and extend it so when we go to extend it that's where we want to enforce additional logic, additional essentially controller logic like what you said we have a method called pass it allows us to say like do this thing and then pass and it will pass up into flexible API server so that's how we handle that this is just to give some background a little bit more background on flexible API and flexible API server it started actually as a separate project it started off kind of internally it started building this thing and we realized that it had value outside so what you're seeing now is I guess the result of also running empty issues like what you said and trying to find ways to fix them and there's going to be a lot more work on this and especially the authentication layer I'm also speaking at RailsConf on a similar topic so hopefully between now and then we'll get to somewhere where it's more usable Can you repeat what you said about the submodules I missed part of that one? Yeah so that pass we use we use git submodules and the idea there is that the submodule are existing application inside of the API so the idea is that when the API starts we load just the model, lib and vendor layers and all of the initializers inside of our application so we have access to all the models that are really existing and then we just go through and we define request levels on the models that exist you mean aside from submodules so it's started out really naively by just copying things over trying to get more of the API working and you quickly realize that's just completely not maintainable another thing that we did early on was we used the path to our application but that's assuming that you have your application checked out alongside and I just think submodules and it's a great use case for submodules versioning your API or something like that yeah so that's a really good point versioning is definitely important and I think that the the adverbs take great cakes with just the version command adding a namespace to the entire thing which would really work actually it might be valuable to consider instead of some option because it's closer other questions so the question is would I run multiple copies of the app to maintain multiple versions at the same time so it's a good question and the way that this the API the reason that we started doing this is I asked both the site citiesbest.com I built the entire back end the cities best not rails unfortunately but the cool thing about cities best it's not a very pretty site the cool thing about cities best is that everything on the page is driven by an API that sits alongside cities best so all of the calls that you're seeing are actually being made behind the scenes too a separate API that makes the same kind of model that we're using here and what we ended up doing there was we would we'd use routes to namespace all of the actions and then you already test the entire time so when you make a new one you initially defensive the completely different version or not but a lot of times you're just doing incremental updates in that case what you can do is point two versions kind of at the same code and then as they do a verge move them off I'm not sure if that's the best approach but data validation the server or the client and then also error propagation or any marshaling errors back up to the client and then retaining that across the platform that's a great question so the question is how am I handling error cases especially validations right so what I do for validations right now if you try to put or post something and it's not valid the validation happens inside the model layer back and back and so the API side so they're just active record validations and what you'll get if you do a post that contains invalidated you get a 422 back with an errors hash so this is the exact errors hash off of the diagram that you're trying to create any other questions are they even responsible for working on different parts of the application in that light? yeah so the question is people working on multiple parts of the application they're just confusion we're just things breaking in general so we're lucky enough as we go to we have really good test coverage on the stuff that we are writing so the API as we're building it and this is another benefit is that if you split out the model layer of your application into an API like this you can essentially unit test the API like you're testing it's like an integration test to do that we're maintaining a test coverage but we have run into issues definitely where people didn't realize that a request level already existed for a certain purpose and ended up recreating it so I think that it's hard to avoid duplication and you need to be watching out for that questions very briefly touched on how you created the API for your IOS calls did you generate that code, objective C code oh ok so the same way that there is this construct base constructor rb for Ruby we have a very similar one constructor for objective C that does the exact what? how does that exactly query the API itself so the way that this works you say set type name people, actually that line is not needed but because of all this just singularized underscore lower case so when you're making calls across this it looks like an active record but it's actually making calls completely across the API boundary and it even works for nesting things like if you were to do person.things it's going to make the call slash person slash id slash things and then it will unwrap the entire result and instantiating the thing object so it uses method missing and dynamically is generating methods as they're used on these so it's kind of introspecting on the format built by the API and it actually the API if you're using it with constructor this is kind of a detail that I glossed over like I said but returns other information about associations that you wouldn't be able to figure out so it'll say like this articles array that I'm giving you is an array of articles and like this recent underscore articles is an array of articles so that construct can pull it back and decide what kind of object to instantiate for every factor that it sees. How do you specify where it's looking for the API if you're running them on different servers or maybe even if you're running two different two different servers how would you do that? I guess currently we're construct-rb as to handle models that exist on separate APIs although that's a really good use case because you might want to fragment you might want to load only a piece of your application that does a certain thing so like if you have a gem that maybe takes up a lot of memory or is like this huge burden and it's only needed in a piece of your application you can completely separate that so we don't have an answer for that like the answer the fix that I would say seems pretty straightforward but we do have an answer for it it's telling you where the API is and it just happens inside of an initializer on the front end of your back end so the controller side of your application basically just says construct-base. there's like a I don't remember the exact name to say where the server is cool, thanks for having me and talk to you later