 Alright, so, my name is John Venter. You can find me in those places. I work at Adobe. I work full-time on jQuery Mobile. The reason that is so enormous is because I'm really grateful for the fact that I get paid money to work in open source. So Adobe is taking kind of a big stance on doing this and I'm one of that factors. I was prior to that, a rubious full-time. So I'm not a total sell-out. Okay, so what are we really addressing here today? So what is this talk actually about? This talk is about really big classes. So I think everybody is familiar with, or just quick show of hands. How many of you in here are Rails developers? How many of you have seen the gigantic user class in a Rails application? Yeah, right. So this is a pretty common thing. These gigantic classes are pages and pages and pages and pages are just my methods. So I kind of want to address that a little bit today with this as a technique. There's a couple existing solutions out there for large, really large classes and so one of them is, I like Pub, makes it into all the things. There's a couple, so you don't need to be able to read this, but if you guys have ever looked in terms of off-logic, there's a session, and to be clear, all the code examples I use for like public projects and people's projects, I'm not here to bag on people. I'm just here to use these examples. But if you look in off-logic, the session base class is just basically a big include. So everything is in other modules and about doing this gets you is you don't have to scroll down, like things are grouped logically. Other than that, it's just moving the code around. Okay, and then the other one is object composition. So this is sort of like when you think about refactoring your code base, a lot of times it involves some form of object composition. And just as a quick review, what is object composition? So if I have a lamp and I'm dependent on finding my needs maintenance method is dependent on finding out the bulb age. This is a very simple example, right? So you're not necessarily going to have to worry about refactoring something that's headlines. But as an example, this works. So the bulb age is sort of contained in the lamp class. So you just switch this out by injecting the bulb in and then asking the bulb itself. So delegate. And that's composition. You're adding, you're using your hazard relationship to move code into another place where it makes sense. Okay, so super simple. This is the most basic form of object composition. There's a lot of different types and ways to do this. And as part of that we'll discuss today, this rack middleware as an abstraction is sort of just another form of object composition. Okay, so Shane counseled me to do this as a survey. So how many of you use middleware when you say middleware if you say it at all in the singular? So when you say middleware, you talk about one thing, okay, there's three, three speakers over here and some people on the back, right? How many of you use it in the plural, middleware? Okay, you have to raise your hand for something. This is not an object. Okay, so most of you use plural. Okay, so how many of you use it interchangeably if that bothers you? I'm really sorry. I might occasionally use middle things as a replacement for you. Okay, so let's do a quick review. So it's for obviously to understand the middleware thing as a general purpose abstraction, we obviously have to look at what it looks like when it's used and it's kind of made an environment. So let's take a look at a quick review of rack middleware. How many of you in here know how rack middleware works? Okay, so that's actually pretty good. So when I gave this presentation like a month ago, it wasn't quite that good. But this will be a good review for those of you who know, and you have to cover it anyway if you don't. Okay, so when I say middleware, middleware means a lot of different things on the top part, right? So when I say middleware, I really mean middleware as it's envisioned in PEP333. And if you've never read this, it's actually kind of an interesting read because the whole purpose is more just to bridge the gap between servers and frameworks, but they sort of threw this in here as like the added incentive. You know, it's like the extra set of knives you get when you're buying, you know, oxy clean on the TV or whatever. The middleware was just an incentive for people to get excited about using this bridge between the server and the framework. And so anyway, it's a good read. You should definitely check it out. It's a little nice little bit of background. It's not very long. Okay, so in middleware, they're very concerned with request processing. So the request as it comes in and the response as it goes back out, it gives them the opportunity to go after them. So this is what a middleware stack looks like. Now specifically this sits between the server and your framework. So in Rails, it's between whatever server you're using and Rails. There's stuff that happens there. Rails at the refactor, the big refactor for 3.0, moved a lot of stuff into this middleware space because it really makes sense because it's just request processing. It doesn't really belong in the framework level. So they kind of pulled it out and made it available. So this is what a stack looks like. And the stack operates top down. So request comes in at the top. The very first middleware there at the top gets the opportunity to operate on the request first and then it proceeds down the stack. So let's take a look at just an example of middlewares. So first things first, before it ever receives a request, it gets initialized. And when it gets initialized, it gets the next middleware in the stack as the first argument to its constraint. So it is aware of the next middleware. And that's extremely important because it is the responsibility of each middleware to call the next middleware. And this is the thing that defines the difference between a middleware and an endpoint. The endpoint does nothing further. It returns the response. So it would never make a call to it. So in that case, the app, the thing passed into the constructor, is the common model. So when you run the server, again, the request comes in, goes top to bottom. First it'll hit the, our example middleware. The call method is invoked on that middleware. It gets a hash. So the hash, the environment hash, and it's funny because they call it environment because this is a big, fat, stink bag. It represents the request. It's got headers. It's got everything that you need in the request to deal with a web request, right? So it needs to be requested. And then you can manipulate it or you cannot manipulate it. But one thing that you have to do if you're being good at this is you have to pass it on to the next middleware. So you can do stuff before, and you'll know here that you can do something after too, right? So you have the opportunity afterwards to do whatever you want with the response. So it's very general, but you have a lot of responsibility. You have to do the right thing with the request. So as I said earlier, endpoints. The differentiation here with vagrants middleware, which I'll discuss in a minute, and rack middleware is that vagrants middleware sort of throws in a lambda at the end so you just call something so you just as a middleware keep calling the next one and don't worry about it. It'll sort of handle it for you. Whereas the rack, you have to check to make sure that there is a middleware if you're going to be both an endpoint and a middleware. If you're going to be an endpoint, just get the request and return response and call anything of this. You're never going to get another app to call. You're never going to get the next middleware. So that's the difference between an endpoint and a middleware. Okay, so at this point I just want to make sure that's all clear because this is obviously very foundational for the rest of the talk. Anybody have any questions about how this works? And I realize in our conferences it's sort of not cool to raise your hand if you're not sure what's going on upstairs, but that's fine. If you have a question before you ask it now so you can understand the rest of the comment. Anybody? Cool. I will just assume that you're all in deep understanding of this. All right, so let's talk about Vagrant. If you're interested in Vagrant after this talk, during the talk, you know, definitely check it out. I co-created Vagrant with Mitch. I'm a first-world contributor to it, though I do not contribute at all anymore. I contribute like a year or less. Anyway, how many of you have used Vagrant? No Vagrant is. All right, that's great. Okay, ops folks, I'm sure. Anyway, the internals of Vagrant we're going to be discussing today will touch a little bit off what it does, but not really. Anyway, definitely check it out. But in general, if you've never heard of Vagrant, it works on virtual machines. So it lets you deal with virtual machines locally. It kind of streamlines the whole process of using them for development. So there's a lot of stuff that goes on when you want to use a virtual machine locally that takes a lot of time. You've got a slot operating system. You have the core forwarding. You've got the folder sharing, all that stuff. That's a lot of clicking around. This automates a lot of it. So there's a lot of operating on virtual machines as I've done. So let's just take a look at one command. So Vagrant, when you have a virtual machine that I'm working with with Vagrant, and I want to suspend the virtual machine. It's a relatively simple operation. So this is what, generally, this is what the middleware stack looks like for suspending a virtual machine. So these are the operations that need to take place for a virtual machine to be suspended. And this is the way that Vagrant does it. It builds a stack of operations, and the virtual machine or the environment containing the virtual machine will get trickled down through those, and some will happen. So let's take a look at it directly. As I said, top to bottom, virtual machine will go down inside an environment, although it's easy enough to think of it as a virtual machine. So let's take a look at validate first. So real, real simple. So when we run the command on invocation, it checks to see if your config is valid. So if Vagrant does configuration, you build virtual machines through a Ruby DSL, through configuration, and it has to validate them, right? If you're doing something wrong in the Ruby config while obviously you can't, it doesn't want to deal with your VM because it might screw something up, so it yells at you and says, you're doing your wrong. And then it calls the next middleware, which in this case is checked accessible. So this is in, your VMs can get into a state in virtual locks where you can't deal with them or monkey with them. So this just checks to make sure that's not the case before it goes off and tries to monkey with them. It calls the next environment, which is spend, so this is where the magic happens obviously. This is really simple, so it just asks the driver. So this could be one of two things, depending on which version you're using. This could be the API to see API the virtual box provides, or it could be actually an exact of the Vbox managed command. Either way, what ends up happening here is your VM gets suspended. And that's important as I said earlier. There isn't actually another middleware here to be called, right? But Vagrant sort of handles that for you. Let's say you're creating a plugin and you just don't have to think about it, right? You just be a good citizen and call the next one. Because there are really no endpoints here. There's nothing to return back out of the stack necessarily. We're just operating on the VM. So that's an important difference between this and that. Okay, so Vagrant up. So this is the big one. This is, I have nothing, and I'm going to create a VM from scratch. And obviously there's a lot more operations than there aren't to spend. Particularly just thinking, you don't have to read this. These are the same two operations that were at the top of the suspend stack. So we're getting a lot of reuse here. Because there are similar operations that have to take place in almost all the commands. When you build them this way, when you compose these operations together this way, it's easier to reuse them. Generally speaking though, if you were building this from scratch, and you weren't using the subtraction, there may be something else you could use. But your VM, you can see VM all the way over there all the way down the bottom, right? If you were to show all this in your VM class, it would be gigantic, right? The VM class would be enormous. And that was the original thing as we were working on it that we came to. We realized really early on we were a Rails developer before we started working on this. We didn't like happy, gigantic classes where it was hard to figure out what your method dependencies were. You knew the balance around. It wasn't easy to test because of all these weird, you had to set up the object. So we knew very early on we had kind of an intermediate step between bloated VM class and middleware. There was this kind of action thing. You can go through the VM history or the VM history and check that out. That's something we wanted to avoid. Okay, is it clear at this point how vagrant uses middleware to operate on VMs? Is that clear to everybody? Okay, great. All right, I'm not completely failing at my job here. Okay, so I want to refactor something that's out there now as a public piece of software. I chose the diaspora user because honestly, I got it as the first Rails application I could think of that was open source. I'm sure there's other ones, but it's the first thing to pop online. Actually, their user class isn't gigantic. It's big, but it's no bigger than any other Rails user class. It's pretty good. Let's take a look. Just to get a sense for how big it actually is. I don't know how long this will take. It goes on for a couple more pages. It's a really, really big class. Ideally, what we do here is we want to find some methods that are dependent on each other, some group methods, some things that fit together so that we can pull those out into an object and then compose it using this kind of middleware idea. Okay, so let's take a method and turn it into middleware. First thing I noticed that it doesn't have a lot of dependencies, so it kind of makes sense for this. They have an accept invitation method. I am not paying attention at all to what these things do. I'm just moving them out into a class. I'm assuming that this accepts some sort of invitation based domain. I think it does. If you want to take a look at it yourself and get back to me, okay, so this is the method as it is. I don't care what it's doing. You don't need to worry about reading it. Just know that it's large. That's enough to know that it should be changed, I guess. It's really big. What I want to do is I want to move this out into a class. Remember, we have the call method convention. We're going to stick this into a class that has a method call. This will be the method in which things are done when this middleware is used. Let's move that over. The one thing that you need to take note of is that there's just that one dependency. This method is actually pretty big enough. It kind of contains everything. It only has the setup method dependency and that's another method of user calls. So we move this over. There's a couple of things that need to happen. I'll explain this to you. We need the user, right? We need the thing to operate on. There's two kind of ways you can do this. You can say, I want just a hash that will have this stuff to operate on and I'll worry about getting it out of the hash. Or you could say it's the user and then maybe another argument. You can do it any way you want. It doesn't really matter to me. This is just sort of easy to illustrate if you can mention having just a state bag. Up there, there's a user in the state bag that we're going to get out and operate on. Previously the method just referenced self everywhere. It's itself not something and it's itself. So now we're just going to change that just to be user everywhere. This isn't like a painful refactoring. We've got a pretty quick step over to doing this. At the bottom, you have the call to the next app necessarily. I think down there it's missing environment. That's about all you had to do is add those two things. Add some extraction of the user object to operate on for the environment. In this case, it has some options that need to be altered and then down at the bottom, putting in the call to the next middleware should exist. Then the setup method has to do the same thing. Specifically, when I got that user out of the environment, I set it to an input variable and then just attribute accessible or whatever. User make the alterations in the setup method which is called previously in the call method and then this sort of handle. Wrap these two things into a class and it's up. This is the controller where this happens just for reference. It gets user finds it, the user buys invitations over and then accepts the invitation. In this case, that should actually be user over here when we change it, but we'll see in a second. It accepts the invitation. What I've done now to proxy this over is I said let's just leave the accept invitation, everything else the same everywhere, and just do what we would normally do, just use the class inside of the user. Take our middleware class of users. I'm delegating this out to another class. The first option is to instantiate the middleware and then just do a call with the options. Super simple. The other one is if you want to spend a little more time, you can put together some infrastructure where you can have a registry of these middleware to use and just have them after the symbols or something and then just run them off like that so it's a little easier to repave. I'm not going to illustrate what this machinery would look like. You can look at rack, or you can look at vagrant as a reference. This has a little bit of an interaction because Mitch wanted to make users to be able to work with, but it's still followable. But this stuff can be really simple. This isn't super hard. Okay, so what if we'd buy ourselves here other than just moving code around like I said earlier? Okay, a couple of things. This is my awesome testing code. Hit mock stuff, you probably shouldn't mock, but the point is that it's really easy to test now. It's its own contained object, right? Your tests can be kind of simple. You can mock things in. There's a sense that when you look at something and you can fit it all on one screen, and you're aware of what all the method dependencies are because it's all fitting within that class, you don't have to worry about breaking something when you decide to refactor. It's all sitting there right in front of you. You don't have to go searching through the gigantic class because of a method. Also, and I'm not trying to advocate prefactoring this type of stuff, but in the future, should you decide to have kind of a set of steps that people have to go through when they accept an invite or something with that effect? So, for example, let's say that when you sign in through the web interface, you're sort of marked as a new, for whatever reason, and then they have some sort of API and you can hit the curl that would make you a veteran of some sort. So you want to, both of these times, you want to accept the invite, but maybe you want to do something extra in those cases so you have your new stack and your veteran stack. Okay, so let's wrap this up. What are some signs that this is a good abstraction for you using your code? So obviously we're used. We just talked about this. When you need to have the same functionality running all over the place, and you're tempted to use a mix-in, think about using object composition. I mean, just use object composition if you're not sure about the middleware thing. Don't avoid using mix-ins all the time, but this works really well. And if you wanted to get a little more complex as a consequence, now you can kind of compose these things together as a whole action. So serial methodification. So a lot of times you'll see methods that are just wrappers for a bunch of other methods. So run this thing, or run this process, where you just call out to a bunch of other methods. You may not even have arguments. So you can kind of map that over to middleware. And this sort of assumes that you have reason to extract these out into a middleware of the software. So var as a backer, ostensibly large methods, or have a lot of dependencies, you can do both ways. Alternately, because the one on the right would mean you're kind of disdaging your stack every time, you store your stack somewhere so that it's available, and you just need to call a class method or something like that. Obviously, I'm aware that that is not enclosurable. So another one is function composition. So when you're composing a bunch of methods where you're just sort of firing off stuff along the way, taking some arguments and then passing the result back and forth, you can do the same thing here by just, again, imposing the methods as middleware. And that is it.