 Betsy Hebel, and we are here today to talk about, let's start off by getting some consensus about what metaprogramming even is. There are a lot of different definitions that people use. When I was explaining this talk to my parents, I used one of the more common ones, which is code that writes code. I think this definition is both inaccurate and also a little overbroad, almost the point of not just meaninglessness, but misleadingness. It can apply to C generators, it can apply to quines, it can apply to a lot of things, the output strings, and also it makes your parents think you're building Skynet. Then there's the definition of metaprogramming that I think a lot of people instinctively use when they're intuitively thinking about it, which is stuff that's magical and hard to grip for. This also can describe Perl, so again we reject this definition. So I think the most useful definition you can have for metaprogramming is code that treats the structure of the program as just another data structure to manipulate using code. And what does this mean in the wild? Let's start with the send method, which is a pretty basic building block of Ruby, most frequently used in the wild to invoke private methods. Ruby's privacy model being more of a suggestion than any real binding thing. But under the hood what it is, is it's taking Ruby's message passing model of method invocation and making it explicit. Ordinarily this is hidden behind syntactic sugar. The dots send the method as well as any arguments you include with the method invocation to the object you're sending the message to. But using object hash send makes this a little more explicit. A dot bar and A dot send bar are semantically equivalent, but when you use send you're making it really clear that what you're actually passing the argument isn't a method name and also some arguments. It's a bag of arguments, the first of which just happens to be the method name and which just happens to have also the requirement that it must be a symbol or a string. This first argument does have some special status. It directly determines what block of code is called next, almost as if an object is just a giant case statement saying, hey, do this, if that. But ultimately all of the arguments in the bag of arguments are just ways of passing information to an object about what you want it to do next. So moving on, let's talk about how it can be used and misused in the wild. When I was a wee baby coder, literally on Friday, the first week of my first Ruby job, I discovered object hash send and what it could do. I'd been working with delayed job all day to do asynchronous transactional email. For those of you not familiar with the library, its core is the method send later, which looks like this and which does pretty much what it looks like. It queues a job that invokes the sent method name along with any arguments also passed. I was looking at it in the context of a method that looked a little like this, a pretty straightforward Rails observer that looked for attribute changes and triggered emails or triggered queuing the email for sending if it was appropriate. I say something like this because actually the method looks a little more like this. I really didn't like this code. It got the job done, but I didn't like how it got it done. I had trained in C in Java and Perl, so I was used to the idea that method names were these inviolable things that were entirely separate from the data that the methods were acting upon. When you're coming from that mindset, you accept a certain amount of repetition and boilerplate as inevitable. You write 30 getter and setter methods for classes attributes, each of them one line long and as formulaic as a Garfield strip. I resented the hell out of this, but I didn't know how to improve on it until I looked at the send later method and realized that probably if there was a send later method it was playing off of the name of something called send. I started Googling. About 20 or 30 minutes later, the code looked a little more like this. I identified a pattern that had been expressed by repetition and expressed that pattern with code instead, namely by invoking methods dynamically using send rather than statically with copy and paste. This is just another view of the same code. It felt like this. If you have issues with animated GIFs, close your eyes for a second. Actually, it felt a lot more like this. You can open your eyes again if you close them. This is how it's supposed to work. I came from this background where I couldn't do a thing and then I learned more Ruby and I could. It was awesome. Also, I did something cool at 4.30 on the Friday of my first ever Ruby job. It was a contract job because I had no experience and they were all prove yourself. I felt really great because I had just done lots of things for my job security. I'd also learned the wrong lesson. The next code I'm about to show you is an amalgam of code that I wrote back then and code I've seen other people write. All names have been changed to protect the guilty. We've got some code that searches within a set of models and then sends their data onto the formatter. Like I was saying earlier, a lot of the time when you're using dynamic method calls, you're using them to express a repetition that had previously been expressed using copy and paste with some variable names swapped out. This is definitely what's going on here. Reflexively reaching for our dynamic method calls to fix this problem has three traps, though. The first is that it breaks crap. Ordinarily, when I'm slunking through new code, I search for method names to see where to find and invoked. But I can't find, like, but right now, I can't find where format whatever results for some user type is defined, because I don't know what method name is really being sent. Similarly, if I'm looking at the definition for format pants results for anonymous users, I'm not going to see that it's called here, so I'm not going to get an accurate idea of what context the method is being used in. The second issue is that dynamic method calls like this introduce cyclomatic complexity. Cyclomatic complexity is approximately the number of paths that data can take for your code, and high cyclomatic complexity is a bad thing because it means you need to keep track of more things to figure out what your code's going to be doing. Conditional statements like if and case increase it. And if you're doing a dynamic method invocation, since there's a limited number of methods that you actually can be invoking, you're in effect introducing this gigantic case statement into your code. And shoving this implicit conditional under the rug by using a dynamic method invocation doesn't actually make the cyclomatic complexity you're introducing go away. It just hides it. The third thing is that using a dynamic method call to reduce repetition at the call site usually actually speckles over some far uglier repetition that's going on elsewhere. And that is also what's happened here, so let's fix it. And just as a quick warning, this is a refactoring in which things are going to get a little bit uglier before they get prettier, so bear with me. The first thing we do is change the method signature up a little. When you do a dynamic method call, you cheat message passing a little by sending multiple pieces of data with the method name argument. Here these multiple pieces of data are the abstract format stuff for stuff method name, the garment and the user type. And so what we can do is we can instead change to using a method that makes the fact that garment and user type are arguments explicit by promoting them to first class arguments. The quickest way to do this is to just switch it up in the call site and move the send call into formatter, which reduces the number of lines of code we need to change at once. This seems like a trivially factoring, but it's in fact not because in moving the call site of the dynamic method call into the class, we're moving it to where, so that it's paired with its context, which means that it's easier to understand it in its context. The second thing we do is we recognize the repetition of results argument because we have all of these method calls that are all being passed results and remove it by turning formatter into a real class rather than a bag of class methods. And then next we look for more repetition. There's some pretty low-hanging fruit here. Both of the format sweater results for whatever methods share a lot of data returned. So we move that out into a sweater attributes method, and then we move that to sweater, which is where it actually goes. This makes the kind of ridiculous indirection that we have here a little more obvious than it was. We're asking the sweater class to return the data we want out of the sweater object and then retrieving the data from it instead of just asking the sweater object to return the data in the first place. So we switch to doing that, and we also do that on hat for good measure. And then, of course, we update the formatter to reflect this. And so the formatter class looks like this now. It's much smaller and much prettier. And since it's much smaller, we can also see other repetitions that we wanted to see before, which is the nice thing about incremental re-factoring. You always see something new. Now, for example, both the format whatever results for user methods are identical. So again, we can see that using a dynamic method call up the chain was hiding some repetition. And so it wasn't really necessary. So we can collapse the two methods to just format for user and format for admin and get rid of that garment argument as completely superfluous. And again, we couldn't see before that it was superfluous, but it is. If we want, we can also get rid of that last dynamic method call to user type by instead moving to a polymorphic approach for formatters. Here, we're using const get to pick out the right formatter. And if you really hate const get, or you really hate dynamism, you can also get the equivalent effect by using a hash to store explicit class names. For the most part, honestly, I think that's overkill. And we're not really fully getting away from dynamism here in either the const get version or even the hash version. We're just choosing which class to instantiate dynamically. But now the dynamism is encapsulated within a single file, and as more importantly, within a single family of classes. What this means is that its effect on the cyclomatically complexity of the app is reduced to within that family of classes rather than spread out throughout the whole apps. You need to think about it all the time instead of just when you're dealing with this small piece of code. Similarly, its effect on searchability has been greatly reduced because you don't need to search for these methods except when you're in the same file and then you can just see them. So you're isolating the negative effects of the dynamism but getting to keep the benefits of dynamism. And when I'm talking about when you shouldn't met a program in this talk, I don't want to lose track of the fact that Ruby's capacity for dynamism is one of the best things about the language. I mean, look at how concise and how pretty this nice little concept call is. You need to write a lot more lines to do that same thing in less dynamic language, which is why I had that gigantic case tree for the mailer code earlier. And that just... It's not only faster, it's not only easier to understand, it's also just more joyful, and I don't want to undersell that. Anyway, next up is method missing, and we're going to get a little less joyful. So let's go back and look at these methods again. These, like most of the rest of ActiveRecord, are defined using method missing, which allows us to dynamically respond to arbitrary method names. After all, again, the method name is just a part, like any other of the bad arguments that we send to an object. In order to explain how method missing works, we need to examine Ruby's default lookup chain. When you send a bag of arguments to an object, Ruby first looks at the object itself, or rather, technically, the object's eigenclass to see if any singleton methods have been defined that match the sent method name. Next, it looks at the object's class, and then at the object's inheritance chain at all the modules that have been inserted into this lookup chain, including extend at the object's superclass, and so on and so forth, and it superclasses superclass until we get all the way back up to object. It may find nothing, which is where method missing comes in. Method missing is a special method that allows you to describe fallback handling in the case that no method matching the name of the sent method is found in this lookup chain. And the lookup change or method missing is parallel to the initial lookup chain. So after that, it goes and it looks up the method on the object's class in method missing, and then all the way at the inheritance chain, again, all the way up to object. It finds nothing, it gives up and spits out no method error, and that slide is missing. Good job, me. So in the wild, method missing looks a little like this. This is an oversimplification of how AFTF record uses it, but this is the general shape. Method missing gets passed the bag of arguments that you sent to the object and decides what to do based on them. This oversimplified version of it that I'm showing here resembles many beginner method missings, including my own, which is a huge problem because it's making three common mistakes. The first is that it's not including a call to super. If you don't include a fallback to super somewhere within each method missing definition, the lookup chain will stop within the first method missing it finds. It will potentially bypass something useful further up the chain. The second is that there's no parallel definition of respond to missing. Right now, if you send this method name to respond to, it will say false because no method's been defined by its name on the object. Restun to missing, which looks like this, allows you to expand respond to's conception of what method names an object responds to. It's necessary whenever you're using method missing because otherwise you're potentially lying to anyone who uses your object about what it can do. And so now you see it doesn't lie. Much better. The third mistake is that you're not defining the method in the method missing call. Method missing based dynamic method lookup is really slow if you use the method multiple times because you need to go all the way through the lookup chain each time. If you're going to using a method defined this way frequently, it's good practice to in effect cache the method or memoize it by explicitly define it using defined method within method missing. Now for a subtler mistake you can make in and around method missing. This is one I've made within the past year. Two of my colleagues from my last job are in the audience right now and I should really apologize to them. This is definitely something that I did. So let's say that you're dealing, you have a mix in that relies on the breed code methods being defined on an object. Like we're running a kennel, we have a bunch of different species of cats, or breeds of cats. And we have two objects, one of which explicitly defines breed codes and the other of which implicitly defines it using active records method missing based strategy. If you call breed codes on a cat right now, what happens? Well, breed codes is defined all the way up in the superclasses method missing. So first the method is looked up on the eigenclass and then on the class and then it finds a definition. The definition is not implemented error. This explicit definition is before the implicit definition in the method lookup chain and so it takes precedence. And so how do we get around this? This is what I learned Zillion. Please don't do this. It means that poor has breeds or whatever will unfairly become a part of every single debugging session when you typo a method name. Its method missing call will be an attractively late part of your back trace. If you do this instead, it'll work. That's because super doesn't really care whether something is defined further up the method lookup chain. It just defers it one up so that you can get to the implicit definition perfectly fine. You can also do this which is a little more explicit and which also stays within the mixin. Or you can get rid of the debugging convenience of the not implemented error because since we're going to all of this work and boilerplate to make it continue to be convenient then it's arguable as to whether it's even convenient anymore. However, the fact that all of these are even questions illuminates some of the hidden costs of a method missing driven dynamic interface. It does not play nicely with strategies for good object oriented design in Ruby. Defaulting to it when you want easy dynamism makes your code far less extensible later. And so you should really consider how much you really need that or, yeah. Or rather you don't. So what is method missing good for? These are the traditional uses that people give. Pretty dynamic DSLs like active records, dynamic finder methods, global config objects like amber bit or delegator objects. I think that none of these is actually good use. Pretty dynamic DSLs are usually better expressed with options hashes than by including arguments within the method name. You can use open struct for extensible config objects. You can use Ruby's built-in delegator classes for delegation. Both open struct and Ruby's built-in delegator classes do use method missing under the hood. But it's hidden under a nice well maintained and well understood interface. You cannot guarantee that that's going to be true if you roll your own. In fact, usually it's not. So given this, how then are we to do dynamic method definition? Because sometimes it really is necessary. To find method is a little bit slow. So if you're doing something where performance at load is really important, then I would consider just sucking it up and copying and pasting. And also, like dynamic method calls with send, dynamic method definition with defined method can be used to hide bad class design. But since it's both a lot more flexible and a lot more explicit than method missing, I like it a lot better. Its syntax is really straightforward. You just pass in a method name and a block defining the method. So let's examine how it can be used to clean up code. We have here some nice repetitive method definitions. Defined method lets us pull it out into an each block. This cuts lines and more importantly illuminates the deeper structure of the code. Because sometimes using dynamism to pull out repetition makes the fact that repetition is going on more explicit and that can be important. So later on in this theoretical project, we decide to isolate the attachment definition code in its own module for later reuse. Despite the fact that most good photos are of cats. Despite the fact that you cannot actually see my neighbor's cat in this photo. Good job of field.js. It's hiding in the bag. It's hiding in the bag off the screen ever so lightly. It's hiding very well in the bag. This code is not awful, but we can take it a bit further. Right now it's not very extensible. If someone using this module wants to override our definition of photo URL, their redefinition will be on the same class that our definition is on. Because defined method is called by a macro within the context of the including class. If they want to include our definition within their extension, therefore, they're either going to need to... They're not going to have access to super to do that with. Instead, they're going to need to copy and paste our definition in, which is going to ugly and brittle and screw them over when they upgrade. Or they're going to need to resort to an ugly workaround like Rails' alias method chain. There's a pretty easy way to fix this, though, which is just dynamic module inclusion. That... Where are my slides? That's fascinating. I'm sorry. Hold on. Having issues with GitHub pages earlier, they did appear not to have entirely resolved. Let me just type this out. Or better yet, let me just... So what we can do is we can quickly define these instance methods on a dynamically created module using module.new. This module can either be anonymous, which is not what I'm doing here, is what I was doing on the original slide, in which case what you do is you include the variable that you've stashed the dynamically created module in. Or if you really hate the idea of anonymous anything, including anonymous modules, you can throw it into a constant with constant set really quickly and then include that. And what this does is it stashes the module in between the object and the rest of the lookup chain, which gives you a nice way to use super and get access to the definition if you need to extend the method later. Next up, Ruby hook methods. There are two classes of hook methods, and these are not formal classes so much as they are just experiential classes of hook methods in Ruby. They're the normal ones and they're the weird ones. The normal ones are the ones you're going to see every day. They are inherited, included, and extended. They are invoked whenever a class inherits, includes, or extends a class or module. The weird ones include but are not limited to method added, method removed, singleton added method, or method added, singleton method removed. There's also method undefined if you want to get really fringe. I really only have one thing to say about hook methods, both the normal ones and the weird ones. Don't. And what I'm saying about self-dot included, I realize is probably a little controversial. We've all seen this pattern before in the wild. It's really, really common. It's also, I think, pretty awful. And I think that for... I think that because it's a way of pretending that Ruby has multiple inheritance. What we're doing right here is we're taking something that wants to be a class and compressing it into a module so that we can kind of sort of inherit from it using Ruby's mix-in feature. And so we're just kind of contorting the language into all of these little bits in order to take something that clearly wants to be an object because it has both class and instance methods and make it not an object anymore. And so what we should really actually be doing in these cases is we should be taking this object that wants to be an object, making it a first class object, and instead attaching it to a calling class with a simple class-level macro. This is what carrier wave's doing. The photo example I was using earlier is stolen very directly from the carrier wave internals and is also repeated from the direct-and-fly internals. I'm not sure about paperclip. Because it's a really good paradigm for taking a bag of functionality that you want to have easy access to within an object, but also separating out the actual dealings into their own objects so you don't break the single responsibility principle and you don't add unnecessarily to an object's surface area. So yeah. Anyway, let's wrap up. What I want you to take away from this is mostly that it's all data to Ruby. So it should be to you too. But while you're carrying on in your bright metaprogramming future, please remember your maintenance coder because your maintenance coder is going to be you in about four months. And consider whether you can search for the code, whether you can extend the code, and whether you can debug the code you will need to do, at the very least, the last one. Also, remember whether you're using dynamism for the right purposes or whether you could just use the language structure of Ruby to express the same thing as nicely. This is me. Again, I'm Betsy Havel. I can be found on the internet in several places. I work for a fine DC-based e-commerce company called Aptoro. Brief moment of commerce. They paid me to come here. We're great to work for. We're hiring junior developers. We're hiring many co-workers, some of whom are also speaking. If you are not a junior developer, don't worry. We can supply all your cheap laptop needs at Blink.com. Also, please consider checking out my colleagues' talks tomorrow. Chris Hoffman is presenting on mathematical coercion in Ruby. Brock Wilcox is doing weird stuff with time traveling debugging. And Josh Schmeida is translating algorithms in Haskell and Ruby and back again. My slides are technically already up on GitHub pages at the URL below, but as you can see, they have some image issues. So I'll be correcting that after this talk. All cat pictures, including the ones you can't entirely see, are credit in my neighbor, Nikki Murray. Does anyone have any questions?