 Okay Hi everyone Thanks very much for choosing to give me your attention for half an hour of your finite lives I'm the good Tom Stewart. I'm gonna tell you about Monads and how we can sort of uncover them by refactoring our Ruby code So the ideas in this talk might be unfamiliar to some of you Before we get stuck into the details. I want to just warm our brains up with something simple before I get into the difficult bit People always use analogies to explain Monads It doesn't help so there'll be no burritos or spacesuits or elephants in this talk But I do want to talk about some related ideas not analogies But just related ideas that will put your brain in a receptive state for the stuff. I want to explain to you I'll start with a rhetorical question. What is a stack? Well It's a kind of value with certain operations and Those operations are called push pop top and empty. I'm talking about an immutable stack here So push and pop don't mutate the stack. They just return a new one You have to use top if you want to look at the top element We also need a class method called empty which creates an empty stack to get us started For a value to qualify as a stack its operations have to follow certain rules For stacks the rules just say that the operations do what you intuitively expect when you push a value onto a stack That becomes the top value Pushing then popping is a no-op an empty stack is empty and a stack with something pushed on to it isn't empty We can implement the stack operations however we like Here's a class called a raise stack that implements the operations in a particular way It stores the stack contents as an array of values push adds a value onto the front of the array and pop removes the arrays first item This class as linked list stack implements the same operations in a different way It stores the top element and a pointer to the rest of the stack So top and pop are just attributes and push stores its value in a new stack instance pointing at the old one We can use the stack operations without knowing how they're implemented if we make an empty array stack and Push two values onto it and then pop one of them and ask for the top value We get the result that we expect if we do the same thing with a linked list stack It works in exactly the same way We can define more operations in terms of the old ones So for example, here's a method called size that recursively calculates the size of a stack by counting How many times it has to be popped until it's empty? So as long as the stacks got working implementations of empty and pop we can implement this size method on top of them It doesn't matter what stack implementation we use size always works the same way an empty array stack has size 0 if We push two values on its size is 2 an empty linked list stack has size 0 And if we push two values on its size is 2 So what do we mean by stack? It's really a specification an Implementation of a stack provides certain operations that follow certain rules There are two benefits of that firstly those operations provide a sort of common interface across many possible implementations and Secondly they allow us to build shared functionality like that size method that works with any implementation Here's another question. What's the collection or at least in Ruby? What's the collection? Well, it's a kind of value with certain operations actually just one operation in Ruby called each and That operation follows certain rules actually just one rule in Ruby Which is that each calls a block with a value zero or more times in immediate sequence? That's it Now we can implement that operation however we like Here's a hard coded collection whose each method literally calls a block with one and then with two all the way up to five Here's a generated collection whose each method calculates those values dynamically in a loop and calls a block with each one We can use that each operation without knowing its implementation from the outside hard coded collection and Generated collection both behave like a collection of the numbers from one to five We can define more collection operations on top of the one operation. We already have for example Here's a method called select that takes a block and then calls each and accumulates all the values that make the block return true So as long as a collection has a working implementation of each we can implement select on top of it It works the same for both implementations Of course in Ruby we have a module called innumerable that already has select and count and map and inject and Tons of other helpful stuff that all sits on top of this one each method So what do we mean by collection? Again, it's really just a specification an implementation of a collection provides an operation that follows a rule and Again, we get two benefits that each method gives us a common interface across many possible Implementations of collections and it allows us to build shared functionality like select that works with any implementation So what name do we give these things are they design patterns or interfaces or apis or duck types? All of those words are appropriate And they all sort of overlap to an extent and the concept of a stack or a collection kind of sits in the middle of all of them But in my opinion the most specific and therefore the most accurate term is abstract data type That literally means a kind of value which has certain operations that follow certain rules Stacks and collections are abstract concepts then but they're abstract in a good way The abstractions give us power and we expect programmers to understand them Nobody talks about stacks in hush tones. They're simple and they're useful Okay, so that's enough priming of your brain. Let's do some refactoring First I'd like to look at some code that has to deal with nils Imagine we have a project management app with different kinds of models Each project has a person who created it each person has an address each address has a country Each country has a capital city and each city has weather information which for the sake of simplicity Let's just assume is a string So let's say that in our user interface for this application We want to display the weather next to each project for some reason That involves traversing all of those associations So here's a method that does that maybe this is the kind of thing you'd write in a rails view helper There are lots of reasons not to write code like this But there are also good reasons to do it and anyway people will always write code like this no matter what we say If we make a city which has sunny weather and a country which has that city as its capital and an address in that country and a Person with that address and a project created by that person We can pass that project into weather for and it works fine But if we make a bad project for example a project with an address that has no country then weather for blows up So Tony Hoar invented nills in 1965 and he does now call it his billion dollar mistake Which he says has probably caused a billion dollars of pain and damage and this is exactly the sort of thing. He's talking about So they may be a mistake, but Ruby has nills so we're stuck with them To make our weather for method tolerate nills. We're gonna have to explicitly check for them So we need to introduce local variables to hold every intermediate result and then check each intermediate result Before we call a method on it While we're at it, we might as well include the possibility that the project itself is now Now this is turning into a bit of a pyramid of doom You can see the code kind of drifting over to the right, but luckily this code looks the same if we just flatten it So this code works, but it's pretty clumsy and it's hard to remember to do something like this every time We might possibly have to deal deal with nil Fortunately rails has got a solution to this problem So rails actually active support Monkey patches object and nil class with a method called try Which delegates to a public send if the objects not nil and just returns nil if it is So when every object in the system has a try method instead of doing all of these nil checks ourselves We can let try do it for us and now we're back to just chaining method calls together So we can take the local variables out again like that. So I'll just make that a bit bigger So this is as good as it gets right now It's better than the version with unless nil all over the place anyway, but can we do any better? Well Monkey patching definitely has its place But monkey patching every single object in the system isn't great. Is it this is kind of a code smell. Let's not do it So, okay, try's gone again now. We can all relax So when we want to add a method to an object so the good object oriented programming solution is to use Decoration and decoration is where you non-invasively add functionality to one object by wrapping it up inside another object So let's make a decorator class called optional whose instances have a single attribute called value Instances of this class just wrap up another value I can make a new one containing a value like the string hello And then I can take hello out again later if the value I put in happens to be nil I get nil out later Now instead of putting the try method on object, let's put it on optional If the value attribute is nil, it'll just return nil Otherwise, it'll send the appropriate message to the underlying object So now we can call try on the decorator and it will call the method on the underlying object as long as it's not nil If the value inside the optional is nil try will just return nil So instead of calling try on the actual project object and then on the actual person object and so on We can write the method like this We decorate project with an optional object and we call try on that Then we decorate the result which might be nil and call try on that And then we decorate the result of that call and call try on it and so on At the end we pull out the value and return it So that's unwieldy, but at least we're not monkey patching every object in the system anymore There's another smell here, which is that the try method does too much We actually just wanted to refactor away the nil check, but try also sends the value a message But what if we wanted to use the value in some other way when it's not nil? Try is kind of over-specialized. It's got too much responsibility So instead of hard coding the else clause here, let's allow the caller to supply a block that controls what happens next Now we can pass a block in to try and do whatever we want with the underlying value We can send it a message or we can use it as an argument in a method call or we can print it out or whatever and this ability to pass a block in to try is actually a little used feature of active supports try method as well So now instead of calling try with a message name and having to remember that it's going to send that message to the underlying object We call it with a block and inside the block we send the message ourselves and decorate the result in an optional And we could do anything else that we wanted with the value inside that block like print out a log message or whatever That works fine when there aren't any nils But unfortunately we've broken it when nils are involved because we're returning nil when the block doesn't run That's easy to fix instead of returning a raw nil here. We'll decorate it with an optional first and Now it works in both cases But there's a new smell Which is that I don't think try is a great name anymore because we've changed it to do something more general than or at least something Different from the main use case of its namesake inactive support So let's rename it to and then because it really just says start with this decorated value And then do some arbitrary thing with it as long as it's not nil So here's the new version of our method which calls and then instead of try and because we're just chaining and then Calls we can get rid of the local variables So this is verbose, but it's nice We decorate the possibly nil project in an optional object and then we safely traverse all of the Associations and then we pull the possibly nil value out again at the end So okay, how's our refactoring going? Well, we might not be monkey patching anything and it's conceptually clean But there's a huge final smell which is that nobody wants to write code like this In theory it might be better than active supports try method, but in practice it's worse But we can add some syntactic sugar to fix that Here's a definition of method missing for optional It uses and then to delegate any message to the underlying value whenever it's not nil So now we can replace all of this and then optional dot new with just normal message sends and let method missing take care of the details I'll just reformat that So there we go. This is actually really good You can see very clearly that we wrap up the possibly nil project into an optional and then we safely do our chain of methods And then we extract the possibly nil weather out of an optional at the end So to recap this is the whole thing an object which stores a value that might be nil and a method called and then which Encapsulates the nil check logic We added some sugar on top by writing method missing and if you're doing this in real code you should also remember to write respond to I'd like to very briefly point out that we only need to do the decorating and undecorating for compatibility with the rest of the system If the rest of the system passed in an optional object and expected us to return one We wouldn't even need to do that and then we won't have to remember to check for nil at all We could just write the method the way we did in the first place and it would just work imagine that All right, that refactoring was very detailed We're gonna do two others, but we'll have to skip the detail to save time Let's refactor some code that has to handle multiple results Imagine we have a content management application with different kinds of models. There are several blogs Every blog has many categories each category has many posts each post has many comments and for the sake of simplicity Let's assume the comments are just strings So let's say we want to fetch all the words from all the comments within certain blogs for some reason That involves traversing all of these associations again Here's a method that does that at each level we map over a collection and traverse the association for each object inside it When we reach each comment we split it on white space to get its words We have to use flat map here because we want a flattened array of words instead of a nested array If we make a couple of blogs which each have a couple of categories which contain some posts Which have some comments which contain some words and you can see here that my example accurately represents the usual level of discourse with blog comments Then the words in method can pull all of the words out. We're not worried about de-duplicating them or anything We just want all of the words But this method has got a bit of a pyramid of doom going on and plus It's hard to distinguish between the code doing actual work and the boilerplate of dealing with multiple values We can clean it up by introducing this class many which decorates a collection of values Like optional it has an and then method which takes a block But this time it calls the block for every value in the collection and flattens the results together So we can replace all the cores to flat map with instances of many and calls to and then and now we've got Instances being returned we can flatten the pyramid and Reformat the code a little bit to get this So again, this is pretty clear, but we can add some syntactic sugar by defining method missing This is exactly the same as optionals method missing except it's calling many dot new instead of optional dot new That lets us replace all of that and then many dot new calls with just simple message sends This is very nice We put the blog posts into a many object traverse all of the associations and then take the values out at the end And again if the rest of the system could deal with instances of many we could just expect one and return one To recap. Here's the class. We just made for our third quick refactoring. We're gonna tackle writing a synchronous code Does anyone near does anyone here know who the most influential Rubyist is? Now, well, let's find out once and for all We'll find out by using the github API to find the person who's made the most commits on the most popular ruby project When you make an HTTP request to the github API route you get back some Jason and it looks more or less like this Among other things this gives a URI template for finding out information about any organization So now we know what URL to use to get information about the Ruby organization When we make a get request to that URL we get some Jason that contains a URL We can use to get a list of all of the Ruby organization's repositories So we fetch the list of repositories which includes information about how many watches each one has from that We can see which repository has the most watches which turns out to be the main Ruby repository as you might expect and the URL for that repositories representation in the API and When we fetch that repositories information we get another URL that tells us where to get its list of contributors So then we load the list of contributors to the main Ruby repository which includes information about how many commits each contributor has made So we pick the one with the most commits a user called nobu and Finally fetch information about nobu from the URL in the contributor list So it turns out that nobu Yoshi Nakada has the most commits on the most popular ruby project. Thank you. Nobiyoshi Okay, that was a bit exhausting. So let's write some code to do it for us Assume we already have this get Jason method it Asynchronously makes an HTTP get request Passes the Jason response into a Ruby hash or array and then it calls a callback with the data If you like you can imagine a single threaded non-blocking event machine equivalent of this So to do what we just did we have to get the URI templates from the github API route Then fill in the template with the name of the Ruby organization Then get the organization data then find the URL for the list of its repositories Then get the list of its repositories then find the URL of the repository with the most watches Then get the information on that repository then find the URL for the list of its contributors Then get the list of its contributors then find the URL of the contributor with the most commits Then get the information on that user and then print out their real name and username name. So this code works, but it's drifting to the right again. It's hard to understand and maintain deeply nested code like this. But we can't flatten it because of the nested callbacks. So very briefly, the solution is to make an eventually class that decorates a block. The idea is that the block computes a value that might take a while to produce. And then the run method runs the block with a callback for it to call when the value becomes available. We don't have time to go into the details, but here's an and then method that we can use to add extra asynchronous processing to the value produced by an eventually. It's more complicated than the and then methods we've seen earlier, but it achieves the same thing. The main detail here is just making sure that all the callbacks get wired up correctly. So we can rewrite this code by putting each asynchronous get JSON call inside a block that we decorate with an eventually object. So we connect all of the eventuallys with and then, and then we run them. This isn't super readable either, but now we can start pulling out each logical part into its own method. The code that gets all the URL templates from GitHub can go into a method called get GitHub API URLs. This returns an eventually which decorates a block which will eventually call its callback with the result of fetching and parsing the JSON. So we can replace the line at the top with get GitHub API URLs dot and then. The next bit of code that fetches the data for the Ruby organization can go into a method called get org. This returns an eventually object as well. So we can replace the next bit of code with a call to get org and then the code that gets all of the Ruby organization repositories can go into a get repose method and then we can call it and so on for the rest of it. Let me just clean that up. So now that we're just creating eventually instances at each step, we don't need to call and then on each one immediately. We can let each eventually object be returned from its enclosing block before we call and then on it. So basically we can flatten this to get this. Let me just reformat that. So this is much nicer than what we had before. Each part is nicely encapsulated in its own method and the parts are connected together in a clean way. This might be a familiar pattern to some of you. It's similar to promises or futures or deferreds that you might have seen in JavaScript or event machine. So to recap, that's the whole eventually class. Okay. So what was the point of all that? You've now seen three decorator classes that all have this and then method. In each case, it takes a block and somehow calls it with information from the decorated object which could be nil or it could be multiple values or it could be a value that arrives asynchronously. All three of these things, optional, many and eventually are implementations of monads. And I think the most useful way to think of a monad is as an abstract data type like stack or collection. Like any abstract data type, it's got some standard operations. And then is the operation we've already seen. We need another operation which is a class method called from value. We haven't seen it yet, but it's very simple. It just takes a value and calls the constructor to make an instance of a monad in the right way. So this abstracts away the detail of exactly how to call the constructor with a simple value which is different for each monad. In addition to those operations, in order to qualify as a monad, there are some simple rules that those operations have to follow. The main rule is that and then must call a block with a value zero or more times at some point. And this is a much weaker guarantee than the one provided by each. It doesn't say whether the block will be called at all or how many times or when. Another rule is that the and then method must return an instance of the same monad and that's what makes and then calls chainable. The and then implementation in many and eventually already explicitly makes an instance of the same monad. Optional just trust the block to return one. We could have enforced that if we wanted. We could have put raise unless result is an optional or something. The third rule is and then and from value don't mess with the value. And that just means that when you construct a monad with from value, you're guaranteed to get the same value out again with and then. For optional, that only applies to non-nil values, but you should really only call from value with a non-nil value. The big benefit of monads is that they give us a common interface which allows us to do one thing, connect together a sequence of operations. The and then method means do the next thing. But the way it does the next thing depends on which monad you use. For optional, it only does the next thing if the value isn't nil. For many, it does the next thing many times one for each value. And for eventually, it only does the next thing once the value becomes available. And as you might imagine, there are plenty of other monads with different behaviors too. So like the size method for stacks or the select method for collections, we can define new functionality that sits on top of the common monad interface, and we can use that when dealing with any monad. So, for example, we can write a method called within. Within is like sugar on top of and then. Instead of expecting the block to return a monad, we expect it to return a single value, and then we automatically decorate that value again with from value so that you don't have to do it in the block. So because the within method hides the implementation specific business of putting a value back inside the monad, you can use it to write code that works with any monad. For example, here's a method that takes some monad containing raw JSON, representing a GitHub user, and then it uses the within method to parse that JSON and then assemble a string containing their real name and login, all within whatever the original monad is. If we feed it an optional JSON string, we get back an optional description as the result, and we know how to get a value out of that if we need to. If we pass in an optional containing nil, it doesn't blow up because the optional monad won't let description from even try to parse the JSON. It just immediately returns nil. If we make a many object containing multiple raw JSON strings, then description from will return many descriptions, and we can extract the actual strings if we want to. And finally, if we make an object that will eventually return some JSON by asynchronously fetching it from GitHub, then description from returns an eventual description, and we have to run that eventual description with a success call back to get the value out eventually. It's extremely useful to be able to write one method like this and have it work with any monad, and that's possible because of the common interface provided by and then and from value and the operations we can build on top of them. Like stacks and collections, monads are abstract concepts, but they're abstract in a good way. The abstractions give us power. As programmers, we should understand them. We shouldn't talk about monads in hush tones. They're simple and useful. By applying the power of monads wisely, we can untangle nested callbacks, make parts of our code more reusable, and generally make our programs better. If you want to play with the monad implementations from this talk, I've put them on GitHub at Tom Stewart. I've also packaged them up as a gem for some reason that isn't quite clear to me. If you're interested in seeing more computer science ideas explained with concrete Ruby code, I wrote a book that does that. I just found out this week that a Japanese translation is being published next week, so if you speak Japanese, that's all I wanted to say. Thanks very much.