 Yeah, I guess I became the last talk of the day, or the last talk of the conference, even. And it's mid-afternoon, kind of hitting that afternoon slump. And what's going to get everybody up and awake? It's monads, right? Yeah? All right. So my talk's called abstract alchemy, monads, algebraic data types, and parallels pipes. And yesterday, I overheard a few people saying, we don't use things like monads in elixir. And I had exactly one response. So a little bit about this talk. I'm going to be talking about a suite of libraries that I've written to make algebraic code easier in elixir. You don't need to know any of the concepts behind this ahead of time, or be able to program in a language like Haskell, or know any category theory. They're nice to have, but you don't need them. And part of the idea is to make this stuff easy to learn. I'm going to be throwing a lot at you and moving relatively quickly. It's mostly to give you the overall flavor of the style. There's a lot of documentation and examples and concrete ways of doing things in the docs. The first part of this talk is going to be mostly the philosophy behind it and some design decisions. And then the second half is more actual code. And the one big takeaway is as soon as it's a dependency, you can import everything with just use witchcraft. And you'll just get the whole library, everything automatically in your module. So who am I? I'm Brooke. Hello. I'm XP'd pretty much everywhere online, GitHub, Twitter. You name it. I'm the principal engineer at Finhaven, where we do blockchain stuff. Previous to this job, I did about 2 and 1 half years of a production elixir doing consulting. And I organized the Vancouver Functional Programming Meetup. We used to have an elixir meetup and a Haskell meetup and all the different languages. We just eventually combined them all together so that we could have people learn a little bit of all the functional programming has a lot of overlap between the languages and just have a lot of cross-pollination and to bring new people into FP generally. And I've written a few libraries in elixir. This talk was actually going to be about Exceptional, which is an error handling library. Somebody in a talk this morning said, don't just always let things crash. You have some errors and exceptions that you expect to happen. And Exceptional is a library to help make that easier. But then I realized that the library was so simple, I couldn't get a half-hour talk out of it. So switch to the Monad one, right? Also, I have witchcraft stickers. If anybody wants them after, they look roughly like that. There's a couple variants on them. So why witchcraft? It's kind of a strange name, right? Running the functional programming meetup, we'd have people show up. And they're interested in learning FP generally. They're rubious, and they're looking to get into maybe elixir, and they think it looks kind of very friendly. And then they sit there, and they have this talk about histomorphisms. They're like, I have no idea what just happened. I think that was very cool. I'd like to learn it. But that's just absolute dark magic. And the word that they kept using for it was witchcraft. So I thought, you know what? Let's just pick the scariest name possible to bring people in. So what's so mystical about this stuff? This is a very high level of abstraction. It's very powerful. And once you get your head wrapped around the ideas, it's very easy to use. But it can look kind of intimidating to newcomers. Monads, the thing that everybody wants to talk about from this stuff. If I see another, you know, Monads are easy tutorial. I'm just, you know, no more, please. There's so many other abstractions that can do, you know, similar things or that you actually want to use instead of Monads, but everybody goes to that one first because it does sort of hit this sweet spot that we'll talk about later. And because it has this sort of, you know, math basis. Which again, you don't need to know anything about. It can be helpful or interesting to go into. But, you know, people start talking about morphisms and endofunctors, just tune it out. You can still use these things very practically. However, for those of you who do know about this stuff, so in Haskell, we are in the category of Haske, and I'm trying to coin that we are now in X because it is slightly different. So here's a really quick example. I have an anonymous function. I'm going to take one or first argument plus a second argument times a third argument. And my first argument will be drawn from my first list, second argument from the second list and the third argument from the third list. And we're going to do all possible combinations of those just in a quick one liner. So I'm using the word algebra a lot. I don't mean to conjure up, you know, grades of a math class and all the pain that came with that. All we mean here is that it's a set of rules for manipulating the objects in your domain, right? And you're already doing this a lot by yourself, just not calling it an algebra. A lot of your protocols are this, a lot of DSLs are this already. Not all of them, but quite a few. And the payoff of this is we have an emphasis on meaning. You should be able to look at your code and understand what it does declaratively. You have a very high confidence because things become very obvious. Because we're using a lot of abstraction, you can use the same code in many different contexts and still have a shared understanding of what it does. And a lot of these end up being more or less functional design patterns, right? I'm not going to be talking about lenses, free monads, you know, how you can build DSLs with these things, but know that they are there and then all of the ones that we will be talking about today are also kind of more or less patterns or patterns light. And to give a concrete example of this, numeric addition, concatenating lists, concatenating strings, merging maps, uniting sets, all kind of have something in common, right? We take two things of the same kind, so two numbers, smash them together and get back another of that same type of thing, right? Or you take two maps, smash them together and you get back another map, right? And the term for this is semi-group, but all you need to know about that is you just can keep doing this thing over and over and over again and not have to worry about it. And we have a single term and a single concept that abstracts over all of those different ones. So a little bit behind the design, unlike some libraries in other languages, we want to be fully compatible with the host ecosystem. So everything in here should be, you know, you should be able to drop in regular elixir and have it work the exact same way. It's just sugar. It should be consistent with the mental model and the ethos of the language, so of elixir, and your code should be portable with other ecosystems. So I should be able to copy and paste Haskell code or witchcraft code into Haskell and change as few characters as possible and have everything still just work. It should be very teachable and an emphasis on documentation and concepts. And we should be concerned more with concepts other than the languages that we're in, right? Concepts are the interesting thing. When you pick up these concepts, you'll be able to learn other languages more easily and vice versa. So Jose gave a nice summary of the design principles behind the beam and very pragmatic, right? It wasn't, they didn't set out to make a functional programming language. They iterated on a bunch of different designs and they happened to come up with something that was a functional language and it's not as ideologically pure and focused as, say, Haskell or O'Cammell. But it gets a job done and it's definitely a functional programming language, right? And you see one of these nice, nice, pragmatic, usable languages like this and we all ask ourselves the same question. How can I make this more like Haskell? By bootstrapping a whole bunch of stuff. So three main libraries that sit around Witchcraft. Quark, Type Class and LG and they're related in this way. So Quark and Type Class are requirements for Witchcraft which is a requirement for LG. Quark was the first library I wrote in Elixir. I got to Elixir and I was actually very surprised. I didn't have some of the really classic common functions that I was expecting to have, like an identity function, something that you get an argument and you just give that argument back, which you can use to do things like stub out functionality in one of your other functions. Flip which reverses the order of your arguments and so on. And these have existed since the mid 1930s, right? ID obviously even further back than that. So we didn't have them. So wrote them. Also brought in functional composition. So instead of having to always pass through an argument, you can take two functions, compose them together and get back another function without having to mess around with the arguments. The arguments just get piped through automatically. And querying. So in Elixir we have this concept of erity, so foo slash one is different than foo slash two, but for some things to work in Witchcraft we need to have the ability to partially apply them very, very frequently and without any over, sorry, without any programmer overhead. We have that both dynamically, so you can wrap things, you incur a small runtime penalty for that because you actually have to go and wrap, however many levels deep. There's also a macro for doing this at compile time and then you don't have the runtime hit. Type class really just de-shuggers down to protocols. It's protocols with dependencies, so we can say I have foo and bar, and I can only make an instance of bar if my data type already has a foo. So using semi-group from before, these things that we can just keep sticking together. If I want to extend that and say that this also has the idea of an empty object, so I want to have an element that when I add it together with this, it doesn't change anything, like in concatenating with an empty list, doesn't change your original list. I would say this is, I'm going to extend semi-group to Monoid and it just has this one extra function that you have to implement. The other one is, so protocols can't have functions defined in them, they have to be in a separate module. So enum and enumerable are the really classic case. You define your protocol for enumerable, or def-inst on enumerable, and define map and a few others, and then you get all of these functions, but they're totally separate. So as a user of enum, you don't know that you have to go and look at enumerable without digging through the docs. We just paper over that and say def-class and then whatever it is, so I can have def-class enum and I automatically get in the same file, I can say these are my functions that the implementer has to do for me, and then out of that you get all of these functions that depend on those. And then on the other side, we have algae, which is a DSL for writing structs. You can do quite a bit more than just witchcraft with this. It kind of cleans up the syntax a bit. It generates both structs, the types for the structs, and lets you do some types. So to say, in a type spec, this is gonna be an A or a B because it's, well, you'll see in a moment. So this is def-data, and I have my three fields and the types that go with them. So this would be like a character in a video game, let's say. This is my def-sum, and I'm saying that these two structs are related. So there's my datas and that's the sum. This def-data at the bottom, I don't have to define the entire struct in all the fields and whatnot. I can just provide these types because these are very simple ones and it'll do that for me automatically. This captures the idea of what's called a maybe. So just like how a list can have zero or more values, a maybe can have zero or one. Those are my only two options. And it captures this idea sort of like a nil, but nil is an actual value. We're saying with nothing that this is literally no value. Don't do anything with this. It's empty. Or just where we say we just have one thing. That macro expands out into these two modules. So you get type, destruct, and some helper functions to do. So it's the def-new there, which is just a constructor, basically. And we get them because this is a def-sum. This is sitting inside of this lg.mayby. We get the type of the two together. One use case of this maybe idea is capturing the idea that something went wrong or that you have failure. You don't really care what the failure is, but you wanna just propagate the fact that you have no valid result back. Don't actually wrap things in a tri-rescue block, but this kind of gives you the rough idea of the meaning behind it. So I'm gonna try to add two things together. If that works, I'm gonna get back a just, and if that blows up, then give me back a nothing and just swallow the error. So if I add together two numbers, I get back just three, and if it blows up, I get back just nothing. So we wanna keep this really elixir-y. We wanna keep it consistent and feeling the same. This hasn't always worked out this way in some other libraries, and I'm gonna call it scholars in particular. So they have a similar library in Scala, and you have this split in the community where you have people that are writing basically Haskell fan fiction, and people that are writing Java without the parentheses, and never, you know, you don't mix the two, right? So we don't want that. We don't wanna split the elixir community. We have this nice friendly community, you know, mostly coming over from Ruby. We have Dataflow function application over composition. We do everything, you know, with pipes. It's dynamically typed, or because there's no such thing as untyped or dynamically typed. It's either type checked at compile time or type checked at runtime, right? So this is type checked at runtime, and it might blow up, but that's one of the trade-offs. This is one of my favorite comics that we want to avoid. This isn't just a hypothetical situation. Profuncture optics are coming to witchcraft, and they don't have to be quite this scary. Really, it's just a super powered getters and setters that are totally pure. So bootstrapping your intuitions a little bit using just pipes, right? Directionality and Dataflow. So we like something called diagrammatic ordering in elixir, which means that we just pipe things through in the same direction as per line or in the same direction as this arrow in our pipe operator. And this is just a little bit of pseudocode in that diagram on the side with, you know, at each step what my value is. So I'm gonna take two, multiply it by two to get four, and add one to get five, right? So what if we just gave this some super powers? So let's do this same thing, but with lists. And we can think of lists not just as a sequence of numbers, but as almost superimposed all of these values, right? I'm still going to multiply by two and then add by one, but I'm gonna do it over all of the values in my list, right? And all we've done is we've changed this vertical bar to a squiggly line to a tilde. That's it. We automatically get the same behavior, but running inside of this context or inside of this container, that's a list. We also automatically get asynchronous variance because we're not making any assumption about evaluation order, right? So by just saying async map, we get the, actually from the last presentation, you had enum.map task.async, piped into an enum.map weight. You just get it as a one liner. And we can keep extending this idea and give you the idea of being able to flip them around too. So in the top right-hand corner, we're going from left to right and below that right to left, but it's still always data going into a function. So if you're coming from, say, Haskell or PureScript or one of these other languages and you're used to seeing the function on the left-hand side, you still have a version of that, but your operator's backwards to how you're used to thinking. But this is the elixir way of thinking. And as we go down this in the second one here, we get more powers we go down. So I've got pipe and there's no reverse pipe because that's just regular function application. That's like a space or brackets, right? That's it. For map, we have squiggly arrow, both directions, apply and chain, which we'll see in a moment also, just using these operators, using varying levels of angle brackets. So I said that we have a couple design patterns in Witchcraft. Here's all the ones that are presently in there and we're going to be looking specifically at this one. This whole hierarchy, Functor hierarchy. So Functor, first one, gives you, you provide to it implementation of map and you get back out a whole bunch of functions. This is the closest one to Enum. Enum kind of mixes a few of these, but this is the closest one. You get the tilde angle bracket pipe operator. You always get back the same type of data as you put in, unlike an Enum. So in Enum.map, if you feed it a map, the data structure, you'll get back a list. Here, we do automatically stick it back into the map for you so that it's always consistent. Your inputs and outputs are always the same kind. Apply is the next one down and I like to call this one thinking inside the box. So all we're doing is taking elements that are wrapped in some kind of container, in this case a list, pulling them out and also have a function wrapped in some kind of container. Pull that out, apply the arguments to the function, stick them together, put them back in a container. In the case of a list, I'm saying that I've got one or more arguments and one or more functions, so I'm just gonna do all of the arguments on all of my functions and get back the result of all possibilities. We also have this handy function that comes with it called lift and there's various levels of lifts, but I won't get into that too much. So I'm saying here's my two sets of arguments, one, two, three and four, five, six and I'm gonna do multiplication and I get back all the possible multiplications between those two lists. So I do one times four, one times five, one times six, two times four, et cetera. And then using this maybe concept from before, if I have two justs, so I'm saying I have an actual value and an actual function, I'm gonna pull those out, apply the function to the argument and stick it back and adjust. If I have no arguments, I'm gonna short circuit and just say this is a nothing and same again, if I have no function, I'm gonna short circuit and say this is nothing. Once you have those, you can go down this side of that diamond shape and provide a function called of, which is sort of like list.wrap, but for anything that implements this type class or this protocol. So it lets you lift a value into this data type so that you can write, it's kind of like how you have struct and then you have to specify which one you want it to go into. Similar idea except on more than just structs. So in this case, I'm gonna say of a list 42 and I get back a single element 42 or I can do the same thing with adjusting it back 42. Chain and monad are very, very, very close. This is the interesting part from monad that most people are interested in. We've just gone here and split it out into two halves just because you don't always want all the properties from monad. So it's kind of like apply, but we have instead of doing it with all of the elements every time, we have what's called a linking function, which lets us, when we pull that value out, we can look at it and decide what we wanna do with it. We can change our branching logic, we can change which function we wanna execute, et cetera, based on where we are in our flow. So it's fully Turing complete, which is the main reason why I think people get so excited about it is because it hits this nice sweet spot between the two. So pull that out, look at it, put it into the next in that chain, and then you can keep doing this forever. The interesting example down here isn't doing it forever. I would say the top one on the right-hand side. So we've taken one, two, three, and then I have a function x over two, two x's, and it just doubles every value in there. So it kind of gives you a nice picture of what this does, right? We take a single one, return two, it's the same thing as doing a flat map. In fact, this is flat map. And then we can just keep flat mapping on all of those, which is the bottom right-hand side. So why are these so popular? Why do people get so obsessed with them? Because of the trade-off between power and generality. So things are very general when you don't make any assumptions about them, right? If I have something that's completely general, I can't use addition on it because I don't know that it's even a number, right? It might be a string, it's completely general. As we start saying that we know more and more things about it and starting to constrain what we're working on, we get a little bit more power, but then there's also a trade-off that we have fewer and fewer things that we can do these things on. So chain and monad hit this sweet spot between the two, right? We only have to provide a handful of properties and a handful of functions, and we get back all of this stuff. The other parts of type class I didn't mention before, or I should have, is they're not just, so they do de-sugar to protocols, but they also have properties associated with them. So an example of property is something being idempotent. So if I do something once or 50 times, it doesn't matter, it's gonna be the same result. So if I round a number once or twice or three times, I'm always gonna get back the same value. Every type class has to provide at least one property, and if it doesn't, it's going to prop test them at compile time, right? So this isn't a test library. This is a compile time guarantee that you have. If it doesn't pass the prop test, it'll fail, right? You can override it if you want to live dangerously, but you shouldn't, right? And sorry, yeah, the final reason why these are so popular is the first one with do notation, which we'll look at in a couple of slides. And then because of this idea that it's almost like a chef. So with a map function, you can't stop three elements into your map and say, oh, this is suddenly over a hundred, I should stop here, right? Each of those is totally isolated. With chain, you can, it's like you have a recipe, and that's each of your chain linking functions, right? And at each one of those, you can stop and taste how far you've gotten through. You can taste how your recipe is and say, I need more sugar or I need more salt or more pepper and decide if you need to go back to another step or what exactly you need to do. So you can change the branching of your program. This is how these all increase in power. So with application, regular pipes, I just have data and a function and get back to just regular results. With functors, I have a wrapped data and a regular function and I get wrapped results. One or more results. With apply, I have wrapped data and wrapped functions and I get wrapped results. And then with chain, I have some kind of wrapped data and this linking function that goes from a single value, like an unwrapped value, into a wrapped value that we can inspect and then give me back wrapped results. With apply and chain, we also have this nice property of because they need to do things in these contexts, right? They have these wrappers. This is how we can model effects in a pure way using both of those. So in the case of say, apply with maybes, right? When I had that apply and the function was nothing and I just returned back nothing, I get that short circuiting behavior. That is an effect in the system. We're saying because this didn't happen, all the rest of these, everything I do after with this is going to be also nothing. Do notation, when I first suggested this to some of my friends, they thought that this wouldn't actually be possible but luckily we have a really nice macro system and it was very, very surprisingly easy to do. And this gives us back that operational feel that we're used to from Elixir. And it lets us write some really nice DSLs which will be in a few slides. So this is the regular chain function using our three arrow operator. As you can see it's becoming this really deeply nested thing because we're gonna be relying on that bound X and that bound Y and that bound Z as we go through. With do notation, it changes the look a little bit but these are completely identical. So this will de-sugar, the macro will expand into the top version. And so when we're doing this binding, we're saying the variable name that we want on the left-hand side with an arrow pointing at it and the thing on the right-hand side. If any of these steps returns a nothing, we're gonna get back nothing. If all of them return justs, then we keep doing the functions inside those and we'll get back a wrapped value. There's one other construct with it which is called return, which is it's not returned like in Ruby or JavaScript. It just means it's that of function that I was talking about before. It just means wrap this up in the data type that the monad is. So in this case I'm gonna say this is a list, monad list do. Here's my nouns, adjectives, verbs and reactions. I'm gonna make a madlib out of this and then I'm gonna return in a list each all possible combinations of these words. So in order to get a return, we need to say which one that I'm in. Here's my inputs and there's all my outputs, right? Which is not bad for 10 lines of code. Here's the entire implementation, three cases. So it's really not that bad. Here's some common use cases. I'm gonna be talking about the writer monad. So writer's pretty simple. We just have a tuple and we need to tell it a little bit more than we did in the previous slide about exactly what's in it. But I'm saying that the left value or the first value is my actual what I'm gonna be doing regular computation on and my second value is a log, basically. Tell is a function that I get from the fact that it's both a monad and then writer and this is available in LG. It says write this to the log or add this to the log. And it's going to append it, right? So it's not just going to overwrite. We're gonna get an appending to it. And then we're just gonna return the square of whatever I take as a number from the wrapping function. I'm gonna run this three times. And because I'm squaring it every time, this is number squared, squared, squared. So it's the same thing as being to the power of the eighth. And because I'm appending every time, how many times it's run with that tell, I know how many times I've done this. So I'm able to say that my input value, which is 42 raised to the eighth is this. Which is kind of a cool party trick, but you can write this whole thing, this entire data structure to a database or send it over the wire or wherever and its log travels with it. So we can stick it in a database, forget about it for five years and come back and write it three more times and it would keep incrementing that same counter. This isn't a global log, this is a local log. Doing the same thing with strings. So you'll notice now the second value is now a string, says log. I'm gonna log my initial value, which is coming in from the wrapping function, and I tell it string. The actual thing that I do with this is I'm going to just add an exclamation point to what I have. I'm going to log again with the new version and add a dot, dot, dot. And then return the value so I can keep binding and keep linking these. I run that three times and I get back out of my logs every update that happened to my value. So I go from a regular high to a high with one exclamation, exclamation to two, two exclamations to three. And again, this travels around with the value. And you could absolutely do this by hand, but we just get a nice DSL because it's a monad. And last one I'm gonna talk about is arrows. So I'm not gonna talk about, you know, more than just the conceptual framework for what arrows are because they're pretty abstract, but we can think of these as data flow on steroids. So with a regular pipe, we would have only one track. With arrows, we can split them and have multiple tracks. And we're not gonna make any assumptions here about if these are running in parallel or linearly. We could run this same graph in either case, depending on how we send it arguments, right? So to give you an idea of how this works, I have an input 20. It hits the split. I make two copies of it and send those down the two tracks. The top one is gonna go to the very end. The bottom one, I'm gonna keep piping through more functions. So x plus one, I get 21. I split that again. The top one is gonna become a string. The bottom one, I'm gonna square. I'm gonna unsplit that by sticking them together as just taking the bottom one, turning it into, you know, inspecting it as a string and concatenating them. Do the same thing with this top value. Stick them together, put them to the output. And that's all I have. So what does the actual code look like for this? So here's a shrunk down version of that graph. Here's my code and how they all line up. So fan out is the splits. And if I rotate this, you can even lay out the code in a way that the formatted, I'm sure, formatter I'm sure would hate. But you can lay out the code in the same way as the graph, right? So this is very, very powerful, right? We can say with just function composition that I have something that might run sequentially or in parallel based on what I give it as an argument eventually. Future directions for these libraries. I want more data types. So there's maybe about a dozen or so in LG right now. I want to have more type classes as well. So I want to keep extending those hierarchies. I want some pretty printing of the ADTs because some of them can be quite large so you don't want that. Or we want to have things like we have trees. So having those layout nicely as trees would be handy. Automatic driving for some things. So if you make a monad instance, for example, you actually get everything, all the dependencies for free, or you can with defaults. So it'd be nice to not have to define five different instances. You could just define one. And alternate minimal definitions. So for some of these right now, you're forced to define a particular function, but you might be able to define another one because they can drive each other. And so it'd be nice to, for example, foldable things. Right now you have to define a right fold or a foldR. Right now that'll give you foldMap, but if you give it foldMap, you should be able to derive right fold. So that's my talk there. Oh, the text is actually really small, but there's my libraries there. Again, I'm XP'ed all over the internet and you can get a hold of me at hello at brooklynzellandcut.com or come talk to me after. And that's my presentation. Thank you. Great presentation, by the way. It's really interesting seeing some of this work happening elsewhere, like enclosure, like to bring some of these advanced functional techniques. Maybe are there some problems that you've seen in the elixir space of things that we can't solve without these sort of very interesting data types? Yeah, so there's nothing that you can't solve because everything, like, you have a terrain-complete language, right? You can solve anything, technically. What this does is it can help you clean up your code, use some nice patterns, and write some more. I don't really see this getting used as much at the application layer. I see this more getting used in other people's libraries to write code that can be used in several different situations when you don't know as much about the exact specific use case, right? So with the writer, for example, I can use that in a bunch of different ways and not just for logging, right? There's another monad called state, which is like a writer plus the ability to read the data back out of it at any time. And you could extend that. There's a concurrent one as well where you could wrap up things like raft, databases, all of that and provide a single interface that's lawful, you understand the properties that it works with, et cetera, so that when you see an ask on a state monad, you don't care where it comes from, you just know how it works, right? And so you only have to learn these concepts if it's in these wrappers, you have to learn them once, and then people can use them the same way. Thank you.