 Hope you had a great functional company now I'm Marcus Hawk. I'm concentric and today I will talk about free all the things So maybe show a friend who knows about the free monad Okay, the free applicator Less hands who knows about the free monoid Okay, perfect because you're here to learn what it is and Basically, we will talk about free whatever you want whatever type class you want to free So I will show you a technique how to do that So the goal is as I said to explain this technique We will apply this technique on different examples so you get a feel how to do it yourself And I hope that you can have a take as a takeaway that you yourself can do this at home find a type class and free it Okay, so let's go Here's the definition and it says a free functor is left adjoint to forgetful functor, right? That's easy What's the problem? It's like the monad meme Okay, here's another definition. I call I I came across when I searched for free and it says a free thing Let's call it free a on a type class that we call a is the a itself So a monoid is a monoid a monoid is a monad and a function We let's call it inject returns it takes an X of type a and returns a free a And it has to obey some rules such that for anything. Let's call it B and a function f from a to B They exist the unique homomorphism G such that decomposed with inject is the same as that So basically what this says is that the G function and the inject function have to be the inverse So inject brings you into the structure and G brings you out again. Okay. This is still very complicated So here's a more much more simple definition that I think Hits the point pretty well a free X is the minimal structure that satisfies the laws of the type class X and If you don't get this yet Don't worry because there's an easy mechanical recipe to how to free type class And it's basically four easy steps that we will repeat in the following slides First we define an AST for the type class operations. We add an inject operator We will see that in a moment We write an interpreter for it and then we check the laws Okay, but before we begin why should I care about free at all, right? Okay, so first of all you get a nice API that is based on the type class That means if you have a free monad every user that knows how to use a monad can use a DSL If you have a free monoid everybody who knows how to use a monoid can use a DSL and so on You can basically then use your free X as if it was a monad That's what I said and you can lift your DSL operations using this inject clause We will see and you get back your program as a data structure And this is really cool because you can optimize the program you can analyze it and you can interpret it in different ways So your your user may not have thought about executing a program in parallel and he just doesn't care But you can do that Okay, some disclaimer What I'm going to do is also called the deep embeddings approach or the initial encoding or the data structure representation and it's different from the final Techless approach Finally Techless would also have been possible for this talk, but especially later when we go into program optimization This is much more cumbersome For some type classes there are also different sets of minimal operations like for monad there at least three different sets of like you can have flat map and Return and you can have return join and bind and this technique actually works for every minimal subset And if you want you can try this at home with different subset I've just chosen one arbitrarily more or less because I can show something cool with it and This talk might include some hand waving at its but it's for your own good and hopefully you wouldn't know this Okay, so let's go into freeing the monad first So here we have the monad type class that has the pure function instead of return and The flat map function that given an FA that is wrapped in this higher kind of type constructor And a function that is gets the a returns an f of b You probably know this type last by now. Hopefully And as you might know if you have been in George talk There are also have to be in some laws for the type class and for the monad. We have three laws We have left identity Which says that when you lift something into the monad with pure and you flat map over it This should have be the same as just applying the f purely and If you have an FA and you flat map over it with pure this have no effect at all and There's also the associativity law which looks a bit a little bit strange with flat maps, but you just have believed me on that There's just associativity Okay, so we know the type class we know the laws yet go to the recipe So the first step as I said is to define an AST of the operations So we go on and we create a seed after a class that we call free And we have one class pure that captures the pure operation and we have one note in our AST That is called flat map and just captures the argument that our normal flat map function would take So we have an FA of type 3 of FA and a function that goes from a to a free of FB and Then we add one more case that I called inject. You can also call it pure or whatever you want Okay, not pure because I'm already using pure, but there are different names for it And then you can have a monad instance for our data structure and Whenever we call pure on our monad we just capture that in the pure note and Whenever we call flat map, we just capture it in the flat map note So in essence, we are just building up an AST when we are using the monad of the free function of the free monad Okay, and then we need something as a next step to go back from our free monad into the land of the normal monads so for that we write a function that is called run free and It takes a natural transformation that translates this higher-kinded F into some target monad that is here called M So here you can see that M has a context bound for a monad and Then it takes the free monad also as an argument and what we do is we just pattern match on the instructions And if it's a pure note, we translate it into the target monad's pure If it's a flat map, we translate it into the target monad's flat map and run free recursively down the two notes And if it's an inject we actually call our natural transformation And this is the part where you reduce your DSL into the target monad as well Okay, so we have our structure we have our free monad and now we need to care about the laws and As you might remember there is a associativity law for free monad's I just had it on this slide here again And when you look at our structure if you translate this to the AST You can have two expressions one and two the one is the flat map of the first equation of the left-hand side of the equation And the second one is the right-hand side equation So obviously this is not equal right, so we are violating the laws that means it's not the free monad Okay, so as a programmer you can argue that okay It's not really the free monad but after interpretation if your target monad is lawful nobody will notice and The metametrician or the category theorist will tell you know this is not the free monad So there's also a kind of a trade-off between enforcing these laws via construction I'm just taking this as a as a problem, but ignoring it and getting rid of it during interpretation But let's see what we can do here in our case So we have this flat map operation for our free monad monad instance and Instead of doing the normal flat map. We can actually do some optimization So we have the left identity law which tells us that whenever we flat map over a pure note We can just purely apply the F. So let's just do that in our flat map implementation here And we can do the same thing for the flat map Where it says that if you have the so that we are already in a flat map and we are flat mapping again And then the associativity law tells us that we can transform this into this Equivalent version and we can just reason this by the laws if you have been in George's talk, you might Recognize this so here's actually the equation again So we just applied the left hand side which has two inner flat maps And we just transformed it to the right hand side with a nested flat map Okay, so maybe a small quiz. What about the other identity law can't we just optimize when we flat map with pure? Does anybody have an idea? Can we do this or is that problem? Yes, it's a problem So actually the problem is that we would have to pattern match on the left hand side And we would have to determine that the function is a pure But we cannot compare functions for equality. We cannot take that as a pure function So we can actually we have to ignore that the law we cannot enforce this in our structure Okay, so as you have as you can see laws are not boring They allow us to do refactorings without changing the program meaning and they also allow optimizations Because for example in this case by doing the associativity to the right we can have a stack-safe free monad Because if you look into the structure instead of building up nested flat map notes that we have to burst down and up again We can now have a outer flat map note We recurse into FA and we have a function and then we apply this function and we get back a new note So effectively what we have reinvented here is trample lining for our free monad and will be stack safe And we didn't even think about this Okay, so for with this we are basically done with a free monad So we we have a DSL with monad expressiveness. We can use this to embed our own DSL We have context sensitivity. We can do branching in our DSL loops fancy control flow But this is not a free monad talk So if you want to learn more about this you go have to go to a free monad talk and Said what we will do is we will go and free the functor and Again, we will do the same four steps. So we define our ST at the intake clause Right our interpreter and we have to check the laws Okay, so here's our functor type class. It's easier than the monad We just have one function that is called map and given an f of a and a function from a to b it returns an f of b What are the laws? Well, we have the identity law which says that when we are mapping with identity over it They should have no effect and we have the composition law that says when we map twice This should be the same as mapping once using the composition of the functions Okay, so again, we we create an AST. We call it free functor There's two cases one is the FF case from our type class and there's one additional case for our inject Okay, that's basically it. That's a free functor, right? That was easy The next step is to minimize the structure again and we have to ask ourselves. Can we get rid of anything? So if you look at our structure again, and you look at the inject case You could actually replace this with F map Because the inject case just has an FA of type FA and the FM also has an FA here type FX But it can be just renamed and then we just have to fill in this F with something and We can write a smart constructor that fills this F with identity So we can just get rid of the inject case completely and we have a more minimal structure Okay, so Does anybody see a problem with this code snippet and as a hint if you're on our program You should immediately see this if you think from our perspective any ideas So we have a superclass, but we have just one subclass. What's the case of a superclass if you only have one subclass and We should check if we could get rid of this one superclass Okay, so and for this we have to ask ourselves. What role does the super trade or the superclass currently play and Currently it serves just a purpose that the free functor has two type parameters and the F map has three And so to get rid of this mismatch. We can use a type member Put the X into an existential type member and just get rid of the superclass completely So this is even better because we only have one node the F map node and The inject just gets a little bit more complicated because we have to instantiate the the type member and If you look closely at this it might get familiar depending on how deep you are into category theory And if you rename it, it's just co-yonida And it already sounds much cooler than just freeing the functor and then you can just write the functor instance for our free functor or we let's call it co-yonida and This is basically the mitigate the next mechanical step and then you run your free functor By mapping over it and just applying the function that was composed inside Okay, that's basically it we freed the functors So what do we get? Well, we get in DSL with functorial expressiveness. What does that mean? Yeah, well, we can only map But what we get for free is map fusion So our DSL users can map as often as they want and they don't have to care about intermediate structures Or whether they have can do a map map future optimization on their own we take care of that for them and What's even more interesting is that functors can often be composed with other structures giving them map fusion as an addition So for example, there once was a free monad that had in a composition with a free functor just to get map fusion for free on free Monads Okay, that was a free functor. Let's go on to free the monoid And again, we look at the type class So the monoid has two operations the first is called empty and the second is called combined You are probably familiar with that by now And if we blindly follow our approach, we start with the structure of the free monoid Then we have the empty case the combined case and our added inject case Should be quite familiar by now And we have three laws. It's left identities. So empty combined with X So the pipe plus pipe is just an infix combined should be only the X and the same for the right-hand side so this essentially just says as empty is as boring as you want it to be and We have the sociativity law Which says it doesn't matter in which order we evaluate this expression whether we first evaluate two plus three and then plus one Or whether you evaluate one plus two and then plus three the order doesn't matter Okay, so now let's try to enforce those laws on our structure and even through monoid is actually simpler than the functor and the Monad it will be a little bit more complex to enforce those laws and We have to make one almost arbitrary decision. You will see in a moment. Why I made this choice We will associate to the right. We could also have associated to the left, but we will get a little something out That is a little different Okay, so to fix associativity we have to And decide as I said for left or right and then we have to forbid the other one So what is okay in our case would be to have a combine With a left-hand side is an inject and then on the right-hand side that can be more combines But we cannot allow that on the left-hand side. There is an inner a combine because this would be left associativity Okay, so we can solve this by having another trade that just says this is a not not a combine So this is basically just a marker and then in the combined case We say the left-hand side has to be a not combine So this can either be an inject or an empty, but it cannot be a combined because combined does not extend not combine Makes sense Okay, so this would get rid of the associativity for combining, but we could still do combine of empty and empty so we have not only one empty we could have two mts and To minimize our structure, we can ask ourselves. Can we get rid of this duplicate mts or can we get rid of empty at all and It's not possible to get rid completely as we will see in a moment But we can limit ourselves to have only one empty in our structure and For this we have to restrict our combine to not only forbid on the left-hand side The combine function might be forbid combine or the empty That's what empty can be only on the right-hand side and combine as well And again the goal is to have the minimal canonical structure for our free monuate Okay, so if we do this we have on the left-hand side of combine only the inject case and Then we have our case class inject that doesn't inherit anything But if you look a little bit closer now inject is kind of silly because it's just wrapping things up. It's just identity So we can get rid of we can basically get rid of it completely So even better we have one structure one constructor less in our AST Okay, so let's try this structure and write the monuate instance So for the empty case we return the empty node and for the combine we pattern match if it's an empty We return the right-hand side and if it's a combine we combine the head with a recursive call on the tail Okay, this might start to look familiar again Does anybody see a familiar structure? Maybe look for a moment Okay, we maybe it helps if we rename everything a little bit Do you now recognize it? so we basically just reinvented the normal single link list and If you associate to the right you get the normal list if you associate to the left you just get the back what single link list Okay, so for this with this we freed monoids and We get a DSL that we combine stuff with which is pretty handy most of the time It works beautifully with folds for example map reduce. It's just if you just use monoids and folds. It's beautiful And we can also for example interpret our program in parallel Recursing down the the nodes of our AST because of associativity So this gives a lot of freedom to us when we have programs that our DSI users have written Okay, it seems like this approach works pretty well So now that we can free basically anything we just need a new nail for our hammer and When I thought about this I remembered an old blog post by Chris Duccio Which talked about free objects and he used free Boolean IGEPRAS as an example So this is a good example and we have our recipes. So let's just apply our steps to Boolean IGEPRAS And if you're not familiar with them a Boolean IGEPRAS has basically a true case the false case Which you know from normal Booleans and then the not that negates the and and the or From this you can basically build your normal Boolean IGEPRAS using other ones that can be derived from this like X or NAND or whatever you want Okay, but let's just apply our normal steps again So our AST looks a little bit more complicated this time we have the free Boole case and we have a case object for true and false and We have cases for the not the and and the operator which all just capture their arguments and Again, we have our edit inject case Okay, so now we just need an interpreter for our Boolean IGEPRAS. We will step will skip the optimization in this case and If you want to interpret our free Boolean IGEPRAS That is given as FB. We need a function that also goes from a free Boolean of A So here's the A need a function from A to B to get rid of this and The target is another Boolean IGEPRAS So we say that the type B has to be a Boolean IGEPRAS We have a free Boolean of A and a function that maps this A's to a Boolean IGEPRAS as well So when we tear down the structure We better imagine our operations if it's true We turn the target true if it's false return the target false if it's inject we use our actual function Well, this is where we have to translate the holds in our DSL that some user would have written Into the target Boolean IGEPRAS as well and for the other cases we just recurse down and use the target Boolean IGEPRAS operations and There were nice animation for that Okay, that was simple although it was a little bit boilerplated by now because it's always the same pattern We will skip trying to cope the loss basically because I don't know how to do it And it's much more involved than Monoid because there are a lot of laws for Boolean IGEPRAS But as long as the target we are interpreting in two is lawful at least morally this is okay So what's more interesting is what can we do with our new discovered structure and as reminder our DSL users can do and anything that the normal Boolean can store like true false and or XOR implies and then or whatever you Boolean operation you want Okay, so here's the example that I came up with Let's imagine that we want to write a new website search basically a new Google because we need that and For this DSL that our users can use We have this search ADT Which has different key different key bits for our DSL and it can search by term It can change search whether something is after a date It can check whether something will cross in the text or whether it course in the US in a URL and After it's leaking in some syntactic sugar our DSL users can basically write predicates or searches like this So we can say that there has to be the term functional programming It has to be indexed after 2018 1 1 There has to be there there should not be the term Java or spring in the website and The URL should contain functional conf So what our users are building up basically is the AST that looks like this So we have the end of the term node Then we have invested and that has the after note another end with the not and the or and then at the Leaves all are just our DSL operations. If you look closely Okay, now we just need a type that we can run our predicate against and for our domain Let's imagine that we have stored website case classes that contain Handily some terms URL the index at date and the full text of the website So all our predicates will be run against this site Okay, we know how to chew down our free bull I've shown that on previous site, but we do not yet know how to interpret our search DSL and For that we have to create a new function. I call it evil search that takes a free bull of search and The site and returns a boolean and what it does is it uses an inner function that I call net That goes from our search to a boolean and Then we can better match on the term and check whether our actual site. We are checking against Concerns the term T here where there was indexed after the date we specified Whether the full text contains the string we have given it and whether the URL contains the the word we have expected and Then we just call our teardown operation the run free boole to get rid of all of the AST and we will get back through our folks So what you can for example do is if we go through all sites We can filter them and evil call evil search on them and we will get back all sites matching our predicate Okay, so let's get back to our predicate that we've seen on the previous slide so if you have a site like this here and You have our predicate you would basically iterate over this you would check that for the term note Whether the term is in fp or Bangalore. So this would often be true But that was indexed out of the first of 2018. This would also be true Whether the term Java is in there. This would be false whether text spring is in there This would also be false and in the URL should be functional con and this should be true So basically you will get back AST that looks like this after eliminating your your DSL and Then you can tear this down to basically true or false at the end Okay, so this is like the basic usage of free Boolean Agapra But that's actually a lot more interesting stuff that you can do with this For example, you could short circuit or do other optimizations So for example here if we go to if we would have faults here You could just forget about all of the right-hand side. We don't have to evaluate all of that You could also do like eliminate double navigation Perform some canonical forms of the Boolean and an or You could partially evaluate the predicates So we will see this in a moment for example imagine you don't have all of the information required to evaluate your predicate fully Then you can partially evaluate it and maybe just load more information or send it on to another party And what you can also have which we are not show today is you could have a core language like our search Predicate that is barely low level and then you could have a higher level language that is a little bit richer and For example encapsulates the term matcher with it's like just taking a string and you could replace that with is fp website and Translate this into your primitive language as term has fp Okay, but let's talk about the optimization first. So this is pretty easy here's like one Simple step in the optimization case You could apply this apply this as a fixed point and just optimize your whole program And what we do is we just look at some basic patterns So for example if we find a double negation We can just forget about the double negation if we have an aura that has a left-hand side of or the right-hand side of true We can just short-circuit and the same for the end case But there are more things you could also just recognize whether there is an X or and stuff like that But the more interesting case is partial evaluation and As I said just imagine you have my totally have a part of the information that is required to reduce the brady-cade For our problem you could imagine that the full X full text of the index website is there is stored somewhere else and You would like to avoid loading the full website if possible So the idea is to evaluate as much as possible if we can reduce it to true or false Okay, that's perfect. We don't have to load the text Otherwise, we might have to fetch it or maybe send it on to another system that can retrieve the full text So for this let's create another case class that is just called site metadata That has only the terms the URL and the index at so the full text is missing here and Then we can write another function that is called partially and instead of taking a full website It just takes a site metadata and our normal search predicate So from the user's point of view, this is unchanged and it returns an option of mboolian and the option is basically the semantics of If it's some then we could evaluate this and if it's none, we just didn't know what to do We didn't have the required information So in our case for the term after and in URL case we can do the evaluation and for the index We just have to say none. We don't know how to do that and after we have this we can write our partial evaluator That just is like our normal evaluator But it takes instead of an A to B here it takes an A to option B Which fits the signature we have written on the previous slide and then it just tries to evaluate everything Here I'm using an either sneakly for the short circuit thing You can think about this if you want to or talk to me in the hall but it's a little bit tricky and then we can just recrust down with this partially evaluator and whenever you can Evaluate something you just replace it with a value and otherwise you have to go on and do more okay, so let's go back to our basic predicate and Here's the same site, but without the full text and we can evaluate everything to to true Except for the intact spring So basically we can get rid of all of this, but we have to keep the index We can rid of the or because this is false So this reduces to only the index node and we have to keep the knot So what we can get out of this after running our predicate is a new predicate That is reduced to only not intact spring So we just have a smaller program now and if you look at the same predicate, but a different website and Evaluate this you can for example see in this case for a website spring IO there's the terms Java and spring and was indexed at some other date and We can also already evaluate the term case up here to false and If you have an end node and the left-hand side is false or any side is false You can just short circuit everything and it's false So we wouldn't even have to go down into those trees here Because we already know that the result will be false anyways, and we don't have to fetch our full text of the website Okay, so we freed our Boolean I get brass with this I think this is a very good example of an underused free structure that you can use to build very nice DSLs for some problems and You can do partial evaluation on it. You can do a whole program optimizations, which are very funny You can serialize the AST because there are no function inside of it So you can serialize it to Jason with Cersei basically boilerplate free and just use the lies it on D zero lies it on the other end What's even better is that free bull actually has a traverse instance and our partial evaluator is just a traverse with an either and You can even do even more stuff with that But actually that's probably the content of a different talk So here are some resources. This is the blog post by crestucula that inspired the free Boolean part you can find the source code on my github and Otherwise, I hope that now you have feel inspired to go forth and free all the things. That's all I have. Thank you