 Okay, so let's start. Thanks for coming to my session. Before beginning the presentation, let me get a show of hands of how many of you have used design patterns ever in your life in programming languages? And how many of you have used design patterns in a functional programming language? Any difference you felt using a design pattern in a functional programming language and using a language like Java or C++, what kind of difference you felt? So any thoughts on how you can get that beautiful structure? Any theory, any theoretical underpinnings that make composition much more beautiful when using a functional programming language? Yeah, so it's basically we have some underpinning of mathematics, right? Category theory and things like that kicking and you have a mathematical definition of function composition. Okay, this is going to be a discussion about design patterns, but I promise that it's not, it's going to be very different from the Gang of Four design patterns that we have learned to use and loved in Java. The discussion that I will start doesn't have any serious mathematics or advanced category theory, but it's about developing an insight that lets you identify code structures that you may think can be improved with a beautiful transformation of an algebraic pattern. The basic underpinning will be algebra and we will see how to encode functional design patterns using the concepts from algebra. In the earlier days of Java coding, we used to feel proud when we could locate a piece of code which could be transformed into an abstract factory pattern and we felt proud when the factory beam could be injected using spring dependency injection. The result was that we ended up maintaining not only Java code but also XML and that too untyped and unsafe. This was dependency injection pattern in the earlier days in full glory. In this session we will discuss patterns that don't look like acceptable artifacts. They are very much part of the language itself. They have some mathematical foundations in the sense that they have an algebra that actually composes and composes organic abstractions. We will see plenty of examples of how these patterns can make your code beautiful and algebraic. So here is the plan today. We start with the formal definition of a design pattern and establish a formalism in the pattern which we are going to use for this discussion. And as you will see that this formalism is based on algebra. In fact, many of the functional patterns which we will discuss are plain encodings of the algebra itself. And we will look at some Scala code. Not necessarily imperative. They are also functional code but for which we will still try to reduce code duplication and using patterns we will try to raise the level of abstraction of the code which we are going to see. The code we will see is mostly Scala code. But the principles which we will discuss apply equally well to any statically type functional programming language. For example, Haskell. We start with the definition of a design pattern. This is the classical definition of a design pattern solution to a problem in context. And anyone can identify this gentleman in the picture is Christopher Alexander. Christopher Alexander is the person who was responsible for first using the term design pattern in the context of architecture. Not software architecture. Architecture as in buildings and bridges. And Alexander was the main catalyst because of whom? Design pattern, the term design pattern has become mainstream in software as well. So we have a solution. We have a problem and we have a context. The problem is given to us. We don't have any choice. We are given a problem for which we need to find the solution. And the solution itself which we are going to implement usually has two components. The first part is the generic component which is invariant across the context of the application. This is the part. This is the part of the design pattern which we came to be. The reusable part across all contexts. This will be the algebra itself as we will see. And the other part of the solution is context dependent which we need to implement separately for every context of the problem. Now I will claim here that this aspect of reusability that we talk about in design patterns has a parallel in the world of mathematics as well. And let's keep this in mind while we traverse through the aisles of a similar structure from the world of mathematics. Algebra. This is the definition of an algebraic struct from Wikipedia. The point to note here is that the definition of an algebraic structure is completely generic. The definition doesn't specify any concrete type. We are not talking about an algebraic structure for integers or for any other data type. Algebraic structure is defined generically. And every algebraic structure can be refined by adding more and more constraints on the set of components. For example, this mentions about operations. We can constrain some of the operations to refine our existing algebra. As for example, we can define the algebra of sets like this. We have a set A. We are given a set A and we define a binary operation on the set which we can express as Phi AB or A Phi B. Once again there's nothing concrete about this algebra. We are talking about a set, set of elements of type A. We haven't yet qualified A. The moment we qualify A with a specific type, it becomes an instance of the algebra. For example, this is one specific instance of the algebra where it deals with the set of integers. When dealing with set of integers we can define addition as a binary of operation. And as you know, that addition of integers is associative, which is an additional constraint, additional refinement that we are tacking on on top of the existing algebra. It was not there as part of our original definition of set A. That's an additional constraint which we are imposing on the algebra. Addition of integers also has an identity operation. You add 0 to anything and the number remains the same. So let's enhance our original algebra with some of these constraints. We bring in this associativity constraint here. So on top of our original algebra we say that the operation that we define, the binary operation which we define has to be associative. Anyone has any idea what this algebra can be called in generic terms? What is this algebra of? This is the algebra of semi-groups. We define a semi-group as having a set and an associative binary operation. We just change the original algebra by imposing an additional constraint. We can further refine this algebra. We add the identity operation and this algebra becomes the algebra of monoids. Now the question is how do we encode this algebra in our code? It's as simple as writing the three lines of code. We have a class monoid. This is Haskell. We have a class monoid. Note that once again this is not specialized for any concrete type. We say monoid A. It's for any generic type A and we define the two operations, memt and map n. Memt encodes the identity operation and map n encodes the binary operation. But still it doesn't have the laws which we set, the constraints, associativity and identity. We can encode them in terms of some laws. In Scala typically a monoid is defined like this. We have the semi-group and we define the algebra of semi-group to get the algebra of monoid. Note it's parametric in the type. Here we have the algebra of monoid which is the basic interface. It's reusable, polymorphic and usually it comes as part of the standard library. In Scala the Scala standard library which comes default it doesn't have any monoid, but you can use some of the standard libraries like Scala Z or CACS which gives you the definition of these algebra. And we have the instances of these algebra as implementations. The first one is the instance of monoid for integers where we define where we can define the operation of addition of integers. And the second one can be a monoid instance for any of your application data types. For example in an application you can have a data type called money which encodes a money, an instance of money. So you can have domain specific instances, but both of them, both of these instances inherit the generic contract from the basic algebra of monoids. So as we go down from the algebra to the implementations, we are going down from generic to specific. Specific implementations which we saw, they use the generic protocol or interface and this reusability is enforced by parametricity. We didn't have any type, specific types as part of the algebra, but we had them as part of the implementation because we are implementing the semantics of the specific data type in order to define the identity and the combine operations. And this genericity implies reusability. The more generic we are, more reusable the abstraction is. The moment we go down the hierarchy we lose on the genericity and we get the specific data type for the application. This is the summary. We have the pattern. The top part gives you the pattern which is the reusable part, generic and reusable. And the bottom part gives you the context specific implementation. So as part of the solution, as we define in the formalism of the design pattern, the top part is the completely reusable part which can be reused across all contexts, across all implementations of a monoid. So how do we define functional patterns? Functional patterns are defined by means of a generic reusable algebra. It has to be parametric on types. The moment you add in a specific type as part of your algebra definition, you lose on genericity. And there is a clear separation between the pattern and its instances. The pattern is the algebra part and the instances are the implementations. And as we will see, these patterns are composable through composition, function composition. Why do we need to use design patterns or functional patterns? One of the premises of the original design patterns, even in the Ganga for book was that it gives you a standard vocabulary. The moment you say factory to a person, she understands what exactly do we mean. It's meant to create some instances. Similarly with functional patterns, we have the standard vocabulary. The moment you say monoid or monoid or applicative to a person who understands the terms, he or she realizes that what to expect out of the algebra, what contract it gives you. So it establishes a standard vocabulary amongst the users. There's a rich ecosystem support through standard libraries. What we will see is that the standard libraries, besides offering you the definition of the algebra, it also offers you a suite of functions. And those functions are implemented totally with respect to the algebra without any concrete type specific implementation. So that makes those functions reusable across all data types of your application for which you honor the contract. For example, you can have 10 monoid instances in your application as part of your application. So if you have a function in the library which takes a monoid, you can use any of those instances of data types from your application and make use of the function. Also with functional patterns, you get a number of privies. For example, say you have a data type for which you have a monoid instance. If you use the data type as part of the value of a map, then that map becomes a monoid. So map kv, if you have an instance of monoid for v, then map kv becomes a monoid. This is the multiplicative power which I was talking about. These functions are from standard libraries and the contracts and these functions only depend on the algebra. So that you get a multiplicative combinatorial explosion in fact. You can combine your application specific data type with these functions. Now let's look at some concrete code and these are actual code base from real life examples, real life applications. And we will see how we can apply some of the functional patterns in order to raise the abstraction level of these code bases. Now this is a definition of a class called money, which is a frequently used data structure in many financial applications. This money abstraction is defined in terms of a map, which is a map from currency to big decimal. For example, say I can say that one unit of money I have in bank, it consists of 10 US dollars and 1000 Indian rupees. So that can be modeled using this map based data structure. And here we have some code that manipulates some money. Perfectly okay, there is nothing wrong with this code, with this piece of code. But we will look more into this add function. Yeah, we will look at this add function. What exactly are we doing here? If we focus on the how part of how we are adding the money instances, we are basically manipulating two maps. We are fusing two maps. We are adding the, we take one map, add the keys from the other and in case we have, in case the keys are equal, in that case we need to combine some of them, combine them using some additional something like that. But the important part is that it's a concrete implementation of fusing two maps. If we were able to abstract away the fusion part of it, fusion part of two maps into some generic structures, then we could get rid of some boilerplate code here. How can we define a generic function which will fuse two maps, which will add two maps? We will see it in course of this presentation. But for the time being there is a red herring, don't do this. This will lead to an explosion of boilerplates and code duplication across your code structure. Now here's another class. The important class is the payment class, which is the payment made for a particular account. For so and so account we are getting a payment of some amount of money on a specific key. Now look at the last function, new balances. Again this involves fusion of two maps because our payment structure is also, payment structure consists of a map and this, this is in fact a map from an account to money. Previously we saw money itself is based on a map and this is one more map and if we have to add these two maps then we would have to duplicate that code base once again here and do some, some other jugglery in order to combine these two maps. So this is in a way somewhat similar to the problem we saw earlier. Now the idea is that whenever you spot similarities of structure or behavior, squint hard, there may be another abstraction hidden within the concrete implementations. Yeah actually it depends it depends if it's a, if it's a small application it may not make sense to go for, go for higher order abstractions. But if you have a, typically if you have large domain models or large abstractions these are the code bases which I picked from some large applications. So we had hundreds of classes and it was a big application. So one thing which you need to keep in mind is that whenever you are going for abstracting something, judge the cost of it as well. If the cost of an abstraction exceeds the value, in that case don't go for it. If it's a small application then it doesn't make sense to go for big abstractions or make the code unnecessarily complex. But if you see that the result of code duplication or the result of having boiler plates is becoming too much, it becomes a maintenance burden also. In that case go for the higher order abstractions. In fact this logic in new balances is quite complicated and I intentionally highlighted it for the purpose of making the slide concise. Now here are two other functions. One of them is valuation and the other is max payment. What we are doing here is look at the contracts of these two functions. Both of them take a list of payments and return a money. So similar contract is there. Any commonality of behaviors that we can extract, is there any commonality of behavior we can extract from these functions? Both do basically the same thing. Iterate over the collection and compute an aggregate. The aggregate is different. In one case we are adding and in the other case we are finding the maximum. So once again there is a chance that if we squint hard we may find some design pattern from within it and we will try to extract and generalize. For the time being we will avoid doing these things. So we are back to this money adding business. One of the things which this code base does and the first impression that we will have looking at this code base is how the money are being added. Now since we are talking about functional programming and functional abstractions we need to focus first on the what part of the function. What does the function try to do? The function tries to add two money objects and even if we ignore the how part of it when we add two money objects that that addition process has to be associated. A plus B plus C will always always needs to be A plus B and then plus C. And even in our code base we have an identity operation defined here. So this gives us an idea of what pattern we can extract out of it. It's a monoid for money. So this gives us an idea that if we implement a monoid for money then the details of this logic will go as part of the abstraction and we can make our code much more sussing and smaller. So any question till now? First of all I don't think I said that patterns are applicable for statically type programming languages. I mentioned that some of the patterns that we will discuss today makes more sense when we have a statically type programming language. For example the monoids and monoids I have seen them defined also in dynamically type programming languages. But I always felt that these patterns make more sense under the concepts of static typing. Any other question? Now we try to extract the similarity of behaviors and structure out of these code bases. So I have flagged these two behaviors here. If we squint hard we will see that both of them are doing basically the same thing. They are iterating on a collection and finally doing an aggregation. The two different ways to combine a bunch of money instances. The first one is doing an addition and the second one is doing a max operation. So this gives us an indication that we can plug in two instances of monoid for money. For one we define the combined operation as the add and the other one and for the other one we define max as the combined operation. And if we do this then we can extract some of the commonality from this part of the code base. Let's see how. Similarly in this case if we have a monoid instance for money which we thought will be a good idea then this map itself will be a monoid. So both the balances will be monoid. So this is how we compose multiple design patterns. We are composing the very fact that money has a monoid instance. It gives us this thing for free that any map which has money as the value is also a monoid. So in that case we can get away with this complicated logic and model this as a combined operation on the two monos. So once again the basic pattern that we are using is monoid for money. So looking back we had similar problems but under different context we could reuse the same algebra of the monoid. Though we need to define distinct concrete instances of the algebra because we need different modes of aggregation for money. For one case we needed to add and for the other case we need to combine. Let's look at this in more detail. There are some more patterns inside these couple of lines of behavior. Now if we look at both of them as I mentioned that both of them are basically alterations over a list of payments. But if we squint hard you will see that both of them are basically fold. Both the operations are doing fold with different aggregation function. So this is one more pattern which we are which we can find out that both of them are fold. Both of them doesn't aggregate on a binary function which we can define in terms of monoid and they start with a list of payment. From there it extracts a list of money. How? By using this function called credit amount. This function extracts the money part out of the payment. So ultimately the fold is done over a collection of money not a collection of payment. Now if we extract, suppose we are able to extract this behavior out of these functions and have a common function for them. Now we have enough generic patterns inside it to extract it as a common behavior. We have the fold one. Fold is a generic operation. We can give it a monoid on which to aggregate. That's also a generic operation. And we can, the very fact that we are extracting money out of this payment is itself a map operation. So we have a map, we have a fold and we have a monoid. So now we need to combine these two. Now where should this behavior go? This generic behavior that I am talking about what is the data structure under which we should club this behavior. Here we have it in terms of list. Here it's taking list. But these behaviors are they doing anything related to list? List is a much more powerful data structure. We are not talking head of the list or tail of the list. All we need is a much lesser powerful data structure than list. And in software engineering one of the guiding principles which we follow usually is use the least powerful abstraction which applies for your use case. So we don't need list here. So the pattern that we will talk about is foldable. All we need from this collection is the usability to fold over using an aggregation data structure. Now what we have achieved here is abstracting our structure as well as the operation. We are using a foldable. Foldable is a much more generic structure than list. The moment we move over to a generic data structure the usability of our function increases. Foldable is a generic data structure compared to list. Array is also a foldable. Map is also a foldable. So the moment you plug your behavior as part of a lesser powerful data structure the range of usability increases. So here we have foldable and we have a function within it called fold map which is exactly what we plan to do. It takes a monoid and first it does a map and then it does a fold. If we apply this function in our context in the context which we saw we can define a helper function. MapReduce, that's exactly what we were doing. We are doing a map, extracting some data and then we are doing a reduce. Just a question. MapReduce also we have seen in the context of big data and Hadoop and things like that. Do we have any small difference between this mapReduce and the one we use in Hadoop? Combiner, you are talking about combiner. Yeah, combiner is not there but combiner is also optional. Combiner you don't have to have. The important point is that in case of Hadoop mapReduce the operations need to be commutative. Here we don't have this restriction. The operations are not commutative. So we have been able to abstract structure and operation. By using foldable as opposed to list we have made the structure more generic. So this logic no longer applies to list only. It applies to everything that is foldable. And by delegating the logic to combine the two elements to the monoend which is a generic algebra we have been able to abstract over the operation as well. So by defining these 4-5 lines of code we have been able to abstract over the structure as well as the operation. And if you look at the code base now with these changes we have this. All that multiplicative reusability that we were talking about. The standard library gives us the monoid for map implemented using the generic protocol and we can reuse that to our domain specific type provided we honor the constraints of the library. So assuming that mapReduce is a helper method as part of our library and fold map and foldable come from the standard library we have the standard application of mapReduce under both of these contexts. We just need to pass the different aggregation function and different map function if we wish. And the different aggregation function comes from the different instances of the monoid. In one case we are using the order monoid and in the other case we are using the add monoid. I will give the slides up or if you can come forward. Algebraed. Actually Algebraed what it does is it defines some of the data structures as monoids. For example those data structures are typically used in the context of big data and streaming applications like bloom filter, count mean sketch and the basic property which they exploited is that they came to the realization that all of them are monoids because they can be merged together. A count mean sketch can be merged with another count mean sketch. There is a definite algorithm for that. So because of that they defined and they released in their library. I have used that also. They have defined all of them as monoids exactly that. Monoid is a pattern which appears in various context and in various use cases. So in case you are developing a large application there is very little chance that there is not a single instance of monoid in your application. Monoids are almost everywhere. And just to summarize this discussion regarding this map reduce method which we defined the most important part which makes it reusable is parametricity. I have been emphasizing the importance of parametricity since the beginning of this session and there is a saying that the way you abstract unless it is parametric there is no value in that abstraction. If you make your abstraction depend on concrete data types in that case that that is not a good abstraction that loses reusability and in our case this map reduce function is completely parametric on types. It does not depend on any concrete types. It is reusable under multiple implementation context. Any question? This one? No, you need to have separate instances of monoid for A as well as for B. No, the basic algebra the basic definition of monoid is that it is for one specific data type, one data type. There are other algebras for multiple data types but this is for a single data type and the other thing to note is that monoid is parameterized in a specific data type. There are other algebras which are defined as on higher kinds to be honest I have used all kinds of algebras in my application and till date I have not faced any issue with this. Actually the point which we need to keep in mind is that not all parts of application need to be blazing fast. If you feel so my idea is designed for optimized abstractions and if you feel that some part of your application is becoming a bottleneck for performance because of additional abstractions then de-abstract only that part of the application. Because once you have an abstraction you can always de-abstract it but if you have a whole application without any abstraction then it is not a problem. So abstractions have to be designed and have to be thought of early enough in the life cycle of the application development. So I move on to the next topic I don't have much time I will rush through it. That's one more use case handling domain validation this is also from a real life code base the entire code base is littered with these kinds of things and you have some exceptions those exceptions at the end of it it will show the exception. So these kinds of codes first of all they are not pure or they are not referentially transparent and it adds a lot to the boiler plates and code duplication stuff. Another example configuration management we see typically every application needs to read from some configuration and this is an example of a code base where one is trying to read stuff from a configuration file once again littered with try-catch things. Try-catch things are never referentially transparent and in functional programming we usually don't advise to use try-catch blocks. So these are typical anti-patterns we think even when people code using a functional programming language it has repetitions and the other thing we saw in that is that the entire code base is based on imperative statements and not referentially transparent expressions which add value to your code base. Hence you can't compose and because you can't compose you have separate statements being invoked in sequence. It's also littered with exception handling code try-catch which violates referential transparency and it's very difficult to modularize such code bases. So the first approach the first approach is abstract your exceptions with an algebra we need to handle exceptions but not throw it upstream. So here we have defined the function which is encapsulating the exception part because this is often required when you are interfacing with a language which has support for exceptions especially java suppose you are writing in a functional programming language but you need to interface with java and you don't have any control over the code base in java there. So typically java code throws exceptions but in functional code it's always recommended that you try to localize those impact in small places. So here what is being done is that the exception is caught here and what is being returned from the function algebra of disjunction which is the either, either data type the either data type encapsulates on the left the exception and on the right the value. So the first step is abstract your exceptions. Now here we have an algebraic data type for the config part the config part we need to encapsulate right we need to abstract because once we are done reading the configuration we need to return an algebraic data type so that's what is being done here even with this algebra this algebra of either note we are returning either and either is a monad we can write beautiful configuration handling code like this it's a monadic comprehension we are reading reading various stuff from the application using our function read stream and at the end of it we are returning the algebraic data type which we define this is fine this what's fine now one thing we need to notice here is that since we are defining the thing in a functional programming language we have lots of repetitions here the config everywhere we need to in all of those invocations we need to pass the config as a parameter and in functional programming it's one of the things which we try to do is avoid repetitions so is there any way we can avoid repetitions here and avoid passing this config parameter everywhere all across the place we usually try to avoid such repetitions and our friend is algebraic composition this is the function which we had for reading the config thing and this error or is actually a type name for either now what we would like to do is we would like to compose on the return type we don't want to pass config everywhere so we need to take config out of the function signature as a correct parameter previously we were having config as part of the parameter in from config now we make the parameter a bit make the parameter correct this is the thing which we get now our function it's a config it was reading config but now it's doing it as part of the parameter and we want to have a better abstraction for reading this stuff in we are doing we are reading in the config parameter we are reading in the config data type so we can abstract this reading part in the form of a this is what we can do one common way to abstract this config abstract the reading part is by using a reader monad reader monad reader monad and since we already have an either monad there we need to fuse the two using a monad transformer so that's exactly what reader t does here I am running out of time so this is the final shape of our function we have this reader t which is the transformer which fuses the two monads the either and the reader and now our function becomes much cleaner it doesn't have we don't have to repeat the config parameter everywhere and here is the definition of the read string which also gets changed because we need to return a reader and claisely is nothing but a reader so quickly summarizing functional patterns are made useful by reuse of already existing algebra we could successfully able to reuse the reader algebra the monad either etc algebraic composition we used composition to form larger patterns out of smaller ones the map reduce was an example where we were able to compose multiple patterns the abstractions remain composable and modular so by looking into some of the aspects of an existing code base very likely that you will discover many such patterns and you should be able to raise your program to a different level of abstraction but once again before doing any refactoring with abstraction we were of the cost of it as martin mentioned in his keynote today morning there are costs to everything so be sure that that you can justify the cost which you are incurring for abstracting your program question yeah mining means the idea was to take some existing code base and find out some mine some patterns out of it explore and mine some patterns out of it and all of these code bases are real world code bases which are running in applications many of these were written by me also so it's as you learn it comes to your realization that these code bases can be improved so I picked many code bases as well exactly yeah that's actually the sign of progress if tomorrow you can find some mistakes in the code base which you wrote yesterday and that's the sign of progress yeah I think the first step will be to pick up the basics of functional programming and I feel that these kinds of refactoring and these kinds of programs make more sense in a statically type programming language pick one of them possibly haskell because in haskell yeah these patterns become much more explicit because haskell has better type inference than scala so the code becomes less noisy so yeah learn haskell learn haskell interact with people who know these things because there's no substitute for practicing in hands-on hands-on kind of stuff sorry I'll have to interject we need to set up the next speaker I'm sure they will be around so we can catch up