 So, before lunch we saw some interesting talks on axioms and compilers and APLs and stuff like that. Let us get to the ground a bit and talk about some stuff on the JVM, seemingly much more boring platform so to speak. Myself Debashish Ghosh, I work for LightBend and I have been attending this conference functional almost every year except one year I guess and I enjoy speaking here. So, thanks for inviting me once again. The subject of my talk is managing effects in domain models. When we are building domain models which are not that trivial it is not only composition of some pure functions there are other things to take care of. For example, I O we need to interact with the databases we need to interact with the file system we need to write to sockets and things like that. The question is how do we take care of that in a purely functional world? We want to have all the goodness of functional programming as part of our model and yet we need to address the effects, address the side effects that will come in our way while implementing the domain model. So, this talk gives you a few options, discusses a few design paradigms on the JVM using a statically type programming language I will use Scala for this. Just to show off friends how many of you were there in the in my earlier talk in the tutorial for the JVM for Scala quite a few and some of the slides in this talk will be a repetition from the earlier one, but I will skip through them fast because they first of all they speak about basic things which have already been discussed many times in this conference so far because I have quite a few slides to discuss and I will keep them for the last ones. So, as I was talking we all talk about the goodness of functional programming, programming with pure values immutability, how functions are just like mathematics and things like that equational reasoning, but how do we handle this stuff which are not not so good means which are not so pure not so referentially transparent. For example, file IO, database reads and things like that we call them side effects. How do we manage side effects so that they can be good inhabitants to the algebra of our domain model. How many of you were there in Tony's talk yesterday when he talked about parametricity quite a few. So, one of the central themes of purity in functional programming is this concept called referential transparency. I will not go into the details of it just to mention that an an expression is said to be referentially transparent when you can replace the expression by its value within the program and and we do not have any other impact as part of the program the program remains the same. So, referential transparency is one of the values which will which will be core to the discussion today and the basic idea is how to use side effects how to transform side effects into values which are referentially transparent. This is a simple example of a referentially transparent expression we have this val str we create a tuple using str and instead of str we use its value and the values remain the same. So, it is a purely referentially transparent expression. Now, what about this there is an iterator right which has a shared mutable state and the moment we substitute a we the value the call to the function the values change the results change and that is because of the underlying shared mutable state which is there as part of the iterator. So, this is an example of an expression which is not referentially transparent. So, referential transparency enables local reasoning what exactly do you mean by this because when we are doing when we have multiple sub expressions and each each individual expression if it does not have any dependency on any external services or amongst each other then we can reason about each of these expressions individually. So, that is what we mean by local reasoning. So, if each expression depends only upon the input and the output without any without any interaction with the external world in that case we can reason about those expressions locally. So, that is local reasoning and if we have referentially transparent expression then we can then we have local reasoning and local reasoning leads to function composition. We can compose functions when they are referentially transparent because side effects do not compose this takes us to this concept of side effects which is something which breaks referentially which breaks referential transparency. So, an expression can either be referentially transparent or it is a side effect there is no in between side effects do not compose they are not modular and here are some of the examples shared mutable state database writes reading from standard input these are all concepts you are familiar with. The problem is side effects are ubiquitous side effects you will encounter in almost all parts of the application unless it is a very trivial application you cannot avoid side effects. Now, the question is how do we deal with side effects in functional programming they break modularity right. Hence, we need to handle them somehow within the realms of our type system we are using a statically type language we have the power of the type system. So, we want to use the power of our type system so that we can still work with the pure values till the time we have to interact with the runtime system. So, we have to have to handle side effects such that they remain pure values till the time we submit our program to the runtime system. Fp is all about pure expressions that is why functional programming is often called expression oriented programming as opposed to statements and actions in imperative programming which tend to mutate things. So, one way to handle actions and statements is to differ their execution we need to be lazy we will not execute the action at that point in time what we will do is we will lift all actions into a containers you can think of it as a container as a container sort of thing where we prepare the recipe of how the execution will take place. Instead of executing the whole thing we are still preparing a value right by accumulating stuff within a container that we are we are preparing the recipe of how to do the execution later in the stage of the program. So, this is one option we can handle. So, and as long as we have an unevaluated action it is still a value. So, we are still in the land of referential transparency and it is this value that we are calling an effect. Here is an example we have a bunch of functions all of these are pure and we can compose them together to build a larger function in the as we can see here we can compose these functions to build a larger abstraction. We could do this because these functions are referentially transparent and the types also align we cannot compose just any two functions the types have to align the type algebras have to align and here the types align and because of that we are able to compose the functions and build a larger function. The challenge is to do the same thing with the same semantics of algebraic compositionality in the presence of side effects. How far can we go? How far can we stretch this paradigm of compositionality in the presence of side effects and one of the tools in our toolbox is the type system the types. Now, as we saw earlier also types are algebraic and one of the things which we can do with anything algebraic is composition. So, we can compose functions when the algebra of the types align. Here is an example of a function which is side effect. It computes the salary of an employee for a specific month and as part of it it does a look up for from the database it reads information related to employee and then it computes the salary. So, it is a side effective function that is not what we want. We would rather have it this way we divide the entire execution into two parts. Step one is building up of the recipe of how to execute and step two is the actual execution. Step one is completely referentially transparent. Step one builds the steps, builds the recipe of how I should compute the salary taking input from the database and in step two when I do an unsafe run sync I actually submit it to the runtime system and then the execution takes place. So, till the time we are at the end of step one we are still in the land of values we are referentially transparent. So, we can compose this recipe with some other recipe for example say we have some other recipe for computing tax. We can compose these two recipes together to build a larger recipe which we could not do if we had executed the function earlier and got the value of the salary. So, this way by doing it in two steps by being lazy with the execution what we are achieving is we are achieving compositionality. So, this is the intuition as part of the first step we create a description of the computation and in the next step we submit it to the runtime for execution. Step one is referentially transparent step two is not. So, here if you notice what we did is we put the recipe into a container like thing called IO think of it as a container the container contains the recipe and we call such structures IO a type constructor because it is a valid type it is a concrete type when we supply an argument to the type. So, abstract side effects into type constructors that is what we were doing and these type constructors are what we are calling the effects. We have a large number of effects these are not all these are some of them IO reads from the reads from the external sources either is the effect of disjunctionality option is the effect of partiality we can have value or not have value that we can we can abstract into the effect of partiality. For state management we have the state thing and if you look closely into each of them they have a common shape it is the f a where a is the value a is the answer that the effect computes a is the value that we get ultimately for example, there we had IO salary. So, salary is the thing which we ultimately get out of this and f a is the f is the computation or the effect which helps us doing that. Yeah the execution the execution needs to be deferred till the time we need to compose till that time we need to compose values. Now the f which you saw f is an opaque type we have not yet associated any denotation to this effect to this f type which we will do later in this talk primarily I will focus on the IO effect which is related to interacting with the with external systems and IO is possibly one of the most common effects that we need to handle when we when we are implementing a domain model. So, let us have some let us take a look at some code this is an algebra of one of the artifacts of a domain model an account repository the purpose of this algebra is to it gives you the contract to fetch stuff from the database or write read and write basically interact with the database it gives you an abstraction this repository term comes from domain driven design concepts and account repository is one of those repositories. Now, I have these methods called query and store the return type is IO, IO option account IO account. So, it is effectful right. So, these functions we call them effectful, but effectful, but monomorphic those of you who were with in in Tony's talk yesterday must have seen this that it violates parametricity whether to use IO as the effect type or something else is a decision which the end user should take and here we have taken the decision on behalf of the user. So, let us turn into turn that into parametric type we are now parameterizing the effect type as part of the contract of the algebra. So, now account repository is parameterized on the effect type which is a type constructor m and now the functions return the effect type as m we will see how this helps us, but before that a few notes on parametricity as we saw earlier when we makes make an abstraction parametric it is more generalized. So, it is less constrained. So, it has a lesser number of implementations. So, it is easier to reason about lesser number of possible implementations because it is more generalized. If you have a if you have a contract with type m you really do not know what m is. So, you cannot really write much as part of the implementation, polymorphic because we can vary m we can we can choose various values of m. So, it is parametric polymorphism and theorems for free as we saw earlier also in earlier talks that if you make your contract parametric you can deduce a number of theorems just by the look of the types which your functions take. There is this paper is a great read theorems for free by Phil Weidler. Here is an example suppose you need to write a function which the purpose the implementation of the function deals with folding some stuff. So, a common usage which we which we tend to do is pass it as a list pass the argument as a list, but if instead of list we pass we pass a foldable foldable makes lesser number of assumptions foldable is a much more generalized data structure than list it has a much more generalized contract. So, the number of possible implementations using a foldable will be much less than what you would have with list so, if only if the only only stuff that you are doing is folding over folding over a collection or folding some stuff in that case it makes sense to pass foldable as the argument type rather than list back to our account repository. Now, this is the contract for an implementation of that algebra note that this is also parametric on m, but here what we have done is we have we are still generic on m, but I have constrained the genericity to include the restriction that m needs to have a monad. I would I would like to handle error through the class monad error through the type monad error which implies that m is a monad this is from this is from cats library. So, note that the implementation 2 is implementation 2 does not depend on the specific type of m implementation is also generic and the immediate benefit which we will we will have is we can now implement we can now instantiate account repository using various types of effect types with IO or with monixival task. So, here we have this polymorphic effect of m since we did not commit to an initial implementation. So, because of that we can use various types of concrete effects in order to instantiate the various various types of repository implementations. Any question till till now local reasoning local reasoning local reasoning is a general concept which can be applied at various level suppose you have a complicated function and this function computes computes two different things by calling two different functions. So, if these functions are referentially transparent then it is easier to understand those functions by considering them reasoning about them locally. You can just consider function 1 and you can analyze that it does not depend on function 2 or it depends on any external sources. So, since it is by referential transparency we mean that it it only takes some input and gives some output it does not depend the output only depends on the input which you give to the function local means the local scope the local scope of the function yeah local scope of the function yes. Composibility is possible, but polymorphism you lose by making things parametric upfront you are leaving some of the decisions to be made by the user which actually should be made by the user. So, here are some takeaways we have a parametric and effectful algebra and an implementation parametric algebra with effectful functions, referentially transparent implementation where effect is constrained to a monad can be used polymorphically with multiple specialization of the effect type IO and task we saw and these types of design tend to be modular they can be tested in isolation because they are pure values referentially transparent and things like that and since it depends on some specific algebra for example, the algebra of monad the implementation depends on the algebra of monad you can during testing you can you can pass on a very simple version of the monad to test for example, in some cases if your function takes a monad you can pass the identity monad for testing whereas in production it will be a it can be a different kind of a monad. So, it is unit testable because it is modular. So, we saw one artifact in one artifact of our domain model repository, but usually artifacts do not stand in isolation they have to be composed right they have to be for example, repositories need to be passed on to the services domain services. So, now we will see how repositories can be passed on effectful passed on in an effectful way to a domain service. So, that the effect remains the same the service will also be effectful right if the and the repository is also effectful. So, in that case we need to we can pass the repository to a domain service using the same effect and there also we will use the power of the type system. So, this we call the composition of effects we will compose the algebras of the repository and the and the domain service and this composition will be through types. So, the next step will be to design a domain service that composes with the account repository algebra which means the same effect is propagated down from the repository to the service module. Just a refresher this is how function composition works f a to b g b to c and we can have we can compose this to get a to c and we have the identity functions and if yeah now compare this with this diagram which is basically the same only thing is that the functions are effectful. So, instead of a to b we have a to f b and b to f c and now we need one and then function this is not plane composition. So, plane function composition will not work here because we have the effect and those pure functions what they do is they take pure values and lift into the context of the effect there is one class called Klysely. Klysely basically wraps a function application it takes it it takes a function it takes a to f b and gives us a wrapper and Klysely actually implements an effectful and then by which you can implement function composition over effect full function. Now, this implementation is not complete, but as you can see that here we have let us try to guess the implementation here I have the run which takes an a and gives me an f b and I need to implement and then which takes a b to f c and I have the run there. So, what I can do I can prepare a Klysely my intention is to prepare a to f c which is Klysely f ac. So, I can prepare a Klysely which takes an a and on that a I can invoke run. So, that will give me an f b right. So, now I have an f b and an argument and and then has an argument b to f c. So, now I have f b and b to f c and I need to get f c any idea how how how I can get there any familiarity flat map we can constrain f to be a monide and if we invoke flat map on that it will give me the desired type algebra. So, this is the simple implementation of effectful function composition in a statically type setting where just by aligning types and constraining some algebras to some specific effect types. Here we have constrained f to the effect type of a monide I can implement effectful composition effectful function composition inverse not only not only f b I I have a to f b and also I have this b to f c. So, if you look at the signature function signature of a flat map you will see that the types align beautifully to give you f c having an f b and an a and a and a b to f c I can reach f c. Klysely also gives us a monide. So, that Klysely you can compose using for comprehension and in monads. So, now let us look at our domain service algebra domain service needs to have a repository and. So, the domain the algebra of the domain service states that it takes a bunch of arguments it takes an account repository and gives us an effectful account right. The first thing to note is that we have the same effect types and since we specified m in both the places this means the type system will enforce me to pass the same effect type when I am instantiating the account service and when I am instantiating the account repository. So, this is ensured by the type system and the return type is nothing but a Klysely. Now, the good part is even with the implementation of a service I have not yet committed to any specific effect type it is still m it is still parameterized on m with the additional constraint that m needs to be a monad and because m is a monad I have monad error I want to do error handling using monad error which makes me think of m as a monad I can use flat map which again comes from the monad and I can use raise error which again comes from monad error. So, besides the constraint that I have imposed on the effect type I am not using any other concrete information regarding the type of the effect because I do not have any. So, this entire thing is from cats effect IO cats effect. So, is this part clear that the implementation is also parametric in the effect type and we have not yet committed to any implementation and most most important of all we are still in the value land all these API's all these methods return Klysely which is a value because we have not yet executed anything. So, what our model achieves it is a parametric and effectful algebra of the service that composes with the effect of the repository and implementation that uses the constraints of the algebra of the effect type and it is still independent of any concrete effect type as I was telling you. Monadic with Klysely errors for effectful function composition we used Klysely for composition and service methods composed through flat map because Klysely gives us a monad. So, it is not related to any concrete type it is not once again a repetition of what I told earlier and Klysely lifts your computation into an effect. So, this is an example of a sample program that uses the account service. Now, there are several things to note here this is the final program which we have written. So, here we need to give a concrete instance of the effect I am using Katz effect IO here. Now, using this for comprehension this looks like an imperative program right do this first open credit debit in sequence right it looks like an imperative program, but in reality it is imperative only to the extent that it creates the recipe when you when you run this when you run this for comprehension it will not execute anything it still is creating the recipe and any guess what the type of rest will be Klysely and Klysely takes a repository like in our case Klysely takes an account repository and the moment we supply that repository Klysely gives me back remember Klysely f a c a to f c. So, a was repository when I substituted a with account repository I get back f c and f in my case is IO. So, the result of this will be an IO which is still a value by the way we have not yet executed the program. So, we are we have written the entire program we have instantiated everything and still we are getting a value which is referentially transparent. Now, this single step unsafe run sync this is the side effective part, but this is when we submit the program to the run time system till that time I have the value and in fact when I have this IO I can composite with other IO as well by getting some larger values I may I may want to compose this IO this account IO with some other component which aligns with the type algebra to gives me a more meaningful abstraction more bigger abstraction larger abstraction. So, what does this computation look like? So, IO as an abstraction it has not yet been realized it is still a value and hence referentially transparent as I was telling. And this is a very important paradigm of design which I try to follow it separates the execution of the computation from the specification. This is not true for future scala future future is eager by default, but this laziness of evaluation is what gives us the power of composition. So, if there is one thing you would like to take away from this talk it is this separation of the two concerns separation of the execution from the recipe IO is a monad and hence it allows composition through flat map. Now IO lets you do some other things also IO lets you defer the choice of execution strategy. Once you have instantiated your IO now typically how we want to run the program whether we run to whether we want to run it parallely or sequentially concurrently we take we tend to take this decision up front, but with IO we have not yet committed to any specific runtime strategy execution strategy that you can do you can either run it using unsafe run sync in which case it will run the encapsulated effect as an sequential impure side effect or you can choose to run it asynchronously and this decision you are taking at this point in time when you have built the complete abstraction you have. So, the abstraction is also independent of the strategy of execution you can do unsafe run a sync or even you can compile it to a future scala future. So, effects are algebraic IO is an algebra modules compose when algebras compose. So, here we have composed the two modules the repository and the services we have composed the two and come up with a larger abstraction how am I doing with the time how much 11 minutes. So, effectful designs are referentially transparent till you submit to the runtime system ok I need to go a bit fast, but domain services are not alone right we can have multiple domain services which may need to be composed together. So, we have seen account service, but in reality we may need to we may have say we can have some other service which does reporting reporting service and your final program may need to use both these services right. So, in that case if you adhere to the algebra of the effect which we stated earlier in that case you can actually mix all the services you can compose all the services here balance by account this function comes from the second service. So, all of them since all of them honour the compositionality of a monad in that case we can mix and match all of them and we can design a larger service a larger program to execute. And once again another side effect of this polymorphism thing is that this entire program is written in written using cat's effect I o if I want to switch to monix eval task I just need to change I o to task the entire program remains same. So, what we have done so far is we have defined the algebra with parametric effect type we have implement implemented with constraints of effect type being monadic or applicative it could be applicative also we can compose with other algebras. So, long as they honour the type constraints the type algebras and do the specialization as late in life cycle as possible. Now the next few slides we will talk about slightly advanced stuff this was the signature of our account service open method right we return the closely M was our effect type, but it may so happen that in case of M you can have a much complicated effect type which is a stack of monad transformers a monad transformer stack. Because the effect types all of the effect types may not be same in that in which case you may need to build a monad transformer stack. Now monad transformers are not performant on the JVM they do lots of heap allocations because of which performance can degrade. There is this excellent blog post by John Degas where he explains the various issues with transformer stacks deeply nested transformer stacks. The question is can we do better the answer is yes type classes instead of using the monad transformer stack for each of the monad if we can have a type class then instead of using the stack we can use the smaller grain type classes and have the same kind of effect without getting into that big stack. So there is a library monad transformer library which it is there in CATS it is there in Scala Z also. So this library gives us these type classes. So we can use these type classes instead of using monad transformer stacks and we can supply hand rolled instances of the type classes. So this will have a remarkable effect on the response. There will not be as many heap allocations as with the transformer stacks and using MTL is one of the recommended ways to develop such applications on the JVM these days. There will be some changes in the code for example applicative ask applicative ask is the type class corresponding to the reader monad or the Clisely. So here we have got gotten rid of the Clisely thing we are using a specialized type class application ask applicative ask and using the ask method of applicative task applicative ask we can query for the repo the account repository. This is a simplified version of applicative ask. It gives you this ask method and if you look at these two algebras you should see a similarity the effect type is parameterized in both the cases. In this case in account service it is M and in applicative ask it is F. So the effect type is parameterized for the user to instantiate. So this type of design we call the tagless final. There are lots of papers on tagless final approach and how it how it leads to better composition of composition of algebras. So you can take a look at those but yeah tagless final and MTL these two are the potent combinations for developing effect based generic applications at least in Scala on the JVM. So any question till now I am almost done. So here at the final take away parametricity is a virtue try to use it as much as you can. Type constructors for modeling effects compose with values. Once again I reiterate separation between the specification of an abstraction which is pure and the execution by the runtime system and use these above to wear the domain artifacts. The algebras need to align and so on. Thread effects to domain behaviors using the selected denotation. Here the example was using monads you can use some other as well. Avoid monad transformers if you are on the JVM. Use type classes for smaller brain effects. Roll out hand rolled instances of type classes. It may seem daunting but once you are into it you will see that it is not at all difficult. Those type classes like applicative ask are really simple to instantiate. So provide your hand rolled instances of those type classes so that the JVM does not have to do much to fetch the proper instance. No it is not monad transformers. It is just the instantiation of the type class. The instance of type class for your specific data type. For example in our case we are injecting the repository. So applicative ask we need to implement an instance of applicative ask for account repository. It is actually 5-6 lines of code. It is not much. In case you are interested in this repo I have a complete working implementation of this same example using all the options which we discussed. Monomorphic IO, MTL and Tagless Final. So all of these implementations are there so you can take a look at it. It is actually running code. Actually the main confusion with MTL is the name. Monad transformer it is part of the name but it does not have any name. The name is possibly because of some historical reasons. Yes, stacking all of them together leads to lots of heap allocations and things like that. So you can get rid of that using MTL. And last of all be lawful. If you are in George Wilson's talk yesterday then he discussed about the laws. So for all of these type classes which you write, write your own laws. So in that case you can verify those laws as well. And I would like to end with this quotation from Rob Norris. In one of the conferences he mentioned this effects and side effects are not the same thing. Effects are good. Side effects are bugs. That is a very strong statement. Their lexical similarity is really unfortunate because people often conflict the two ideas. And in fact Rob Norris has a couple of videos that he put to you on effectful design. So watch them. They are always recommended. And in case you are interested more in this on this subject I have written a book which came out a couple of years back. So this also discusses some of these things. Indian edition, I don't think so. I am not sure. Yeah, I don't think it's available. A lot of functional constructs. Actually if the language supports functions as first class artifacts, first class citizens then you can go to a lot of extent implementing these things. But I prefer always typed language. So unless it's strongly typed I am not very sure. Language provides for manipulating types in addition to, you know. Yeah, if it has a decent type system in that case I think it's flexible enough. And at least two of the languages, Kala and Haskell I have implemented similar stuff in both of them. So you talked about separating the recipe from the cooking. Where you extract the recipe into a value. So now when you get to testing the recipe, I mean here's where difficulty with intuition because I can't, I have a recipe in memory but I can't run a search statement to see if the recipe is correct. I have to actually run it through a monad to kind of see if the recipe is working correctly. Yeah, actually that's what I was trying to say that if you have the recipe since it's a value you can always test it in isolation. For example, you have a monad. Replace it with ID, identity monad. Right, right. But what I meant was unlike the typical types with value and here when I say value I mean simple types like ints and strings. I can just run a search statements. But one of the difficulty that comes up here is you can no longer just do assert equality checks. You actually have to run it through a... Yeah, you can do asserts but you need to replace those effects, replace those effect types by simpler ones. Simpler ones. Okay. Thank you. I think we've run a lot of time so. Okay, so let's get around. So in case you have any further questions.