 Welcome everyone to the session on typed functional programming with Saurabh Nanda without any further ado over to you Saurabh. Hi everyone. Just a quick shout out how many of the participants have heard of this language called Dal. So just a quick thing this is actually the Hindi Dal pulses. It's just that some like a foreigner has given this name so it's D-H-A-L. So it was going through a talk and someone was pronouncing it as D-Hall. So it's not D-Hall. It's not D-Hall. It's not Dal. It's Dal, Dal Roti Dal. So that's what it is. I don't know why it's named that but whatever. So yeah, so essentially the premise that I'm going to walk everyone through is Dal is probably a good stepping stone towards learning type functional programming. And mostly the kind of programming that is promoted by Haskell, the kind of ergonomics, the kind of type system, the kind of purity that Haskell has. There's a big overlap between Dal and what Haskell does and Haskell I think is pretty much the flag bearer of type functional programming. It's the purest form. It's the most unforgiving form. It has got no escape hatches. So that's the premise that I'll be walking everyone through that Dal is probably a good gateway drug towards type functional programming. So yeah, this is like something which I've learned firsthand while learning Haskell myself. Again, I'm going to be using FP and Haskell pretty interchangeably because that's what I'm talking about, type functional programming as embodied by Haskell. So doing FP the right way is hard. Right. I've learned this firsthand while learning has to be myself and I have witnessed this second hand while mentoring and training other members in various teams to pick up FP. Right. So in fact, I would talk about this based on all of these learning so much longer talk why Haskell particularly so hard and how to deal with it. There's the link to the talk if anyone's interested, they can go and revisit that the problem with learning FP the right way is that apart from Haskell and probably say a pure script. Most FP languages have too many escape hatches right yesterday we had a talk great talk from by Luca from who has there they took a similar decision they decided not to use type script but instead go with Elm because TypeScript had too many escape hatches and was trying to retrofit a functional programming paradigm on top of JavaScript. Right. So that's that's the problem. It is too many escape hatches and what what ends up happening is if all your FP features are essentially opt in. You like people who've done imperative or programming for the last five years six years seven years. They tend to use their muscle memory and that muscle memory is tuned towards open imperative program. So they end up adopting a new syntax, but they're essentially still doing or imperative whatever you want to call it they're not doing FP. So like a firsthand example very recent that I've gone through is Kotlin. I tried sort of migrating a team from Java to Kotlin it was an Android team. So, all of them learn the Kotlin syntax, but it did. You know, I have to keep reminding them to not use mutable variables to not sort of get down to writing for loops to use filters to use maps. You know, like, you're using Kotlin. The whole point of Kotlin is that it has a large number of FP features. But again, coming from six years of Java development and Kotlin having so many escape hatches. They're still writing Java code, but in Kotlin syntax. So, this is the hypothesis that is slowly becoming stronger in my head. And it has been sort of echoed by other panelists, other speakers in this conference itself. So the idea is that no escape hatches is a good thing when learning it. But the flip side of that is that it makes the learning process that much harder. You have to unlearn, you have to build new muscle memory, you have to change the way you're thinking and how hard that process can be is absolutely embodied by Haskell. You have so many firsthand experiences that people have given up learning Haskell after spending about two or three months. So, while having escape hatches makes you feel good, makes you feel like you're making progress, but there's a very high chance that you're no longer doing FP. But now if you pick a language which has zero escape hatches, the learning curve becomes just so steep that it becomes a frustrating experience. So, what's the middle ground? There are some middle ground that I have found, there are Haskell inspired languages, and they seem to be a good first step, a good gateway drug towards learning full-fledged type of functional programming. So, these kind of languages, they have a general learning curve. There is a lot of concept overlap between Haskell and these languages. There is a similar lack of escape hatches, but they have better tool, they have better compiler errors. So, the developer experiences is better. So, and most of the times, these are special purpose languages, and they say that this language is to solve this problem, and they solve the developer experience for that problem, and they avoid the complex parts of the general purpose Haskell. And there are a lot of complex parts in general purpose Haskell. One example is Elm. Again, two talks that I personally attended in this conference, which were about Elm, and they sort of accord this understanding of Elm being a gentler, a more approachable stepping stone towards full-fledged type of functional programming. So, Elm is a production-grade language for building front-end apps. No writing is using it. I learned Rakuten is also using it. And it is heavily inspired by Haskell, a lot of overlap in concepts, and the compiler itself is written in Haskell. Similarly, now, the other language that I've discovered is Dal. Now, Dal is a specialized language. Someone is on the chat. Dal is an embedded DSL for config management. Absolutely right. It's a specialized language for managing large and complex configurations. It's heavily inspired by Haskell and Nix. So, Nix is like a whole new ballgame for package management and managing OS dependencies, which is highly functional. And Nix is also very purist in nature. So, it's taking ideas from Haskell and Nix because it's at the intersection of functional programming ideas from Haskell and it's picking package management, configuration management ideas from Nix. Again, the compiler is written completely in Haskell. So, irrespective of the learning sort of like the learnability of Dal, the fact that it has a gentler learning curve, why is Dal worth learning otherwise? So, Dal is not a toy language. It's not a learning language. It's not like how most people use scheme. They pick up scheme when they're going through the SICP book, but they generally don't tend to use scheme in production in their day-to-day work. But Dal is not like that. And why is it worth learning? What problems does it solve? So, when I first discovered Dal, this was the question in my head as well. Like, okay, complex. It's like, how complex and complicated finds me? Why would we need a special language or tool for that? And then I came across two quantities. So, Dal itself seemed overkill to me till a few years ago. In fact, that underline is a link to an issue on the Dal GitHub tracker where I'm asking this question itself, that what is the rationale for Dal? And then I immediately understood it while when I started writing Kubernetes configuration types. So, don't get me wrong. Dal is not only for Kubernetes. Dal is a general configuration language. It's not specific to Kubernetes. But the thing is that just dealing with Kubernetes, which is becoming so commonplace nowadays, dealing with the configuration explosion of Kubernetes makes you immediately realize the power and use case of that. So, again, it is not only for Kubernetes. Big caps, bold, whatever. You should not walk away from this talk saying, oh, I will use Dal only when I'm using Kubernetes. No. But that's the thing that makes you immediately realize the power of Dal. But if you're in a mid-sized to large-sized company where you moved all your infrastructure to cloud, you are doing infrastructure as code, even without Kubernetes, you're probably dealing with a configuration explosion. You don't realize it. But there is better tooling, there is better developer experience available for managing these conflict files, which are critical for our deployment space. So, what are these problems with these conflict files? So, the second half of the talk is about Dal versus Haskell. Like, we go through a comparative code between the two and see how they are similar, what concepts overlap between them. But before that, I just want to build the case as to why Dal is... The problem that Dal is solving is a genuine problem and you should probably have it in your tool bet, in your tool gem, right? So, the first problem is you have different conflict files which don't agree with each other. For example, there is application A and application B. Application A is writing to a topic or a queue. Application B is reading or consuming items from that topic or queue. Both of them have configuration files and both of them should be subbing from the same topic or queue. So, two different conflict files, if they don't have the exact same topic name in them, your application won't work well in production. Another example, your application, your container, your load balancer, all of them should be using the same port. So, your application should be opening a socket on that port, your container should be exposing that port. Your load balancer ingress should be using the same port as the backend sort of service, right? So, okay, I think the talk... I just have this chat open in front of me. I think the talk should have started here with the problem. Cool. Let me just... whatever. 5 minutes into 10 minutes into the talk. But anyways, so that's one problem. The second problem is repetition within the same conflict file, right? So, again, taking an example from Kubernetes, there's a lot of repetition across standard. You need to make sure that you have the name, namespace, key, selector, label, et cetera, et cetera, is the same across multiple places in the same conflict file. If you don't have it, most probably your POOP CTL apply command will work happily. It will accept it. But then you will end up with a service down payment. Then you will spend hours debugging where you forgot to copy paste the exact same name, namespace, key, selector, label, et cetera, et cetera. So, repetition within the same conflict file. Then comes duplication between conflict files for different environments. So, at a lot of places we've seen that there is like a 100-200 line conflict file for dev, 100-200 line conflict file for staging, 100-200 line conflict file for production. Now, there's more than 70% duplication between these files. These are physically separate files. Even if the values are different, which most probably they are, then the keys need to be exactly the same. And what happens if you add a key in the dev conflict, but forget to add in production? Same, your app will start in production and you'll be dealing with a downtime after deploy. Malsum conflict files. A lot of times, even though tooling is available, we end up writing YAML and JSON conflict files. YAML has indentation error, JSON, trailing, comma, unterminated codes, et cetera, et cetera. Types and key names. Again, all of this can be caught by lenders. But really how many of us are really setting them up as a standard plan? We've seen so many teams that we are helping, consulting with, even in our over projects that I am guilty of not setting these up myself. So then another problem, structurally incorrect conflict files. It is syntactically correct, but there's an incorrect schema. So for example, key added at incorrect level of nesting, instead of adding a list with a single element, you've added a scalar value. Again, schema validation is possible. JSON schema is there. But how many times do you really write schemas for every single conflict file that we are using? This is a slightly subtle take on the same problem, completely untested conflict files for higher environments. What happens is when you have physically separate conflict files for each environment, when you're signing off on a code plus conflict artifact in a lower environment, say in a stating on QA environment, you are expecting that when this goes, when this is promoted to production, you will have very few surprises. But that doesn't really hold true when your conflict files are physically separate between the lower environments and higher environments. Because suddenly when you promote, you are introducing a completely new conflict file which has been completely untested in the lower environments. But if your conflict files are being generated from the same source of truth, then apart from the values being different, you are absolutely sure that the keys are going to be the same, the schema is going to be the same, the white space is going to be the same, everything is going to be the same. So the percentage of file of your conflict file that has been tested in the lower environment is far higher than introducing a completely new untested file suddenly before going to production. So that's the kind of problems. Now, yeah, okay, I'll just take a pause here because on the next section of the talk is all mostly about code, code, code and FP concepts. Okay, any questions? Hang on, no questions till now? I'll just take a sort of one question in the Q&A box. Hi, and Nareesh is asked, is this Nareesh's question? Yeah, yeah. Yesterday's team keynote, his advisor was not to use static type language for conflict management, he said just use a single template. What is your take on his point of view? I think this is using a simple template is would sort of lead you to this, can very easily lead you to this malformed conflict files problem, or sometimes there are things that you need to do in templates that is not possible with simple text templating. Like you want to structurally repeat a JSON object, right? Now what is better, treating it as a text template, or treating it as a JSON object, which can be repeated and transformed using a programming language. Wouldn't it be a better solution to avoid such problems with K8S files? Probably not, I'm not very experienced with Helm to know that, but again, going back to that, this is not only for Kubernetes, right? So for example, I'll take your word for it, Jyotish, say Helm is the right approach. What about Helm is the right approach for Kubernetes? Given, I don't know that, probably it is. What about NGNX conflict files? What about HAProxy conflict files? Those are also kinds of things that Dal can handle for you. Helm is not a solution for those, right? Please correct me if I'm wrong. Cool, shall we move ahead? Okay, I'll just move ahead from your now. So yeah, so let's see what we are talking of when we say Dal is similar to Haskell. Now this assumes that you at least come across the Haskell syntax, at least peripherally. This talk is not a Dal tutorial as such, right? So for example, the next three slides will take you through a Dal piece of code snippet and Haskell code snippet on the Dal on the left, Haskell on the right, which, where we are generating the conflict file for two applications, where they need to be writing to the same PubSub topic and they need to be writing to the same logging destination, right? So we need to make sure that both of them are logging into the same place so that it makes it easier debugging. And for the app to work properly, both of them should be writing to the same Q topic or same PubSub topic, whatever you want to call it, right? So left side is Dal, right side is Haskell, right? So let's go at the very first thing. The first snippet which I've highlighted in a yellow box over there is a union or some type for logging, for the logging configuration, right? So the logging configuration can either be a log file or it can be syslog, right? You can compare the two snippets side to side, right? So the concept of union or some types is common in both of them. The syntax might be different, but conceptually they are the same, right? Next thing, here is a record type, a structured record type or a schema, whatever you want to call it for configuration. That is common across the application. So just what I said a minute ago that your logging configuration across the two applications should be same and your topic should be same. Again, you can compare these two ports snippets side by side and see how it's almost like a direct transliteration happening over here, right? Then here is the, so this thing is the schema and this thing is the value, right? So this is for my development configuration. So in my development, I want the logging to happen in a log file called log slash dev.log. And I want the topic that is used in the queue to be dev. Again, just minor changes in syntax, but otherwise it is almost transliterated, right? Now let's see what's like, if you were doing this in Haskell instead, what are the kind of complexities that you would have to deal with, right? So very first thing as a beginner, you will have to deal with this string type nonsense. So Haskell has some three or four types of string types. And when you write a string, it can either be interpreted as a string or a text or a byte string or a lazy byte string or a lazy text. And you need to use this language extension to be able to do things sanely and easily, right? So as a newbie, as a person just stepping into functional programming, this is just pure noise. You shouldn't be having to deal with this, right? Then next thing, even till date, having two different record types in Haskell to have the same name, right, causes problems. So you have to learn how to prefix. So if you notice on the left hand side, the field names are just called logging and topic. And on the right hand side, they are called CFG logging and CFG topic. There's a CFG prefix over there. Now this is the artifact of some Haskell history and baggage that two records cannot share the same field name. They can, but right? Sorry. So like even in something. Guys, I'm sorry to interrupt. You always broke the, just, I'm not sure. It was just for me. Sorry to interrupt. Please go on. All right. I'll just go on right now. Let's move on to the next thing now. So you, so over here, you've defined the data type and you've defined the value for it. Right. This is all statically. I checked for that you're writing now, then you're writing a function which says that, which takes that native dial data type and converts it into a property spine. So that dot properties file, which are typically used in sound is fine. Sound is fine. All right. Cool. So now this is the, so the previous slide was just defining the data types and the values for the data types. And this slide is defining a function. It takes a value of type common CFG, the common configuration, and it generates a properties factor. Right. So this is a function for converting a common config to a spring, which represents a properties, properties type. Right. There's a slight difference in how the functions are defined. The syntax now is slightly diverging. And I'll cover this in the, in slightly later in the presentation. Now this is converting the logging CFG, some type to a property snippet. Also notice that you are having to deal with nested letter expressions. Right. Something very common in functional programming and also dealing with some types. Right. So I will get a sum type and I need to consume it. And for every single branch of the sum type, I need to return a text value. Right. So that's, that's a, that's a tiny little functional paradigm, functional programming paradigm, which you are getting exposed to while dealing with that. Right. Then finally you're tying it all together in a, to get a final value. Right. Also again, a big let dot let in expression. Right. So the whole thing that you have an expression and then you have it's supporting definitions in a let binding. Right. That whole concept of doing this in functional, it doesn't come naturally to everyone. Now again, what are the problems in Haskell that you will have to deal with? Multi-line string littles are not supported. You need to use, over here, I've not used multi-line string littles. I've just used string code, catenation and slash and slash and manually, but there is a way to support multi-line string littles in Haskell, but it requires you to use a language extension and a special purpose library for that. On the other hand, on Dal, because it is, it is for configuration files. They know that they need to support multi-line strings. It is supported over the books. Right. Then finally, another code snippet. This time, instead of outputting a string, like a flat text file, now you are outputting the structured JSON. Right. So this is for the, so one application expects a properties file. The other application expects a JSON file as its function. Right. So again, function for converting common CFG to a JSON. So on the Haskell side, you're converting it to a JSON object and on the Dal side, you're converting it to a native record. There's a reason for this. Now, yeah, converting the logging some type to native record on the Dal side, because Dal has a built-in way to convert a record to a JSON via this Dal to JSON utility and converting this to on the Haskell side, you're converting it to a JSON, but by doing a whole lot of AST maps, ASTs abstract syntax tree. So we are using the AST library, which helps you serialize and decelerize JSON and we are being forced to manually sort of construct the JSON AST. Right. This is not how you typically do it in production code. In production code, you would end up using something like generics and auto deriving of the JSON instance type class instances, but to be able to reach that level, you have to understand generics. You have to understand type classes. You have to understand auto derivation. So a whole lot of, again, steep learning curve to do something as simple as convert an object to JSON. So then you tying it all together to get a final value. Right. Third and final part of the presentation. I'll pause here. Any questions? So Kim, to answer your questions, I guess everyone can read the chat. Kim's question is that why aren't we focusing more on the problem of how Dal solves the configuration problem, but that's the topic of this talk. Your question is valid. It's not an invalid question that why are writing config files in Dal better than writing, you know, Helm or Kubernetes config files. That's a valid question. It's a question you should ask before adopting Dal in your tool chain. But the topic of this talk is how Dal is a stepping stone towards functional programming. Right. So that's the primary focus of this talk. So now we will cover the kind of concepts that you will be forced to learn correctly while actually solving the configuration management problem using Dal. That's the angle that this talk is taking. So what will Dal force you to learn correctly? Right. The basic premise of this talk. Stepping stone towards time functional programming, not having escape hatches, but still giving you a gentle learning curve to be able to use certain fundamental concepts of functional programming. So immutability. Dal has immutability. And this is like the very first tumbling block of every imperative programmer. Sometimes when you when you've not written code with immutable variables, immutable bindings, it sometimes is incomprehensible for someone like, how can you solve problems in a programming language where you're not allowed to change the value of the variable. Right. So that whole concept of working with immutable bindings and variables. That's what Dal will force you to bring this into your muscle memory. Right. Explicit nulls. Again, another beginner stumbling block they are used. Everyone is just used to putting null checks in the middle of their the core of their code, but having to explicitly define that this can be a nullable value using the option type using some and none. Then you sort of end up going through that hot process of saying, hey, do I really need this to be an optional type throughout my code? Or can I check this at the boundary of my program and at the boundary of the program either I accept or reject the input. And then I don't need to deal with the null. So that kind of thinking of pushing these expectations on data to the boundary and starts developing once you have to deal with explicit nulls. So this is going back to this. I said I'll cover this later on in the presentation. I'll see if I can go back. Right. So in Haskell, all functions are carried by default, but the type signature doesn't make it obvious to you. Right. But in contrast that with Haskell contract that's contrast with Dal. The way of defining functions syntax might seem noisy at first, but to a newcomer it makes it immediately obvious what's happening that every function is a carried single value function. And on top of that, it's a lambda until unless I give it a name. Right. So what's going on underneath the hood and what all you can do with carried functions becomes very obvious in Dal whereas it's hidden in Haskell. Right. So you sort of tend to start understanding and using this concept more effectively in Dal forced to use higher order functions. Dal A doesn't have for loops. It doesn't even have general recursion. So Dal purposely doesn't have general recursion. One thing which I have not mentioned over here is that Dal claims and it claims it very proudly that they are not during complete because the designers of the language feel that a configuration language should not be during complete. So it doesn't have general recursion and apart from using higher order functions that come with the library, map, filter, fold, merge, there is no other way. There is no escape hatch. You cannot drop down to loops. You cannot drop down to writing recursions by hand. You have to use those or you have to express all data transformation using these higher order functions. Functional purity, no side effects. Dal A in any ways has no way to do are you accepted at the very boundaries? And again, no function can have any side effects apart from returning the language. Homogeneous list and some end unit types. I sort of take these two things together is that what I've seen in practice is that because of the constraint of homogeneous list, people end up exploring some end unit types. Specific people coming from dynamically typed languages where lists like a list can have integers and strings and rules and some reward. It doesn't matter. But in languages like Haskell and also other statically typed languages, your list needs to be homogeneous. But then you will come across a situation where you need to have a heterogeneous list. That's when you start exploring some end unit types. So that is one of the reasons why devs are forced to start defining their own some end unit types. And then that opens up those gates of using a richer data type, richer data types to solve the problems. This one, this is something which is subtle takes time to dawn upon you that everything in Dal, everything in Haskell is an expression. It's expression all the way down. Everything, giant if conditions, giant switch case conditions, functions, functions, let in whatever, everything is an expression and the expression needs to evaluate to a single type. This realization itself that in the end I'm writing a big expression which is being evaluated. It comes faster in Dal because the nature of the problem, you're taking something, you're doing a data transformation and you're outputting it. So you can see the expression being like smaller parts of the expression being chained and composed to form a larger expression. Then first you have the type system. This is something specifically in type function programming how to use the rich type system that something like Haskell has, something like even Scala has probably the hardest problem. There are many talks in function function itself about how to build, how to look at your domain and model that in types, et cetera, et cetera. Dal sort of forces this upon you. How, for example, because Dal is in the configuration management space, it is very easy to start doing things using strings, which is called stringly type programs. So for example, Dal does not even have a stringy quality operator, no rejects, nothing. You cannot do stringly type programming in Dal. It just prevents you from falling back to that. You have to start learning and using the type system, some types, record types, list, whatever, whatever, whatever, to express your solution. So there's some amount of forced usage of the type system which starts building your muscle memory around that. And the whole point of all of this is that while it's forcing you to do all of these things, it still makes the journey slightly easier. A, it's modern. It wasn't built so many years ago, like Haskell was. It has very little baggage. It is special purpose. And it has evolved with good tooling. It has built ground up from good tooling. So it has decent ID support. It has a Dal language server protocol implementation, which works very well with VS port. It has a super fast compiler, super fast runtime. It has more runtime errors. It has good compiler errors. And then even when it is, so there is this thing inbuilt into Dal where it can give you semantic diffing. Right. So if your record has three fields and your schema or your type has four fields, it will do a semantic diff very nicely. And it will show you that this is the exact field that you're missing across XML, across JSON, across native Dal records. So that's, that's actually very useful when it comes to good compiler errors. And like there's a whole lot more the way it uses checksum to ensure that your expression has a change the way it uses the way it has a very interesting import and package management sort of solution built into it. So all of that are like the good parts of Dal which make this journey much more easier. Right. So yeah. So Dal solves a real problem which is configuration management and obviously there are two people who are asking the same question that you know, sure it can, it can solve this problem, but so can help. Right. I'll leave that to you. Give both a try. See whether hell makes more sense or whether Dal makes more sense and then why not JSON net? Why not Q net? So just because another tool exists to solve a problem doesn't sort of preclude another tool from solving it in a different way. Right. So and the good part is that Dal opens up your mind towards type functional programming. So it is a stepping stone towards perhaps becoming a better Scala program. Perhaps becoming a better Portland programmer. Perhaps becoming a better Ellen programmer. Perhaps becoming a better Haskell program. Right. So yeah. That's the end of my talk. Oh, we have one question in the Q&A. Have you used this in production? Yes. Can't name the company because we are actively consulting with them, but very large company with I don't know about 4,000 engineers and again, explosion of conflict. So most of the problems that I mentioned in my slides were actual observations in the way that this company was managing configuration rights. Right. So yes, I have used it in production and others are also using this in production. You can go to the Dal homepage and there's a section over there Dal in production, which actually lists companies which are using this in production. We have one more. Any resistance from ops people to this. Right. So actually that's a that's a very good question and Dal is like not the only place where we are trying hard to answer this. Right. So as happens with every tool, there's a tool which solves a problem. You can see that it solves the problem in the small. So like, I don't know how many people are aware of this in the small and in the large sort of debate. So that solves this. It really does solve this problem very well in the small. So when you're scaling it across multiple teams where there are handoffs involved between the devs team and the ops team, when you have to think of how does the conflict file? How does the release pipeline? How does the build pipeline work with the branching strategy of the board? How does it work with the testing static strategy of the QA team? Apart from the poor tool, what I'm saying is apart from the poor tool, there's a pool of process angles around it. Right. So those process angles are different for different companies. Right. So for example, if in your if in your organization, the ops people are used to managing variables via managing like these kind of configuration variables via something like Azure DevOps release pipelines. So Azure DevOps release pipelines has a UI where you can manage these configuration variables that has its own set of pros and cons. Now if they're used to that and you suddenly want them to move to Dal, there can be a pushback. Right. So does this work for the processes that your organization is using today? What would it take to change those processes to sort of be now built around Dal? That's a question that every organization will have to answer for themselves. That's not something that the tool can answer and Dal doesn't even attempt to answer those kind of process applications. But yes, it's a very valid question that when it comes to fitting this tool inside a larger process, there are challenges. Cool. We'll wrap this up. All right. That was a great talk. Very informative.