 So welcome everybody to my talk, I'm excited that you came to it, not scared of the parenthesis. Well, that's a joke aside. The talk will be about half an hour long and I will show in the beginning there will be a bit of theory, some explanations and later on about from the starting and the half, there will be practical examples. If you have any questions, if they are short, you can ask them in between. If they are longer, perhaps wait for the Q&A, I plan to have five to ten minutes Q&A, so we should be on the longer side. My name is Adam and I will be presenting here about superpowers for closure and closure script. Who am I? I am co-founder of OrgPAD, as you can see, you can write me an email and I'm responsible for infrastructure and security there, so perhaps some of the stuff that you will see, you will think, oh my God, he is connecting to some production environment or whatever, but that's fine. I know what I am doing. So about Lisp, why am I talking about Lisp at all? Well, closure and closure script are dialects of Lisp and there are many people talking about Lisp and the hacker news main page is there is every second day something about Lisp or closure. Just today I have looked, there was a discussion, why was Reddit 1.0 written in Lisp and then they switched to Python and what is it all about. So I have put down some quotes about Lisp. You can see the XKCD in the middle. These are your father's parenthesis, elegant weapons for a more civilized age. Stuff like that. Many people are saying very positive things about Lisp. Of course, there are some people that tell very negative things about Lisp and so what is the buzz all about, right? You can see the quote by Eric S. Raymond. Lisp is worth learning for a different reason, the profound enlightenment experience you will have when you finally get it. The experience will make you a better programmer for the rest of your days even if you never actually use Lisp itself a lot, right? So about that enlightenment experience, I hope I will convey some of it to you. So what is closure about? Why should we have another language? Why should we develop stuff at all? Why should we think about these problems? Why should we consider different approaches to programming? Well, we have many problems writing, running and evolving informational systems. Among other things is that we tend to solve puzzles. We have difficult constructs that we occupy our mind with and then we don't solve the problems. We solve puzzles but we don't actually solve real problems that customers are willing to pay for. As the systems grow bigger, we tend to encounter a tower of bubble that we have so many different approaches, so many different modules and everything is kind of interconnected and not in the right way. It's getting difficult. It's tangled spaghetti, right? Also when we write some languages, we tend to discover some hidden semantics. We discover that perhaps concatenating two arrays is not an array or concatenating an array and object produces different things depending on the order of operations and so on. So these are all small puzzles again. It's hidden semantics, hidden syntax that we don't intuitively know what to do with. And the problem is that we don't have any predictability. So we cannot go to the boss, we cannot go to the customer and say, okay, we'll finish our job in like half a week and then we'll do some integration tests and so on and it will be all hunky-dory. We don't know. We are not sure. We cannot promise and that is really bad and that's keeping the whole industry back. So are we perhaps not using the right approaches? Do we have the right foundations? Well, that's a good question. Well, there are some approaches, well known, usually more in the functional programming space, that we try to focus more on the on the what, not the how. We don't tell the computer exactly each step, like we are not micromanaging the computer, we are more telling, okay, this is like the overall idea I want to have. And that is obviously taken to the extreme with the AI bubble right now. We don't even know what the computer is doing precisely. Perhaps we are too much focused on rigid and mutable data structures that are perhaps good for performance because the processor can work more efficiently using this, but we as humans are not very good at understanding these small places connected to somehow to another. We are not good at shuffling bites and bits as humans. And obviously, all the time we have problems with state management. Concurrent programming is very difficult and somehow we don't have any good tools to cope. So what can we do? I would like to focus on what is actually data and what is what is code. So data is information. We don't, it's information in itself. It has to be processed to convey some some meaning, right? And it's especially data doesn't or is not instructions how to do something. It's not, we can record some instructions but for them to be instructions, we have to do something extra, right? For instance, run this as code. So what is code? Yeah, well that is the set of instructions I was talking about. So that is the point five in the Merriam-Webster dictionary instructions for a computer or within a piece of software. So yes, that's about the definitions. You are probably not very, not much smarter right now. So I try to approximate or show some examples what I mean. Well, we have many complex systems and we have, they run and we don't usually change much in those systems. We configure those systems. So to some degree it is foreseen by the designers of the system that this will be possible somehow. If you configure the system correctly, you will achieve some goal. And so you have some redhead technologies for instance, like Ansible, and of course you have some configuration files like this, right? So that is quite understandable. But what about web browsers? Do they have any configuration? What do you think? Well, HTML is kind of a configuration for a browser. The browser has to render this based on the input, but it's a configuration, so to say. Yeah, we have Maven. If you are in the Java world, then you probably know. It's a build tool and also it has a configuration, of course. So is this data or is that code? Well, it's an instruction how to put pieces of software together, right? But it's represented as data. It's not Java code, it's XML. So there is some duality to data and to code. So what can we do about it? Well, we could, for instance, think about it in the context of compilers, another big system that we configure by putting in a stream of characters out of which a program gets compiled, usually, right? And that is very complex. Well, we can perhaps think about some other ideas to represent code and data with the same means. This is a joke about Lisp and Panentasis heavy syntax. So I want to introduce you to Clojure, finally. What is Clojure? So Rich Hickey, the author of Clojure Broad on the website. Clojure is a dynamic general purpose programming language, combining the approachability and interactive development of a scripting language with an efficient and robust infrastructure for multi-threaded programming. Clojure is a complicated language yet remains completely dynamic. Every feature supported by Clojure is supported at runtime. Clojure provides easy access to the Java frameworks with optional type hints and type inference to ensure that calls to Java can avoid reflection. Clojure is a dialect of Lisp and shares with Lisp the code as data philosophy and a powerful macro system. Clojure is predominantly a functional programming language and features a rich set of immutable persistent data structures. When mutable state is needed, Clojure offers a software transactional memory system and reactive agent system that ensures clean, correct and multi-threaded designs. So what's this all about? That's very condensed, right? So dynamic, what does it mean? Well, there are no explicit static types. You don't work for the compiler. The compiler works for you, right? In general purpose programming language, it's suited for application programming, for all kinds of real-time systems, web development, right? Perhaps it's not as suited for driver development, but if you can imagine writing drivers in Java, you surely could write them in Clojure as well. So that's about it. Interactive development of a scripting language. Well, it's dynamic, so it's much less code. It's not as verbose and therefore it's much easier to type interactively and do something. Efficient and robust. Each time you write something in the wrapper or evaluate a program, then it gets compiled down to JVM code. So the efficiency should be similar to Java, right? Depending on how you write. And everything is supported at runtime. So everything, if it wasn't, it would be very hard for the interactive development because you wouldn't know what you can evaluate at runtime and not it would be very difficult. It would be nonpredictable, right? And what it means to be a dialect of Lisp, I will try to answer in the next slides. So the language has some common literals. I think you recognize most of them. However, I want to point out the last four lines, the NIL, ratio, symbol and keyword. Well, NIL is similar to null. However, in the closure script dialect, you know in JavaScript you have null and unknown, right? And so closure script doesn't have it. It doesn't have it and both is NIL. Ratio is very useful if you want to represent fractions exactly. So you can basically differ losing precision to a later point. Symbol is useful. You can think of it to some degree as a variable. It's something that can hold a function, that can hold data. It's a name for something, basically. And the keyword is something like a label. So it represents itself has some properties of a function in certain places. And especially useful thing is that it can be namespaced as pretty much anything in closure. So you can, if you represent users, then you have some data structure. For instance, if you retrieve a list of users from a database, then you can say, okay, I get all those users as a sequence of maps. And each of the maps has keys that are named user name, user last name, user date of birth and so on, right? So I have spoken about sequences and collections. So there are four, the typical list, vector, map and set. They each have some interesting properties, very useful for programming. And I will show them in a moment. So, yes. So how does a function call look like? Well, you see it's basically a list that has two elements. The first element gets evaluated. It's the print function, print line function. And the second element and any other element would be arguments to this function. But you could view it as a list. So how do you define a function? Well, again, it's a list has five elements. First is the definition, then the symbol that holds the function, then a string that is basically a comment or not documentation string, basically. And then a vector of arguments here. You can see that is one argument. And then there is function that gets called. It's a string function. So it basically concatenates hello and then your name. And you could view it that is the same as I said. So what is great about consistent and predictable syntax and semantics? Because everything is a data structure. So that is very simple. It's easy for the compiler to evaluate it. The whole parsing is basically 800 lines of Java code. And that's it. So that is the first stage of your compiler, more or less. You have persistent data structures. What's that all about? I will show in the example. So because we are a bit short on time, I will change to the example. Just let me set up. I'll connect to our testing infrastructure. And I have SSHed in. Basically, we have a running application in the staging environment, the application server. And I'm now connected directly into that server, into that application. So that is quite common. You can imagine that is like if you had a Python program and you had the command line basically would be directly connected to the running program. So is that big enough for you? Can you read it? Or should I make it bigger? I will make it a bit bigger. So what I will do? So this is the replica. I could write something here and it will evaluate and that runs on the server actually. But that is not how you develop software, right? You have an IDE and you have everything in an editor. And that is what I will do in the last few minutes. So imagine you have some problems in production. You have people connect to your server and then for some reason these connections drop in some circumstances. And you want to debug this, right? But how do you do that? You could do TCP dumps and you could do all kinds of things. But why wouldn't you use your program to debug these problems, right? You have everything that you need. You should have all the data in the program. So the idea here is we have engines and the application server hides behind engine X. Basically it's proxy, reverse proxy. And so I need engines to give me more data so I can debug this problem. And so I told engine X that engine X should put port information, TCP port information and the round trip time into some headers that I'll parse and work with this information on an ongoing basis, right? So I'll just show my notes. Okay. So what I will do, I will introduce just to simulate something, right? So you can see something. I'll introduce a new route. And this is all data that is actually running in production, right? This is code that is actually in OrgPet. And I'll introduce a new route. Let's say it's net info and I'll label it as utils net info. Very, very basic, right? So and I'll take this label and of course I need to tell my handlers that there is a new route and that something should happen with it, right? So I write utils net info and then I give it a function. I just give it the symbol basically. So let's say it will also be called net info. Okay. And of course, the IDE tells me you don't have this function, right? So I need to implement it somewhere. And that's what I will do. I have something that looks quite similar over here. So I do the same for IP addresses. And so I will do something similar for ports as well, right? So I just copy the function, name it differently, of course. And what does it do? Well, I know that the header is called xforwardedport, right? So then I get the request. Sorry, I have two screens. It's a bit difficult to navigate. I get the request right here and then I do something to it. And this arrow that is so-called threaded macro. So what happens here is basically that the first thing evaluates and then gets put here, like behind the first argument or before the first argument, so to say, right after a function call and then this gets evaluated. So it's basically a pipeline, right? It's very handy because I don't need to read from the right to the left. I can read from the left to the right or top to bottom. So just to speed things up a bit, I will parse the address from using some other utility functions from the request, right? So that is, I have now the address. I was looking for the port. So I'll write in the port and I don't have this function, right? So I need to implement that function to get the port. So what do I do? I switch to the other namespace and I see here is the function for how to get the IP address and I need to implement the port function. So I'll just to shorten things. I'll just from my notes copy the functions in. You see that I basically look up the header and then because port number is always a number, then I can just make it a number and not a string. So that is what I do and these other things are basically the same thing, but for the round trip times for TCP connections. So, okay. So now I have the, I can send it to my Apple. So it evaluates and I have it available client and to see the ideal gives me these functions request, right? And then I want to have the socket address and that is get address and in that address and in that port, right? So what does this do? This is interrupt with Java. So in that socket address is something that is built into Java and enables me to make sure I have a valid IP and port and it needs the IP address for that to work. So I want to know what type of address that is. If it's IPv4 or IPv6, that is just more information for my debugging. So I do that very quickly in that address type and just get address and socket address, right? So that is, that just gives me the address format basically. And now I need to distinguish what type it is and then I will just ask about object that is instance and in that for address or write it got edit at the top, in that address type, right? So if it's instance of in that for address, then I just return four. If it's instance of in that six address, I will return six, right? So now with all the information in place, I can return something, right? So that this would be a bit insufficient. So what I do, I will return it as JSON because why should I complicate stuff? JSON, CLJ, JSON. Now I can give it just a map of keys and values, get host strain, socket address, my port that I was talking about that I didn't have before. And I can give it the TCP round trip time. And I just use here the other functions because I don't have to do anything else for that. So, and you should be able in a minute to go over to, oh, and I need to, of course, change this to application JSON, right? And so, of course, I now can send it to my repl and you should be able to go to a specific address and get the information, right? So let me send these routes there. I need to evaluate basically all these spaces and you see there is some state management already built in. So some things need to happen in some specific order. So that got all handled by the infrastructure for me. So I don't need to know in which order it has to happen. It gets evaluated from my source code. And so just to check in the last minute, we can skip here. And basically, I should be able, yes, right? And every time I call it, I get a different number you see. So it works, right? And it seems to use the same connection each time, but it just has some different parameters for TCP. So that's about it, a short example. I think we are about done. We have one minute, so maybe if you have any questions you can ask. It's all quite fast, I think. Any questions? In that case, I will show you one more thing. I thought that was all closure. I think for some of you it would be interesting to know how it looks like when you develop in closure script. And just a very short idea. I now have in development on my local machine this copy of OrcPad, one OrcPage basically with one cell and one image. And now if I want to just get, I am connected into it. And if I just want to get some of these units, what they look like, okay, I need to connect to it. Okay, so here you see that basically my Boromir is right here. And here I have some of the image. So the content is just some HTML image.