 Hello. My name is Atamart. I've been learning closure since 2013. And this is a lightweight talk. Relatively, I have very few code examples. Only just one free graph. I'll try to make it count. Initially, my interest in the SSI enclosure started when I was trying to, not trying actually, I did implement it. I was working on a functional, relational, programming environment. Have you heard of, out of the carpet? Anyone? Awesome. So, I written the language part in a list dialect, very simple list dialect. And then parsing and evaluating was done in closure, in the initial prototype. So, what would be the use case for using a DSL? After all, DSL is just an abstraction, right? I have two use cases that I think are most important, but we can discuss after the talk, if you think otherwise, we think there are other more important cases. The first one is about what we are saying, about how we are expressing the problem we are currently trying to solve. And when doing that, speaking a common language, by common, I mean, there are two sides of every program, right? There are the insides of the program, the white box sort of thing, and then there are the consumers of the program and what they understand and what they want the program to do. So, this way, speaking the same language, by that I mean, expressing the problem to the system inside. And of course, just speaking, only just speaking the same language is not enough, we have to be able to completely express the problem. So, we use the DSL abstractions to state the problem we are trying to solve and state it explicitly and correctly. And I think that is one difference of DSLs from other kinds of, I would say, lighter weight smaller abstractions like types, classes, records, enclosure, protocols, enclosure. And the, no need to read this. This is from, this is a philosophical, this is in the philosophical concept. This is about concept forming in real life and there are two parts to this. And I think there are parallels to designing DSLs with this. The first part is about how we take the similarities of existence of things when forming a concept while leaving out some of their differences. So, we take the similarities in, we leave out the differences and we call this new construct a concept. And the second part is about how concepts that are in another, in other concepts are differentiated between each other. Meaning that a concept can have, maybe it doesn't have any existence, maybe it has only concepts, because concepts also form a hierarchy. And within that kind of concept, the sub concepts, we have to be able to differentiate them somehow. And the second part is about that. To illustrate, we have existence, right? Three tables. We all, we all immediately recognize that they are tables. But we also know them collectively as the concept table. When I say table, you don't think I refer to a specific particular table. So, this is, this is about the first part. And then when we form the concept of table, we take the similarities. A table is something that, that is lifted from the ground that we can put things on, right? Same thing, chairs and they collectively form, of course, chair concept doesn't include only these three particular chairs, all the chairs get inside that concept. And the same thing is for couches. And tables, chairs, couches, we can take these three concepts. We can put them all together and call them collectively furnisher. So, so we here, we form the higher level abstraction here. The first abstraction was from the existence and the second one was higher level. And within this abstraction, we can still see the differences of different types of objects. They, they are not completely melded into something else. They still retain their identity. The second use case I came up is like all other abstractions. We use them to hide details. And similarly, if I go back, but when I say furniture, I mean a lot of things, but I say only one word. So, I've hidden a lot of things. Maybe you can say there's some, some ambiguity. There's some weakness. But actually, if I'm actually talking about furniture, that's exactly what I should be saying, right? So, two use cases, speaking the same language, effectively stating our problem. And the second one is the two height implementation details. These were my use cases. And this is my three graph. I have this. Obviously, this is another binary tree. The interesting, this is slightly interesting for me. If I take the leaves of this tree and replace them with symbols, yes, this is actually about closure, by the way, not just iron. If I replace it with symbols, because closure is a symbolic language, right? That's actually a very important differentiator of closure than pretty much every other popular language. And this is the code we write. And internally, it can be represented as a tree. You can look at the parenthesis, so many parenthesis. And then, actually, if you're used to writing closure code, you will start seeing actually trees. You will start seeing forms. And because of this simple representation, closure is very suitable language for building DSLs. Maybe suitable is not a good word, but it's easier, I would say, it's easier to build embedded DSLs in closure than pretty much any other language. Because we can change the syntax. We can play with the syntax. We can use macros to sort of replace the code at compile time, even though there's not really a compile time for closure. Or at runtime, we can still accept forms, quoted forms, and we can play with them. It doesn't even have to be the form at runtime or compile time. The form we accept doesn't even have to be valid code. It can be anything. I categorize DSLs as independent and embedded. This is more about embedded DSLs, but the difference between them is basically embedded DSL sort of extends the host language. It's within the host environment and extends the host language. The host language, host environment can be fully available or it may be partially available. For example, you can have a macro that accepts some arbitrary code, but within that code, you may disallow certain structs and you can remove them. But the example I will give later, actually, all the closure is available. An independent DSL is outside of the environment. It's in a way above the environment or it's executed outside of the environment itself. So you have full control of what to allow and what to disallow and what to expose to your DSL. But it's actually more work, especially if you have a very flexible language like closure. It's more work to build an independent DSL. But another advantage of independent DSLs is if you have your language, your own language, you can replace the implementation language and nobody has to know. So before we move on to the embedded case, this is an example written with Instaparse. I'm so surprised. It always has shown the horizontal score bar here. Awesome. This was my list dialect grammar. Anybody has worked with Instaparse? So, some are familiar, right? So basically, let's take a look at this. This is an application, open parent and a close parent. And in between, we have any number of, it can be white space or atom or list or another application. By the way, just again, just only looking at the application itself. When Instaparse does its job, all I have is AST because it doesn't necessarily have to be valid my language. It doesn't have a name, actually. It doesn't have to be valid. So this is the first part. I get the AST. I still have to validate it and then I still have to evaluate it afterwards. So the main subject of this talk is this little shape language. It's a very, very, very simple language. In real world, I suppose it would have many more bells and whistles, but this is very simple. And my inspiration comes from this paper. In this paper, they are given a some sort of defense program prototype to the participants and they use different languages to write a prototype. And there are some interesting findings about how Haskell ends up producing less bugs in less lines of code, etc., etc., and good performance. So basically, according to this paper, Haskell is much better than C++ in every way, as far as I remember. So this is a graphic I stole from the paper itself. They define these shapes and then they define, on top of this language, they define rules of engagement. Like this may be a ship, maybe it's going this way and maybe this is a plane flying like this and then can this ship shoot down this plane, things like that. In my language, it will be because I think I have something like 15 minutes or so. It's very, very, very simple. We will have points, circles, and very simple transformations and very simple combinations. This is quite complex actually. Yes. So we will have the ability to describe simple shapes and compound shapes formed by combining simple shapes. And we will be able to define points and the only purpose of this language is to check whether a point is inside the given shape or not. This function, inside function, takes a shape, takes a point, returns boolean. This is from structural interpretation of computer programs. I'm guessing some of us are also familiar with this. There, Abelson and Sussman says, to understand the language, you need to ask three questions. What are the primitives of the language? What are the means of combination? And what are the means of abstraction? So let's take a look. I have two primitives, circle shapes and points. Very simple. X and Y coordinates. And this one, this one is, this is a function of course. This one is a value. It represents a unit circle, a circle center that origin with the radius of one. The means of combination is, so basically all the combinatory functions, they only operate on circles. I said this is a very simplified thing. No, some of them also operate on points. Anyway, they take a shape and they return a shape. Translate moves shapes and scale, basically scales them. And if I, if I added things like rectangles or triangles or more complex polygon shapes, etcetera, etcetera, I would have to rotate and things would get exponentially complex. So I just kept it simple. By the way, the code and tests are online right now. After this, you can take a look. The second set of combination functions, they take multiple shapes and return a single shape. You can build a union of shapes or you can build an intersection of shapes. And actually by, just by, you only need to supply two, union or any intersection, union and difference or intersection and difference. No, not the last one. You only need to supply two functions. Based on these, you can also derive difference for example. Means of abstraction. Since this is a embedded DSL, there is nothing further that I have written you can use the entire closure. How to think? Well, let's say, let's say we have decided we are confident and we really need the DSL. Nothing else will solve our problem. How to think? How to start designing the DSL itself? I have three guidelines for this. I would suggest thinking outside in or top down, if you will. I would suggest following the closure principle. And okay, syntactic sugar is not a goal in and of itself but like ease of use, let's say ease of use. The third one is. By outside in thinking, I mean, I suggest starting thinking the interface, the consumers of the DSL, the users of the DSL will use instead of the internal details. As little internal details as possible, we should expose to the outside world and give them like a clean interface and small interface as much as possible. Let me show you this again. So we have a point function. We have a value origin and we use the same value. We have another value circle, unit circle. And our functions are directly inside this protocol. But hopefully, if I didn't overlook something, you can just import these as normal closure functions and you can just use them. The consumers, they don't know that this is a protocol. And points implementation of this protocol is here. Let's just take a look at translate. Basically, I get the deltas. I add the dx to x, add dy to y and indentation is wrong. Anyway, I build a new point. Closure principle is our means of abstraction if they result in the language's own primitives and not something else. That means we are adhering the principle of closure. Because otherwise, if I combine things and I end up with something that is not usable by my language itself, then it's useless from my language's perspective. So as much as possible, we want our combinations to end up other primitives. Let's say, let's move the unit circle and scale it. We can do it like this. We first scale and then move. And then we can check that origin is actually inside this new circle. It should return true. By the way, means of abstraction, I can just make some small changes. I can make this definite and I can change this with variables. I can build a circle building function. So building abstractions is easy, actually. And the syntactic sugar, I already mentioned this. This is an internal detail. I don't want my users to know there is such a thing as a record or there is such a thing as a protocol or what is the name of the protocol. I just want them to understand and use points and circles and translate, et cetera. If closure was better at hiding things, I would hide it even further. And this is another example. This is from test check. Are you familiar with test check? It does generative testing. So to explain this real quick, we want to generate a random list of bytes in this case. Generate a random list. Give me that list. But along with this list also give me one of the elements in the list. The element you give me, I want it to be guaranteed to be inside the list. So it returns a couple. This generator returns a single element of this and then this is basically just returning the list as is. So I'm using bind. And I use two generators but the second generator depends on the first generator in this case. And test check has a syntactic trigger for this. You can write it with let macro. As you see, I first generate my list, this line. And then I pick one of the elements and then I return a list. And this is much more readable I think. And basically what happens is let macro converts this code into pretty much this code. But the advantage is maybe for two things it doesn't look very interesting. But if you have ten things, good luck writing ten bytes inside each other. It's not easy. So we define the primitives. We have the means of composure. And we are adhering to the closure principle. And we have a nice language now. Like let's say the external API is somewhat stable now. Now without changing the external API, we can improve the internal API. And maybe this is one of the advantages of using a DSL. We can suppose we have two circles. One of them is inside another one. We can replace a union of these two circles with the larger circle. Because there's no need to compute and there's no need to go, okay, am I in this circle? Yes, okay, then check the next one. No need. If it's in the larger circle, it's also in this, sorry. Yes, it's either, right? If it's in the larger circle, we don't need to check the second one. Second optimization we can do is again we can take a complex form without evaluating it first. And then we can transform unnecessary, we can eliminate unnecessary transformation if I'm moving something this way. And then I'm moving it back in this original place. I can just remove both transforms, both translates. And the ultimate elimination is I can use a transformation matrix. If your language is two-dimensionally, you need a three times three matrix. And then you can, within this matrix, you can encode translation, rotation, scaling, shearing, everything you can encode in this matrix. And change your language without changing its API. And that's all I have. Questions? As your example implementation, did you use the InstaPath to pass a real code? Or you just, you could do the code? You're familiar with tarpit, right? Tarpit paper. So it's kind of like a database and a programming environment all inside each other. I will say it's like stored procedures, kind of, but not horrible like that. The language itself is purely descriptive. It has descriptions of relations, descriptions of queries, descriptions of functions, etc., descriptions of constraints on your data, like unique constraints. This is the language and there are absolutely no effects in that language. So I pass this language, and based on that, I get the schema for my data, and I validate my schema with my store, and I build the functions. But the closure prototype, it's easy to build the functions. It's just transformation from my very restrictive expressions to closure. And then in the system, the system sits in the center, and then you start feeding data. And then there are observers. They subscribe to particular queries. And then data that touches, that changes the results of those queries change. The observers are waken up and then they get the updates. So in many applications, like we read some data, we do something, we write it back. But in this case, it completely decouples the reader side and the writer side. And in the middle, because you strongly specify your schema, it doesn't allow you to do things that violate your schema. So the language part is not very heavy, but the writing and reading the data is a bit tricky. So you got another abstraction about this closure? Yes, because if I accept, you can also read the CLGA file, right? You can read it. And then I think the function's name is read or eval, one of them. You can just read and execute arbitrary closure code within your program just by giving the file. But then the entire closure language is available and the user is allowed to do anything with your data, allowed to violate the constraints example. To constrain that, I had to use a special language. It's crippled. It's crippled. Another thing I had to build myself is the schema is typed. Types are very simple, but it is typed. And it is typed in a way. This is, by the way, this is relational algebra. It's an implementation of relational algebra. And let's say when you're creating a SQL table, you say this column, this type, this column, this type. And then sometimes when you read the schema back, it may come in a different order, but actually it stores those columns in a particular order internally. It's exactly like, its layout is like an exact sheet in a database. But in relational algebra, there has to be no order of the tables. And the underlying system should be allowed to optimize, taking advantage that the fields are not ordered. So what I had to do was I had to create a product type that is not positional. If you're familiar with Askel, you can have like my type a, b, c, right? And then a always comes from b, comes before b, b always comes before c. So it's positional. You have the product type, but it's always positional. You cannot say, okay, my type takes three things, but in any order. You can never do that in the type system. So I had to build that as well. Thank you very much. No problem. My question is two questions actually. I have any other important use cases for DSLs? Can you think of? Because I'm not a DSL guy and I think many times you can solve your problems not writing a DSL. So are there any strong arguments to build a DSL? Is my question. And the second question is, again something I'm not entirely sure. Yes. Can you think of anything else about the design of DSLs? Any opinions? Because none of this is really a mind bending, right? Because you've already covered it properly, but I've used DSLs before for security in terms of receiving input from users to describe the problem and then running it in the cloud on a shared system. And the sandboxes can do the same thing. You can guarantee the time they can run it. So this is a performance critical environment. So DSLs you can guarantee there's no looping. There's nothing that can run for a long time. So you can make high performance guarantees about how long a problem will take to run. I've never used a sandbox in Clojure, but when I was simple DSL, really simple DSL, I can explain this to somebody maybe less technical. I think easier than the entire Clojure, right? Also, if you have practical experience, is it like more work to write sandbox Clojure or less work? Or the same amount of work? So it's like a seamless experience. So it limits the amount of keywords or kind of expressions you can use to make other dangerous constructs. Okay. So you parse Clojure like code in Instaparse and then pass it to sandbox. Yeah. And everyone knows if you said this. But coming up, coming up with this, coming up with this for me was, where is it? Coming up with this for me was quite challenging. Even though this is like maybe one percent of the complexity of Clojure. It's very, very, very simple. Yeah. So it's a little more reliable, fully defined grammar. So this part works well. It could be a bit slower in my experience. So if you want to run, parse things at runtime, you can have some problems because this part needs more memory. Also, there's no streaming. There is no streaming. Instaparse cannot stream data. Or you cannot stream data through Instaparse. Okay. Thank you very much.