 My name is Zeishan Lakhani. I work at Basho on Distributed Systems, and I'm also a founder and organizer of Papers We Love. And this title is The Meeting of LFV. It looks at the meaning of life, if you look at it quickly. And the title actually comes from Michael Fogus. We were talking about Liz Slayward Erlang, and then he had this idea. So I'm stealing from him, and he's a good person to steal from. OK, so you know it's a good talk. We have Nicholas Cage here. You know this is going to be awesome. So this is an adaptation. And the reason why I have this in here is because I'm not sweating as much, but the movie adaptation is like a movie about a writer who writes himself into the film, and there's a twin, and this whole thing that happens as well. But they're writing herself into the film. So as I was preparing this talk, and it's only 30 minutes, and I decided to actually make it shorter, it could go really into why Erlang, why Lisp, why everything together. And there'll be a little parts of that here. But it's too much to cover of all these languages. So I said, let me write myself in into the parts I found really interesting as I was learning LFV. Yeah, so I write Erlang every day at Bachel. I did not know Erlang before I went there. I still consider myself someone who's still pretty new to Erlang. There's still new things I get from it all the time. So as I went through this talk, I really got a sense, a lot of the history that is around. And one important person, not maybe as important as Joe Armstrong and the history of it, but Robert Vernding is very, very important. So he joined the Erlang team in 1988. At that time, they were trying to do more of an interpreted Erlang. And then in 2008, he created Lisp-flavored Erlang, and I'll show a little bit when I made my LFV slide why he did, and I think they're really interesting reasons. But let's talk about Erlang. Who here maybe has played with Erlang, written Erlang? Cool, that's actually nice. Okay, cool. So 1986, there was an initial work done in the 85 range, but 86 was the first release of it. We don't get OTP, which is where we have all this great sugar for all the stuff that Erlang gives us. That doesn't come out to 96. Erlang's open source to 98. But there was a lot of early years. There was an initial version where they had Erlang running on Prolog, which was Jam, Joe Armstrong, yeah, Joe's abstract machine, which is pretty great. So there's actually an amazing paper on the history of Erlang written by Joe Armstrong, which I highly recommend if you want to get an idea of all the things they tried and why there's some similarities to Prolog and the syntax and things. Why there's also a lot of benefit when they went to C. So the idea of concurrency-oriented programming language just comes from Joe Armstrong's dissertation, which was only finished, actually, I think in the early 2000s. So he kind of went back to, I guess, to finish it or maybe was waiting there for years. But these things are really good. I think these are the kind of fundamental tenants of why Erlang is so great. Everything is a process. Several processes operating on the same machine must be isolated. A fault in one process should not affect another. This is why I work on a database. Erlang is very good for all these tenants. Each process must have, we have PIDs. With PIDs, we can do a lot of things. We do a lot of tracing. We can feed PIDs and get an E-flame graph of everything going on in our system. There should be no shared states. So everything is on a process. When I talk about some garbage collection stuff, you'll see also why that no shared state is really, really important. Everything is done through message passing. This is what we know as the actor model, which when they were creating Erlang, they did not call it the actor model. It just kind of happened. This is how it worked best for them when they were creating it at Ericsson Labs. And yes, those last tenets should be possible for one process to detect a failure in another process. We should know the reason for this failure. Let's talk more about that. So some important tenets, too. Like John actually talked about in the programming Erlang book, so they're really early, that Erlang was created with the sense of writing for concurrency in distributed systems. Whether that was on one core, on one machine, and later, as things became truly distributed, now we have all this hardware. So in 2005, 2006, they even had S&P support, which now allow that you can have all these schedules on many threads. So even though there was obviously work to make this happen, as the hardware improves and we have more ability, Erlang has really went forth to adapt to those things and give us a system that maybe not as performant of some things like in C++, it's still very performant for the needs that we have in this kind of soft, real-time way. So as I talked about things that people might know, the idea of a mailbox, one per process, messages sent and received, we're not sharing any state. We pass messages around. We pass data around. And that's how we can deal with things. There is some mutable idea of state when you think of process dictionary and ETS. The whole talk, we can talk about each of those things specifically. But in general, people know Erlang has passing messages around. That's how you can do things like side effects. But on a per process level, everything is single assignment. It's pure minus, again, these things of ETS and process dictionaries. We're going to talk about. But here's just an example, some code where I'm not using any of the OTP sugar here. Just using a spawn here. But I have a process flag where I can capture an exit. So this process that I'm actually calling, this sync process later, if it crashes, I can actually catch that and do something. This is the part, the tenants that was in the couple slide, where I actually know about when other processes, another process, fails. And I can do things based on that. OK. So one of the things when I was learning Erlang that really, really jumped out for me was this ability, well then, an OTP actually makes it much easier where I could have these different kinds of supervisors. So the one for one, this process fails. It gets restarted. And people say all the time, Erlang, let it crash. That's kind of the whole thing. To me, it's, yeah, let it crash is a big thing. But also having so much knowledge about why, there's so much you can do to figure out why these things are happening and to trap these things and deal and deal and deal with it. And the supervisors are really great for that, because these are really the different restart strategies. So for example, we have the one for all. So maybe if all those processes are kind of linked together, if one fails, we can crash them all. And then they'll all restart. And the thing for the rest for one. So this ability to have processes, watch processes, and determine how they're linked in that structure gives you a lot of power to understand your system. And one of the earliest slides I had in the bottom, which is from a talk that was one of the recent Erlang factories or EUCs, where he says it's not about, Erlang, the let it crash thing, we talk about that. But it's about resiliency to bugs. You might have bugs in your system. You can write a lot of probably most production Erlang systems have bugs in the system. But it's a resiliency to know that maybe there are points when those bugs won't happen all the time. And if they do happen, it does crash the system. We know that we have ways around it. That's really important. And from a production standpoint of larger systems, sometimes all you need is just to keep, keep running. And obviously, there's other things like Erlang has, like hot-co-loading, which was probably more used at a different time. But to have this process that don't matter what's happening in the system, we have ways to keep going. So now we're moving the Lisp land. So where Erlang has, there's obviously a background there in prologue and other concepts from functional programming. Lisp has a lot of other power. I mean, it was a great talk by David Nolan that he gave in New York that I always like, which is called Lisp is Too Powerful. And he was talking about it more in the sense to how things in ClosureScript have happened where JavaScript programmers are not ready to handle the power that you have in Lisp. And I think this is Paul Graham Crowe. I mean, obviously John McCarthy is the foundation behind this behind Lisp. But this Graham Crowe, he says, the whole language is always available, which to me is an amazing thing. And actually, as I was playing with LFee again, with Erlang you have like a shell. But with LFee, I was back at a rep. Well, I had done a lot of ClosureCode previously in my previous job. And I forgot how fun it was. Lisp is really fun. It's super powerful. Yeah, Haskell won't let you do certain things. And that's why they have a really great type system. And in Lisp, you can really do anything you want. And I think there is kind of an amazing thing there. I also mentioned, like obviously, so when you see some of the other Fee stuff, there's a lot of stuff from Common Lisp. But there's a lot of stuff from Scheme as well. And I love Scheme. It's on the Lambert paper here. I also am a huge Racket fan. I wish it was a little bit more performant. I would just use it every day. OK, so here are a couple of examples from the little Schemer. I think it's one of my favorite books. What is an X expression? I was going to talk about Lisp. These are S expressions. And that's what everything is. I mean, the beauty of Lisp is that in some ways, in many ways, it's really small. Obviously, there's always other functions that people add, and macros, and languages like Closure. There's a lot of stuff added to Core in the standard library. But at itself, the way to process Lisp and do things with Lisp, you can do almost everything. But there are Monads. I won't get into it too much. And there are a couple of great examples of actually, like, Monads and Erlang, and how you can bring those to Lisp-lavered Erlang. But there's a great paper about kind of Monads in a more Lisp point of view by Adam Folter and Daniel Freeman Folter, who gave the Crypto Workshop here. OK, so there's a quote by Andrew Rapelle that says it's in one of the compiler implementation books, where the notion of abstract syntax is due to McCarthy, who designed it. The abstract syntax was intended to be used by writing program until designers could get around to create a concrete syntax with human readable punctuation, instead of lots of irritating silly parentheses. So I don't know how true this is. It's definitely in his book. Maybe it's a joke. Maybe it's real, that they were going to have a more true compiled system at some point. But it just took off. And maybe I think it took off because of some of the reasons we'll see. And a lot of it is, you know, Rich Hickey, the last talk, referenced it in one of his talks. But that idea of code is data. I can take code. I can output it. I have data. I can then eval that back as code. And we'll see some example of that. Homeaconicity is the term for that. It gets, I think, used in a lot of things. But there's an ability that, especially in the runtime, I can interpret what I'm doing. And no matter what it is, whether it's the output of a function or passed around to various things, it's all the same thing. The data structures are the code. So here are some of the reasons I won't go through them all. Why Robert Vernding decided to create Lisp-flavored Erlang. Obviously, he had been working on Erlang. He obviously is on a lot of the early papers. He's on the garbage collection, the initial garbage collection paper as well, worked on a lot of the initial tooling of Erlang. But he was an old Lisper, as he talks about. As you read through these reasons here, he wanted to experiment with compiling our language on top of Erlang. He was not working with Erlang at the time, so he's looking for some other interesting project. He likes languages. I mean, nothing in here, the fun problem is solved, nothing in here is like, you should use this because it will make your app better or it will make your system better. None of those reasons. There's reasons why people use Erlang for that. There's reasons why you have Lisp. But at the end of the day, it was something to do for fun because these are languages. We forget now, we have so much discussion about which language is better for this. They're all really good for various things. I use a lot of different languages for all kinds of software that I've written. But it's sometimes about being fun and creating a language and that's what I'll be talking about going forward. So now we're moving away. So this is like a very Erlang-y example in Lisp-flavored Erlang. Basically, I'm sending messages here, spawning a PID. And I'm sending these messages, and the output there on the bottom where I send the message, that's me just doing it on the shell. And I'll call actually the REPL from now and because in Lisp-flavored Erlang, the REPL, unlike the Erlang shell, you can actually define functions. You can actually define macros. You can slurp in macros from various files and play with things there. So that's actually something very different from my everyday working in the Erlang shell was I was like really happy again to have a real REPL where I can define functions and interact. So in this here, I'm in the REPL. I send a message to it. I send a couple other messages and then I flush, which means then I get the receiving, the received block, that synchronous block back. So I get these messages. This is a pretty typical kind of Erlang receive block, but here I am in Lisp-flavored Erlang. Nothing too crazy here, but we'll get into I think more interesting things. But I guess one thing I will mention is the let. So here I can define the local scope this way, which coming from obviously closure and scheme, I was very happy for. Now, you know, Erlang has pattern matching. For me, I think it's really great. I had done some SML where we're doing pattern matching on types and various things. But in Erlang, the kind of the hotness is that you can pattern match on binary. We do this in some, obviously a lot of stuff at Ryok, but there are people using it for all kinds of reasons. This is a really simple example of doing in Lisp-flavored Erlang, but you get this, you get binary pattern matching. So you match on the bits. It's really, really great. And the pattern matching in general has made it so easy. I mean, of all the languages I've learned, whether I was doing Python, Ruby, and the more functional languages, I will say, maybe because I've done some scheme in closure before, but Erlang has been the easiest programming language for me to learn. I think the concept of single assignment, functions, first class modules that you can just kind of call and do dispatch on, polymorphic dispatch on, and the ability to do all this pattern, and all the pattern matching that you can from a functional standpoint, not from types, is really amazing. All right, so now we get into some more Lispy things we come here. So this is really, I think is really cool. This is a do call, which is not like a do from closure, but more like a do from Lisp, where I basically combine these variables, N, M, and C here to an initial value that will then increase or apply that function that I have to the right-hand side. And as it goes through the loop, it just buzzes like a do until, constantly applying the body, do until this, that N is greater than M, and then give me the return value. So this is actually something that you will not really get to see in Erlang. This is when you start moving into like, now I'm doing these things in a Lisp kind of way. If you see here, I have this little print statement as a different function I had, where I'm actually passing a function, printing that out, and then the print function actually has evals that function. So again, this idea of code is data, data is code. Okay, so here's another example of like side by side. This is considered Joe Armstrong's favorite program, just like snippet of it. It's like a factorial server. So you would see the Erlang version with the received block there, and here we have the LFE block. I mean, these things don't look too crazy. You see like we have tuple, there's no concept of like the squiggly bracket in LFE for example. So we use tuple. There is a shortcut you can do with types like tuple and binary. Like you saw with the binary, it's a hashtag or pound B, and then yeah, so that's just an example there. Okay, so I was kind of going through this. LFE is really interesting. I'm getting to know things. We're gonna talk a little bit about how this compiles to intermediate representation, and then to the VM beam. So, but I was going there, I was like, you know, garbage question. This is the cool thing. So how does garbage collection work in Erlang? Cause really we're talking about the same thing. And we have other languages. I mean, I'll talk really briefly about elixir as well and how it does something a little specific to macros, which is really interesting. But there's a great paper I was reading. They've obviously probably changed the garbage collector around time, through time. But the one pass real time generational mark sweep garbage collection paper by Vernon Armstrong. It was really cool. It also has this idea where it talks about generational GC, where it, you know, the idea that most objects only live a very short time, a small portion live longer. So we can collect, if you think of it like that, like we have this horizontal thing, we have those that are younger and those that are older, it will collect the younger ones, but does it on a per process level. So that's why, you know, one of the kind of really cool, awesome things in Erlang is that even though there may be the garbage collector on one process is taking a long time, it doesn't affect the rest of the processes. Each processes own stack and heap. So, you know, the one caveat to that is things have gotten newer from the original market sweep paper is that we have that single heap on each process for objects that are only up to 64 bytes in size. Now when binaries are greater than 64 bytes, there's a separate heap for the application that is reference counted. So it's a little different. There's some issues, you know, good and bads that occur with that depending on how large your objects are. There's a great book, Erlang in Anger, for more like production system stuff by Fred Herbert that goes into that more. But it's really interesting. So as I was kind of going through them here, yeah, go for it. So the, yeah. Yeah, for the, again, only the larger objects, yeah, go on there. Okay, so as I was, you know, I'm getting the GC, I'm getting to this again. This is my kind of path as I'm learning List Favorite Erlang because I kept going back and forth. And so the next step for me was, okay, now let's talk about interop. And I think maybe in earlier versions of List Favorite Erlang, I heard maybe interop was a little bit harder. There is definitely no problem now and I'll show that in an example. But actually you can actually run, it's not 100% working from what I've read. I didn't try this myself. But you can actually interop with Elixir as well in List Favorite Erlang. So, you know, all these things that are being compiled on Beam can kind of coexist and you can use them together. Use libraries, use this. And it all works out really well. So, yeah, so at RIOC we run out, we have, we pay even for a license for Erlang QuickCheck from a company called Cubic where John Hughes works, John Hughes who created QuickCheck, the original paper and worked on a lot of the early Haskell. And we use it all the time. And I was like, I was not sure if I was gonna get it to work because I can't actually explore the QuickCheck code because it's like under a license. If I can actually get this to work in List Favorite Erlang. So I have these HRL files which are kind of like Erlang header files where we can put in types where we can use Dialyzer, we can put in any macros we're gonna use across an application for all different modules. So when you load EQC and you have your license installed, you just add these libs and things are just supposed to work. And we see here, you know, I can give, you know, talks about property-based testing and holding our time in QuickCheck. But we have a simple like just checking the property of reversing a list here. But all this works, this List Favorite Erlang with QuickCheck, you see that I have the colon EQC that's me calling into the interop there. There's a couple ways you can do interop. There's like the colon first which is sometimes really nice from a naming convention but I could also just say EQC colon QuickCheck. But this works. I run this, boom, you can even see I have some real tests, they pass. It's a real thing. Interop is really, really easy even from doing QuickCheck tests. So we talked about the REPL, how great it is, how it can define functions, how it can define variables in the process of the REPL. I can define macros. LFE is a List Two, so it's like common Lisp. Erlang has kind of a flat namespace because you can actually pass modules around and do things with it. List Two allows me, it has a different namespace for variables and a different namespace for functions. So you see this example here. I can set this variable xx for, I can have the function xx, I can call it three, this is the matter. Obviously, we'll talk a little bit about how Genic macros when I get to the macro section. Here in closure, List One for example, that would fail here. Getting the variable, calling the function would fail because it doesn't understand that it's two different namespaces. Now this is protecting you. Again, in a more common Lisp varied world, you can do whatever you want. So I had done a lot of Racket, I took a class in programming languages at Coursera by Dan Grossman, how he recommended, how I got into functional programming really. And I said, oh, let me take some of my Racket examples and can I just convert them, List Favoriter, how hard or easy would that be? This one is like, it's hard to tell the difference. I'm just defining here like a sequence with a stride or a step that you can do here. There's obviously a difference, I have null, I have an empty list, but this works pretty well. So coming from a scheme or Racket land, LFE is like, wow, and has all access to OTP, any kind of airline program you run or run. And yet, if you know scheme, you can just do it, it's pretty amazing. Something a little more interesting, the concept of like having a con. We can do pattern matching too, which I've shown in a couple of previous slides and I'll show them a few more, but this is more like I can actually do these checks in a con fashion where it just basically tries each one until it goes to the bottom. Pretty simple stuff if you've done scheme, but to have this I think is really nice from an expression level. So here is something a little more interesting. I was like, oh, let me create a function. I'll show just ahead what this looks like, where I put in a list of one, two, three, four, and then A dash C and it basically cycles through the list and creates these pairs. That cycle list function, that's actually returning a function that I'm actually like a stream. And then in stream for end steps, I'm actually executing the thunk lazily. So I mean, Erlang usually promotes eagerness, but you have the ability to do laziness. And I think here from a list kind of flavor, it's actually like almost easier to do and easier to understand. So here I am just conzing lists, returning functions, and then basically applying those functions. So you have into something interesting here called flat rec. This would be something like let rec, where you're doing a mutually recursive scope here for functions. There's also like let star, so you can basically have as you have your bindings for let the next binding can call into what the previous binding was. And these are really, I've always really liked how a racket scheme and lists have dealt with scoping. You have a lot more control over scoping, and this is a really good example. So with flat rec here, I have this function that I can then call anywhere else within that scope. Erlang. Yeah, so as I go through, these comparisons as we go back and forth, yeah. So yeah, the test pass, and yeah, there's a great testing framework in LFE that looks a lot like the closures, the death test stuff. So they've done their own. They had some work initially just working with E-unit, which is what people use for closure for Erlang, but they've written their own library. It's really nice. All right, macros are kind of the really, really cool thing. I think this is the coolest thing about LFE and applying it, and you know, David Nolan had talked about in a talk, he gave an early talk about why this macro is so great. And we know these things, code generation, DSLs, DSP, we, you know, it gives, you can do dry better. You can do a lot of template stuff. Here's just like the, actually the first introduction to macros and LISP was in 1963 by Timothy Hart. If you ever read the evolution of LISP, it's a really, really great kind of whole history thing. So I was reading this paper here, just to give you examples, so Erlang has a, has preprocessor macros like you do in C. Yeah, so where that's based on token substitution macros. But here we have, you know, in LFE, like we do in LISP, we have real syntax macros that operate on the ASTs. And I'll show you some cool examples of that now. So just a, yeah, it's a little bit, so one thing that comes with LFE, you have this idea of the back quote macro, which is a self a macro, and I'll show how we actually use it in macros. It's more like the tick. For some reason in the beamer output here, it should look more like a tick than a back quote. No, it is called a back quote. So yeah, so here's a quick, some of the talks earlier, we always look at like an example like the Rust one about destructuring. So destructuring is not something I always think about in Erlang very much, but with LFE, this is something that's really easy. So you see this match lambda here. Match lambda comes out of common LISP. I'm actually, I have this list where I'm duplicating the count and item. I can actually match here on the item count. This is an anonymous function. So again, match lambda. I am matching an anonymous function to get these values. So this is something that the back quote can do because basically like creating like a template for variables. Okay, so I wrote a macro. Like one of the macros I always really like from closure is the single threaded and the double threaded macro. Use them all the time. Basically kind of do like a pipe operations. So you don't have to have all the parentheses all the way go through. So I was like, oh, let me write that macro in this favorite Erlang. So this is an example of what it does. I'll come back to the show. But so I have here, I just define syntax rules. It comes out of scheme actually, where I can just define the syntax rules here. So like on the single threaded macro, I have, you know, I basically bound each of these things. If I have a list of, you know, I have one element and I have this cons of various elements where the dot dot dot just means what the rest might be. So you keep going through here and actually this is stolen as a racket example of the closure thread macros that I basically just almost stole the couple of little different things that I had to do. But it works, it works right off the bat. So all this power of macros that you have, you have at your fingertips with LFE. And that's really cool when you're thinking of it. Now you're gonna deal with concurrent distributor programming in Erlang. So you see how I call it right here. I'm getting the cutter. Obviously we're taking stuff from list again saying the cutter, which would give us two, three, four, five. And then we're just adding one to it. And you see this works, I, you know, test for it is the real thing. It's just that it happens. So cool things you can do, you can do this in the REPL. You can macro expand. So you have the ability to expand macros. So we have like really simple one here where we're just adding elements across. When I get six, if I do the opposite way of the double threaded macro, I get a pair. But you see here, so yeah, Erlang, when it's calling the Erlang, it shows the actual arithmetic calls. But you can imagine like as you're experimenting and going through things that you're writing macros and you can have macros that call into other macros, you can get a whole expansion tree and do a lot of interesting work with this. This is something you can't normally get in Erlang, but you get in lists, but you have everything together. Now we show that this is a list two. So it's unhygienic. This could cause issues with collisions and semantics. The whole point, there's work trying to do gen-simming. Elixir does something called lay binding, but they're trying to get gen-simming in, but it's gonna take some time just the way as it goes through dealing with Erlang. Yeah. Unhygienic versus unhygienic. Unhygienic, so yeah, so hygiene basically has a concept of having unique variables. So there's no catching of scopes. If I have a global variable and I have a macro, there's not gonna be an issue where that global variable is caught in scope, like is replaced by the one in the macro, right? So in the list two, you have this harder time with that. I have to go a little bit faster. That's, I got a few more. But at the end of the day, so the couple of other things that are really cool about LFE is that it goes into an intermediate representation called core Erlang. I won't go into the specifics, but like everything with Erlang, you can do so much where you can go, you can get the core Erlang with one simple call. This is some of the expressions and variables in that. So I have a simple function here. This is what core Erlang looks like. It gives you every line, everything like that. I'll skip the parser generator part. It's also really cool how right now in LFE, they have a hand-done table going through each of the elements, how they do the pars from LFE to core Erlang. They're eventually rewriting it to it's actually a real parser generator. L01 parsers are really interesting top-down parsers. So there's a lot of work going on in LFE. It's a really cool language, has a great testing suite. There's a guy named Duncan McGregor who's doing great work to kind of make the community really something to happen and you should hit him up if you're interested and talk to me more, we can run a repel and write some macros. Thank you. Thank you.