 Hello, everyone. I'm Abhijit. I work as a back-end developer at Housing. I, alongside with my colleague Pranav, here to present about how we're using Erlang VM at Housing, what our experiences and learnings have been from the virtual machine. So I guess everyone here, I mean, they must be probably aware of the Erlang program language and the virtual machine. But still, just for sake, Erlang virtual machine, Erlang program language, they were both invented by Ericsson for designing distributed, scalable, and fault-oriented systems. So Erlang uses the Actor model to achieve concurrency. And the virtual machine has excellent in-built support for SMP scheduling. That means it can utilize multiple scores of a machine very efficiently. And it has been doing that since its inception. Erlang is an excellent language for programming distributed systems. So actors in Erlang, which are Erlang processes, can be scaled to multiple nodes, and they can act in the same way as they act in the same machine. I'll specifically talk about Elixir, which is a Ruby and Clojure-inspired syntax, and can be compiled directly upon the Erlang virtual machine. So I was talking about the Actor model. Basically, there are two types of model to achieve concurrency. One is the CSP paradigm, accommodating sequential processes, and one is the Actor model. Languages like Go, Haskell, they follow the CSP paradigm, but Erlang follows the Actor model. So in Actor model, what basically they do is they'll you can think of actors as objects, although that's completely untrue. But for, I mean, for programmers, for observational programmers, we have to think of it like that. It's not like that, basically. So each actor can communicate with any other actor. So you can think of actors like people who are not having a telephonic conversation or not having a face-to-face conversation, but communicating over emails, communicating over some kind of asynchronous messaging medium. So each actor, each actor in Erlang Virtual Machine has its own mailbox that mailbox receives messages from the other actors. And each actor has to read its mailbox and decide what to do after reading that message. So that's how it works. So you have multiple actors. They will dequeue their mailboxes. They will decide what to do. So yeah, I talked about this earlier. So actors in Erlang can be transparent upon note. That this means that the semantics of messaging between actors is remains the same in the same machine or across different machines. So if you want to say PID, process ID 1, messaging process ID 2, that remains the same for any distributed PID or in this local PID. If you have to think about the problems in which you have to send messages to an actor on a different machine, the semantics doesn't change. That makes this distribution dead simple. So this is a brief overview. So in the operating system, the Erlang Virtual Machine process will be a central process. It will have lots of actors, which are processes, lots and lots of actors it can have. Now what the Erlang scheduler will do is this will schedule those actors upon OS threads. So these are OS threads. And then it's the job of the OS scheduler to run these threads on the CPU. So basically, the scheduler, the same piece that I was talking about, so the scheduler has to be very fair in scheduling these processes upon OS threads. And that's what they have tried. That's what they have tried and successfully done. So we can exploit this feature of Erlang VM. Now what are the usage contexts for housing? So we all know that we have to build products very fast. So we are a Ruby on Rails stack, simple as that. But lately, many services written in Ruby on Rails started facing a lot of bottlenecks and a lot of scalability bottlenecks. So then we decided to chance upon several alternatives, maybe Go, maybe Haskell, maybe any other language. So the thing that we chose Elixir, so it is very similar to Ruby. And we thought that without taking much time, we could code our services in it and exploit the concurrency features of Erlang VM, thus making our entire web service architecture scalable. For interact, now we have lots of microservices at housing and these microservices have to communicate with each other. Now for communication, we could either use HTTP, but that is not a scalable pattern for communication. So asynchronous communication is best achieved by messaging queues, so we use RabbitMQ. So that in the RabbitMQ also has our language, is built upon our language machine and CouchDB is used for some use cases. We'll also talk about that. So this is like the code written Ruby, the code written Elixir. So you can just see how similar it all looks like. So the inventor of Elixir lang, Joseph Alim, he had this primary aim that writing Elixir should be as simple as writing in Ruby. That's what its aim is all about. So you see class becomes def module, def becomes def do, puts becomes IO puts. If you want to call any Erlang module function, you can just write column, the name of the Erlang module, the column name of a function of the, you have to call an arguments list. It's as simple as that. For lazy operations, you can use stream module for Ruby has an excellent inner module. If people, you know, you have program in Ruby, it's very simple, very intuitive. So the same way Elixir has also copied Ruby in that. So this is how one of our rewrite looks as. So this is, I'm talking about the main filter service. Now, first it was written in Ruby, how it looked like. Now it has been written in Elixir, how it looks like. I mean, the screen is not particularly otherwise, you'd agree that it's not that different. So here we are actually mutating objects. Mutating objects, this is like an object. We are mutating the hash of the object after every operation. But here we are not mutating. We are transforming the value of the function. So transforming piping to another operation, piping, piping. So Elixir has this concept called pipes where you can transform one value to the other. So you have to always keep in mind that Elixir, since it's an Erlang machine, it has the concept of immutable variables. So variables are not mutable. So you have to always keep transforming those variables to new variables. So what are the experiences of using Elixir? I mean, it has been an excellent choice, turned out to be an excellent choice for us because the ease of usability, given its ease of usability and productivity gains, yeah, it has turned out to be a great choice for us. Now, there's a problem with, I mean, people who have been coding in Erlang will agree that rebar was a big problem for making releases, builds, and generating all kinds of DevOps stuff in Erlang. So that has been perfectly solved by using Mix and EXRM. So that is not a problem. It's as intuitive as go, go build, or go run. People who use go build, they agree to the fact that go run or go build, go generate is pretty intuitive. So in the same way, Mix aims to be as intuitive as that. So you can do mix depth.get, you can do mix test.compile, you can generate releases pretty intuitively. Test cases are very easy to implement, as easy as Ruby. So you just have to use EXUnit to use test cases. That is not a problem. The problem comes when you have to think about the problem in a functional way. So you can't think about classes and objects. Never, you can't. So you always have to think about, okay, this function, can it be done in a process? Can it not be done in a process? Or do I have to multiple process and use pipes? So you always have to think that way. So maybe sometimes we make mistakes in thinking that way. So that is a bit problematic. Until you get comfortable with the language. Also the library support is not maturing. I mean, it's maturing slowly. So we have had hands full sometimes to make it exist or work for us. So we have to code. You have to have a Redis connector or a Postgres connectors. They have some problems. So we have to go in the library itself. We have to code and make that thing work for us. So that was an issue of this. So the second context I was talking about, hello. RabbitMQ, so these are many services which want to send messages to lots of other services. So RabbitMQ is a standard message broker for multiplexing different messages to other services. So RabbitMQ, you almost obviously know about it, but still is a message queue which has any of the concepts of a publisher, consumer queue and an exchange. So publishers normally publish messages to exchange. Consumers listen messages from the queue and the exchange and the queues are bound by routing keys that is specified by the client itself. Now, we found topic exchanges to be pretty suitable for us because using topic exchanges, you can extend the flexibility of rejects pattern matching and then we can agree upon some conventions and then we can have generic publishers across all our services. So let's say you have a service A, let's say you have a model A and then you have an action of the model A and then you can, let's say, let's say you have a buy service in our system and okay, I'll speed it up. So, I mean, using routing convention topic exchanges, you can have generic publishers, that's my entire point. So generic publishers are pretty easily, you can embed it easily across the entire services. So, I mean, also I want to suggest about how some catches that you might fall upon when you're using the virtual machine. So always write tail because it functions. Never in our line, add as a list. So in the exit, add as a list. So list, link list are always inefficient. So always try to append on the head of link list rather than trying to merge it from the, trying to append it from the end. That's the basic, most basic point. Large files in Erlang are always reference counted. So Erlang has the concept of soft real time. So soft real time means that each process has a local heap, that heap gets cleaned after each processes dies. So the garbage collection is incremental. So that Erlang, so by that Erlang can give you soft real time behavior. There's never stopped the world garbage collection but that can happen when you have large boundaries. Then Erlang starts sharing that boundaries across all the processes. So they would copy all the boundaries. So they have a central boundary and they can have a reference to that boundary. So those boundaries are always reference counted and reference counted. It means that obviously at some point in time when you have large boundaries, stop the world garbage collection will trigger and you might face problems in production. Yeah, so some of our messages like polygons were pretty, pretty large. So we tried to compress them. We tried to make those messages small and fit that into the Erlang process architecture. And distribution is one thing. So if you have connected nodes, Erlang connected nodes, so you will try to ping one node with each other and try to establish a connection with each other. But always have this thing in mind that when anything breaks, so what is your failure? What is your fault or this mechanism? How do you want to start with a clean slate? So always those things are very important when you're having connected cloud of Erlang machines. So OTP gives excellent behaviors for achieving fault tolerance and you have to think about, you just have to think about how you can start with a clean slate. That's it, guys, yeah. Any questions? Hey, hey. Oh, yeah. Yeah. Hey, I have pins. Me? Yeah. I have two questions. Like one is like you are following up your asynchronous model in terms of factors and things like that, right? So any thoughts on transaction management that we're interested on that? So transactions are always implemented through a pipeline. So we have a kind of FSM, FSM, we make FSM for transactions. So even each part of, so we break transactions into events. So that, like you can model a transaction as state machine. So always after a failure of a transaction, you can go back to a state which you have clearly defined. So it provides excellent behavior. OTP has a FSM module, which is pretty easy to code in. Okay. And I wanted to know the thoughts on the choice of Erlang VM itself. Did you guys explore the choice of JVM probably with Scala and Akka for? Thing is that, yeah, we love Ruby. I mean, yeah, I mean, it might be slow. It might be single thread and all that stuff, but I love the Ruby syntax. So I wanted to have some kind of syntax like that. So Elixir provides me that syntax. And most of the programs in housing love Ruby. So yeah, I mean, it's kind of stereotype. It's kind of preconceived notion. Yeah. Okay. Hello. Yeah. Just, I don't know, are we? Yeah. Here. Thank you for the excellent talk. I wanted to understand, like I'm curious to understand the tooling support. Tooling support. Yeah. How is it for Erlang? The tooling support was pretty good, but it was kind of unwieldy and obscure in Erlang. So if you have used rebar, if you have used, if you have used other Erlang.make, so you have to dig in quite deep to actually make, to actually do easy tooling. So with mix and EXRM, that is being done pretty easily. So Elixir has tried to solve that problem. And I believe that it has been solved to an extent. They are solving this problem, obviously. And is there any operating system constraint? Like, no, absolutely not. Okay. It has both windows. Yeah. Yeah. Yeah. So you said that you went to Elixir mainly because you had problems in scaling room. Yeah. So do you have some internal benchmark sort of approximate numbers? Yeah, we were. The thing is that we can always scale. So did you like first try things like event machine and do you have numbers to compare, say event machine based code versus Elixir? Yeah. Can you say it? Yeah. So I mean, with the same amount of Ruby instances, I mean, for the same amount of load, we are having a lot of Ruby instances around 10 or 11 for one of the service. And for when we installed, when we, I mean, it was hardly three to four days of coding Elixir and then it was, it dropped onto two or two. But you had, you were using event machine based code? No. In Ruby. So had you tried going to Ruby event machine? So yeah, event machine, no, I know, yeah. We didn't try.