 Okay, welcome everyone to the Intro to Earling workshop a little about me. I'm Proctor I'm the founder and organizer of the DFW area Earling user group We've been going on for about a year and a half now still pretty small, but working on building that up Do you have a podcast functional geekery where I talk about various functional topics and? languages with different people across all the different languages and then I Have set up a fee a planet site for Earling called Planet Earling So if you were interested in Earling you can check that out and That'll be an RSS aggregator for you So about you real quick How many of you actually have functional programming? It's or let me rephrase that how many of you don't have any role functional programming and are getting into it Okay How many of you who have some experience have any Earling experience? Anybody coming in interested because of what they've been hearing about via elixir, okay? So some background for those of you don't know it was created by Freeman Joe Armstrong Robert Verding and Mike Williams Way back in the 1980s for telecom switches. They worked at Erickson. They had a little lab and Their mission was to create software That would be able to run the network telecom switches that they were designing Their goal was not to invent a language But to write the software and they realized that what they were doing was having to event language around that and so that's how Erling was born and Then it became open sourced in 1998 So it's got a long life and it's been around longer than a lot of other of the mainstream languages at this point But it's usually still pretty niche and it has been open source for a long while too as well Just a quick detour Since we're doing a interactive workshop The Erling shell for those of you have it installed you hit erl to get into the Erling shell For those of you have rebar you can also do rebar shell You'll probably have some problems if you're doing it from the problems because the template is not complete And it's got some stuff that stuff But you should be able to get in and start executing some commands But the real question is the real reason I show you this is how do you get out? Well first you see abort with command with control g So you can see user switch command and then you can see cue to quit Erling So that's one way you can get out or the other one is you do cue open print open print close print period So you should be able to see it and come back to your normal shell So that's the quick detour so as we go through this if you start getting in a weird state You can always kill your shell and come back in detours over so More about Erling and if you want to pull up your shell and start working on some of these things as I'm typing these in as I go over some of the stuff feel free to and then we'll get into the actual coding and other examples Erling it has a very very small syntax. It's got a small set of predefined data types also known as terms It's got pattern matching Variables, but they're not really variables in the way you think about it in most languages They've got function function clauses and modules and that's pretty much the extent of the most of the Erling language as far as the language goes Data types They're immutable and you have a very limited set of data types that you can pull from you got numbers We all know what numbers are atoms atoms are like keywords or symbols in other languages You have bit strings and binaries references function identifiers port identifiers Pids which are process identifiers a lot of these identifiers are just ways you can communicate through that and then how you find ports are part of Communicating with outside systems as well then you have list tuple and map and Map is actually really new in the newest language edition with release 17 But that's it. That's what you got. So you may be asking yourself. Where are my bullions? Don't have them They're they're they're atoms You've got a true. You've got the atom true and you've got the atom false and all atoms are are if you look at atoms They're just lowercase values You can do some special stuff with it when you get more advanced if you need more better characters by enclosing them in quotes But atoms just represent themselves and so we use this atom true to represent a true value and an atom false to represent a false value Where are my strings? We don't have those either in Erlang. Sorry The best you get are bit strings or if you want strings in the normal sense. They're just a list of integers under the covers So you can type in this 72 101 108 108 111 32 87 111 114 108 100 as a list You put the period at the end which terminates your statement and it tells you hello world So it interprets that list of integers as a string Where are my custom data types? Sorry don't get custom data types as you think in other and other languages as well Pretty much. They're just tagged or named tuples So you've got a tuple which is curly braces and it kind of looks like a list But it's a it's a list with curly braces instead of brackets square brackets So you just say hey, I've got this Muppet With a string Kermit and a string frog or I've got this Muppet. That's Fawzi and bear and That's how you signify those things and so you essentially tag your data to represent what it's supposed to be You may also use records records are just built on top of it So you define a record of type Muppet That's a name and a type and So when you create it you create them up it using Using the hash sign and then the record then the record type and you say name is Ralph and type is dog And it gives you that and you create one But if you look you can do a tuple to list which just represents a List under the covers and it and by doing tuple to list you can see that a record is actually a tuple Variables where are my variables? I said we have variables, but not really The problem the catch the reason we don't really have variables in Erling is they can only be bound once if you use that variable You've assigned it. You can't you can't reassign it in Erling So it's not really variable It's only variable to the extent that you can assign it that it only varies once and they all start with capital letter So variables So we can do a queue and we get an message saying queue is unbound We say a queue equals 42 and we get a result of 42 And we do a queue dot then this queue period to end the statement and we see it comes back as 42 Periods the end of statement marker. So It's the primary end of statement marker and it's really an expression everything in Erling's an expression not a statement as well and Then you can get into you've got a couple of other kind of Clauses that you can use and so then you'll see later. We have commas and semicolons as well, but The period is the big is the big boy of statement terminators or expression terminations We have pattern matching in Erling I don't know how many of you have dealt with other languages that have pattern matching I know there's a couple out there with Haskell and some others but We say 13 equals 13 We get 13 We say a the two the Adam a equals Adam a great. We get the Adam a We say the variable a equals 13 We get 13 we say 13 equals a This looks weird. We don't do this in most other languages because Your this looks like an assignment statement. It is not an assignment statement is a pattern match and we get 13 We say a equals 15 if we have this in our shell and we're following along and Uh-oh we get an exception error We would kind of obtuse if you haven't seen it before but once you kind of know what's going on It kind of it starts to make a little more sense Which says no match of right-hand side value 15 So what's going on is a we've set a to 13 and a is 13 and so essentially this is saying 13 equals 15 nope doesn't match these things don't go along together We can do a list so we have a list that's one two three We get one two three We can do destructuring and pattern matching via pattern matching of the list so we can say first Why third pattern matches against the Adams X Y and Z? Let me see a result with X Y and Z and we look at first first is X third is E and Adam is just represents itself So it's a unique value that just represents what it is So when you have an atom and they're all expressed in lowercase And so an X is X and that's a self identifying item And so a lot of times when you use atoms or you use them to To use things like tag name like names and tag two bowls or things like that and a bunch of other stuff uses Atoms under the covers Like function names and modules, but it uses that to look it up because of you express it But generally atoms are just atoms You can only but you can only bind it if it's on If it's before the equal, but you can pattern match and make sure that it matches on both sides So the a equals 13 if it's unbound it'll take the value of 13 But you can use it on the other side if you're wanting to match that whatever's in a Match is something else. So you could do a b equals 13 and then it b equals a or a equals b Because they match they represent the same thing. You say 13 equals a you do 13 equals a And it's gonna say a is unbound I was was it quite sure of the exact error message of it. So I wanted to pull it up in the show I'll just say you'd have Have the express so you would know what to look for. Thank you for pointing that out though So pattern matching you can also do multiple pattern matching So you can say some list equals a list with the variables foo bar Baz, which equals the list of atoms a b and c So you say some list what's in some list? Well, that's a b and c What's in foo a what's in bar? B what's in Baz? C and then you can also do list list destructuring. So the pipe operator is kind of the cons and Head operator as well So you can say head pipe rest of the list And you're gonna get one two three four five You're gonna see head and head is gonna be the first value and rest is the rest of the values Stop again stop if you have any questions if I'm going Yeah, so and that one two three four five comes from the fact that it's an expression So it evaluates the result of the pattern match, which is everything Yes Okay, sorry, let's see if I can okay this one Yeah, it's a pattern match so you're just matching the pattern so you can pattern match Yeah, so you can essentially say some list equals the destruction like if I want to get the full list Equals a destructuring of the list as well because sometimes That a b and c is going to be passed in via an argument to a function And so you can destructure and get the whole value at the same time if needed The two the two some list and the XY yeah, yeah, so we get head and we get rest and then That's pattern matching. So now we go on the modules. Yes Yes Yes, it'll create the atom if it's not there. So you it's just something yeah So we'll move on to modules and Well as we'll go into more examples, you'll see some other examples of pattern matching but Want to give you a good brief overview before I actually go into the codes when you start typing it out You kind of recognize what's going on modules. They're used for name spacing and organization They're the level of code reuse and They must match with filing and Then when you declare in your file The declaration of the module is the first line of the file you get aside from any comments that you have there But it's the first statement of the file as well as being the level of code reuse It is also the level of code reloading One of the more advanced topics that you can get into and you'll hear more about Erling is that it supports hot code swapping and It can hold two versions of the same module in memory at any given time And so a module is that level at which it knows that Everything here is going to be reloaded and reused at one point So you can actually run two versions of it Which is what allowed Ericsson to get nine nines of uptime using Erling on their telecom switches Some modules so you have some you have my Erling project and in that you have a source file and then you've got a File named my Markov chain dot URL so the module decoration just as a dash module and then the atom of Markov chain and parentheses Then inside of the module you also have export statements There are preprocessor directive that allows you to declare the API of the module So you it takes a list of function identifiers and You can have multiple export statements in your module So you would declare a module say I have a math function I have a math module and I want to export some slash one Some slash one's a function identifier Which is the function name and the slash and then the number after that is the number of arguments that function takes Anything that you declare in your function or in your any function you declare in your module that is not in an export statement Is essentially a private is a private function Exports are your public API that everybody can call So if you're working in OTP, which we'll get to in a bit you might have an API in this case start link add following word that takes two arguments picks next word after which takes one and Then you'll have another set of exports that represent your methods your functions needed For implementing a gen server behavior, which is an it handle call handle cast handle info terminate and co-change So you kind of document it that I've got an API. So this is what I expect everybody to use Then here's the exports that I need so that the gen server behavior and stuff can call my Can call my code That's that's the syntax of it. So it's a prepositive So directive so you do dash export to let it know that this is the export And there's no difference in one statement or two stations now Generally, you'll kind of split it out based off grouping of things So you we could put all these in one statement, but it kind of makes it hard to see what do you expect? people to call from a from your from what you provide as an API and what do you expect the Gen the behaviors that you implement to call Yes, sorry, yes, thank you the percent percent up there is a comment and The convention is essentially different levels of percents based off the different scoping. So usually when you start your File declaration, you'll do like three percents and give the overview of author and everything else and Then you'll go into APIs and then you'll do a single like a do it double for like big method block comments or something Or function block comments and then the single one for like a line or end-to-line comment So functions there as I mentioned a little bit ago They're identified by their name and they're already so the number of arguments that they take and they also allow for multiple clauses They use as a pattern match to match the clause that works and the order of the clauses that you declare are important The first one to match Wins so that's the that's the clause you're going to evaluate So if you have a clause if you have a match all clause up at the top Doesn't matter what you pass in if it matches everything it's going to get executed You'll never execute the clauses below it. So something to be something to think about as you declare your function clauses Those are technically seen as two different functions. So here we've got We've got some sort of my list URL sample file We declared the module and we export some one and So we take some which is a list and then we call the sum which is our aggregate Which is our aggregator function that takes the list and zero for an accumulator of how you and So then we come down and we pattern match on an empty list and the sum and We'll return the sum Otherwise we're going to pattern match on something that takes a head and the rest and Return and the sum and a sum so we're going to destructure that to get a the head value and the rest of the list And then we're going to pass in rest the rest of the list and then the sum we've gotten so far and the first value from that list and Here we assume numbers. So if someone passes in a list of strings, we don't can't count for that a little advanced stuff as you could add guard clauses, but there's also a Thing that says if you're going to pat you if you're going to do it Expect what you expect So probably we should I should probably have put some guard clauses in here Which we may see a little later in some of the examples, but it's definitely in the app code if you browse through it But it so you can do pattern matching that says I expect a list where a head is a number And if not, it'll pattern match and say no clause found like I can't find a clause where I'm pattern matching with a List with an item that has and then as the number at the first Calling a function to call a function. You need to prefix it with the module name colon and then the function name So you give it my list and some This is called the fully qualified function call if you're calling across modules This is how you need to do it if you're calling inside you can just call Your function and if it's a private function or something else as we see here, so this sum here Can call some without needing to qualify it because it's in the same module But if you're outside of a module You call it with the module name so it knows which some you're talking about That means we're going to export the sum function that takes one argument So we've got a sum here. We've got a sum that takes one argument Here and then we've got a sum that takes two arguments here and this is seen as a private method Because we only export the one that takes one argument That's your that's your how you're gonna call it if you don't export anything you're never gonna be able to call anything from your module No, it won't know it won't know how to export it you'll get an error saying it doesn't know what you're trying to do Functions there are occasional times where you might not need any of the arguments so if you have some function that takes a head value and an empty list and you're pattern matching that because You just need the empty list and you're gonna return okay, meaning you're done because you don't use that head value anymore You could put an under the convention is you put an underscore Before the variable name and that's gonna say hey, this is an unused variable Don't complain to me that I've declared it, but haven't used it now. We'll get into Markov chains. So the app that the template is is based off of Markov chain I don't know who's familiar with Markov chains here So a few people recognize the name so You can go to Wikipedia and look at Markov chain pretty much it says it's Kind of like a state machine that does state transitions based off What your state what state you're in and a probability of going to the next state? Really it just keeps tracks of everything and says okay, if I've got this state, what's my odds of going here? You might have an iPhone and I've heard other phones have this now as well is the predictive typing where if you type a word It'll pop up to say hey, here's the next word. I think you might want or here's the net one of three words I think you might want So it builds up a state of saying okay, if you put in this word these words I'm gonna know that these words are likely to follow this other word And I have a probability I have a 95% chance that say when I when you say I your next word is gonna be want or when you say Functional that 97% chance it's gonna be programming So but if you you could do you could have another set that says I've got a I've got a 1% chance that you're gonna follow that functional up with the team because maybe you say functional teams instead of functional programming on occasion But it tracks that and knows that and will speak pick one of those words according to its probability So part of the park Markov chain is we got a prime the Markov chain, so we parse out words and Then we're gonna iterate over those words and then create an association between the word we see and the word following it And then to generate a Markov chain We're going to print we're gonna start it with the word So we're gonna give it a word and we want to pick the next word based off the probability of word of the occurrence And then we're gonna keep picking the words until we reach the number of words to generate So we're gonna start we're gonna say here's your first word go pick me 25 words Go get me something that's 25 words long go get me something that's a hundred words long go prime this Another example thinking about this now the kind of smarmy usage of Markov chains is Some of those spam emails to try and get past your spam filters So they'll pull in text and say hey this makes it look like it might actually be something reasonable But when you read it, it's mostly gibberish get nonsensical But from the analyzing of some of these tools on Basic spam filters it'll get by those basic fans spam filters because it's like oh this looks somewhat reasonable But when you actually read it in context you're like okay, these are nonsensical So real quick is if you look at the source Here's the structure of our application so we've got a We've got a source config file. That's the Markov app source We've got the Markov app. Erl. So this is our starting app that we start up We got a suit Markov supervisor We've got a Markov generator supervisor The scp is short for supervisor, which we'll get into a little bit later when we go into deeper into OTP We've got a Markov generator We've got a Markov word supervisor. We've got a Markov word and Then we've got Markov and Markov is going to be our public-facing API that we want people to call when we try and do that So Markov is pretty simple. It just says here's what we want people to consume when they use our app So we're going to get into some coding. So it's going to be Little sorry that was the application structure review So we're going to do a function to add a word to a list of following words and Then we're going to do a function to pick the next word So pick a random word from a list of words So first if you go open up source dot Markov source Markov generator dot Erl You're going to find a tokenize method in there That tokenize method is going to be just have the atom undefined So it doesn't do anything and it just returns undefined. So we're going to fill it in And we're going to fill it in with a call to string The string module and get to call the function tokens in there And we're going to take that text in and we're going to tokenize it and we're going to tokenize it by a space a Tab or a new line now if we want to get fancy we can change how we tokenize those strings and do some other stuff No, it's not a white space significant language if you're talking about like indentation But the convention is for spaces as the indentation level if you look if you're going into an editing other projects So I'll give you a moment to fill that in Sorry, yes, take out the underscore there the under the underscore was there because It's not used at this point when it just returns undefined. So it's removing a bunch of warnings So we'll want it to be take away the underscore on it so we can actually use it the spec above the thought the above The function is yeah is essentially a rough type declaration there is Yeah It's not drip. It's kind of part of Erling but it's kind of not so Erling is a dynamic language. So this is the specs to allow you to do Progressive typing on it So what you can do is there's a tool called dialyzer and if you give it specs it will do What it can as far as not reporting anything wrong to you and say if you've got these specs declared and you're calling this other thing with these specs declared Based off the types. I know about I'm going to make sure that as much to the extent that I can that all the types line up So if you say this thing's a string and you're calling Something that takes this int and you're passing it in it. It's gonna say mmm. This these types don't look right to me go so the next So yeah in the same file We'll go update the parse text method so parse text takes text and And we're gonna take a first word and a list of words and We're going to destructure the result of calling tokenize on that text And then we're going to call load words with the first word and the word So here tokenize gives us a list of words and We're going to destructure this to get the first element out and the rest of the words So this one is in the generator still so there should be a load words template And so right now it's underscore word Empty list and then defined so we want to fill that in and we'll fill it in with This and so here's a function with two function clauses So the first so the first function clauses the first function clause does word Which I don't care about empty in an empty list and I'm going to return okay And if we look at this function clause what we see here is That this one actually has a semi colon so a semi colon splits function clauses and The period ends the function statement And then if we have multiple lines to a function we use the comma to separate lines from our function so this is an end of an expression in a function and then this is a This the semi colon is a end of a function clause And a period as the end of the complete function definition Okay, just says we're going to return something and so we just returned that We're good So everything's an expression so everything has a return value and so we just say hey we got to the state We're good. Okay, so it's just the atom. Okay, and so what you can do is you can then Do that and pattern match that says if I call this I expect to get an okay if some if I didn't get an okay and Some other form then I could Then blow up crash because something else won't wrong It's the it's an atom which is just represents itself So it's not it's a it's a value, but it's a value that represents itself, so if you're familiar with symbols or Or Keys and other languages You can it's that's what it represents Well from what other languages are you familiar with I was about to say I think Haskell has something like Yeah, yes exactly Yes, so we're gonna call through load words and as long as we keep getting Words and a list of following words We're gonna do it and then once we get to a word that doesn't have a following word We're done because we don't need to load anything else because there's nothing else to follow it It means we've hit the end of our end of our string Otherwise, we're going to call the add following word function and Markov word and We're gonna pass it the word and we're gonna pass it the following word So this is gonna be what's associating a word with the word that follows it and we're gonna and This is building up that state which we'll get to in a little bit And so then we'll do another example inside Markov generator Dot URL We've got a function add word to list and it takes a list of words and a word And all we're gonna do is this looks familiar. So this may look familiar Sorry, okay I looked at it. I was like I don't think that's right, but thank you Markov word that URL Yeah, remove the underscores because that underscores are signifying that you don't really use them on that variable delegation Yes It'll throw out a whole bunch of It's convention and it means something to Erling and the fact that it's smart enough that says it'll throw out a whole bunch of warnings to you saying You've declared this are you but you haven't used this. So are you sure that you're using like Yes, yes, yeah, so it's a convention that's kind of happened and has been built into Erling You'll find things like that in some other languages too where the underscores represent a convention This is I know this I got to take this because this function takes two arguments And it's the same argument, but I don't need any of them. So as I was starting to go on is this word Pipe words looks a lot like destructuring and Taking the first word off of a list Taking the first item off of a list and getting the rest as well, but it also works to add it to the list So doing this in this case Actually puts word at the beginning of the new list with the words following it So this statement here instead of doing a destructuring pattern match We'll actually build up a list for you. You can it's not really useful because it becomes a It becomes a lot a big O of n because you got it you're having to divorce the whole list So generally you're Prepending to a list and then if you're operating on some of this stuff you might call reverse on it Usually because you're building up a list of unknown size you don't necessarily want to It's a link list So essentially under the covers it looks like you would have a head and an empty list when you get down to a List of one item it'd be a head pipe empty list Yeah, it's not that So it's not that you can't but there's a lot of cases where it's just safer to append to the front because Especially if you're coming with outside data that you don't know about Or cases like this where you're getting in a string of text that's Arbitrary length so you don't know what words are going to be in and how many words Like how many times you're gonna have a in there with every other word in the dictionary following a kind of thing Now you're gonna blow up your blow up your list in your runtime of trying to append Uh Everybody makes sense to everybody here. What the my word to list Yeah, sorry, it's in the Markov word file. I and I have put all these slides in as a markdown file This is revealed JS under the covers with some plus so it's marked down So I can I'll just give you the repo at the end if you want to go back and reference these slides as well So the next one is gonna be in Markov word dot erl again and We're gonna fill in pick next word So in our case of pick next word, we're just gonna fill it in with pick random and We're gonna pass in words So the way we pick a next word at this point. We're just saying it's gonna be a random word And then here's the definition for pick random pick next word Just calls pick random It's gonna it's expecting that's one of the notations for saying I'm expecting a list of strings So it's expecting a list of type strings all the way down sort of the like it you So the definition of pick random Pick random is gonna take the list We're gonna get the length of the list We're gonna do a random uniform from that length and We're gonna get to get the index and Then we're gonna call list that nth which and pass the index and the list and that'll get the nth item out So we're gonna pick a random one from the list and if you kind of notice the way We're building up the list is we're just adding that word to the list So if we see that word multiple times that word is just gonna be in that list multiple times So that's how we're balancing it out. So if that word follows another word 15 times it's gonna be in there 15 instead of having something like that word and a map of 15 count So not necessarily the most super efficient thing to do but that there's a challenge at the end that says you can go clean that up like Go clean up the implementation of how we Stash in the word. We're coming up on an hour. So we should be and we're close to I think we're close to finishing this section of coding So when we're done with that we can take our small little break and So what you can do is if you actually have rebar you can do rebar compile and Rebar compile will take your project and compile your project into that You can do it outside of the repel so you can quit the shell if you do if you have it in the repel there's a couple functions of C and L but rebar will take care of figuring out where you're Where where your stuff lives? Does everybody pretty much have this? So let's go over here real quick and You can do rebar if you have it you can do rebar compile. Sorry rebar 3 compile There is a previous version of rebar and they're updating it to rebar 3 so and then You can do rebar 3 shell and This will open up and give you a live running shell and a repel with your code loaded So in this case you can just then you can just call any function that you've got so I could do So it's your full shell. Yeah, that's the next section. So We're good for a break. So we're now at OTP and we'll go take a quick OTP so if you follow the early mailing list, you'll see a bunch of stuff about what OTP actually stands for and What it actually means and I it originally was meant it was originally defined as the open telecom platform from from Ericsson Because when they open sourced everything Open telecom and platforms were hot keywords. So they just did it some people say OTP Just now stands for OTP and you'll occasionally see mailing list flare-ups saying Why don't we rename this to something else? because open telecom platform doesn't really mean anything but The name is not necessarily as much of the point as what it is and why you use it so why OTP so OTP as a set of libraries for applying lessons learned and building distributed asynchronous concurrent applications with high availability And I know what you're thinking to yourself buzzword bingo But what it really what it is because they had to design telecom switches and you can't and these are lessons learned from Implementing that and having to maintain things like nine nines of uptime because if you pick up the phone You need to be able to get a dial tone and be able to reach 911 or 999 or whatever emergency hotline in whatever country it is without fail and You can't have your cold dropped in the middle Unexpectedly if you're going through these landline through these old landlines There's regulations around it about having to be so it's got to be if one of the machines goes down It's got to be able to instantly respond and recover and manage it You can't take servers down. You can't say sorry everybody. No phone calls for the next five hours We've got upgrade our system So They've taken these lessons on building these machines that run everywhere across multiple nodes multiple data centers in the sense of data center of the telecom platform, but This is what it is Robert voting one of the co-creators I mentioned at the beginning There was some stuff floating around and I don't who here has heard of the CD 10k problem Okay, for those of you who haven't it's 10,000 concurrent connections to like your web server. I Saw a tweet from Robert voting a couple years ago. He's like I Don't get like wait. This is a problem. He's like we were like Like he couldn't he couldn't feasibly understand this how people were not doing this because of the experience He had with our line in OTP where The connections that he's like we were having this in 1980s when we were just beginning like the language So a lot of that is a lot of this stuff is very lightweight and they've designed it to be able to handle high concurrency That's part of the implications because they were building telecom switches If you had a 10,000 limit of phone calls going on you probably be gone out of business pretty soon So but first we're gonna go into the actor model So RTP is great, but we kind of need to explain the actor model a little bit first So I don't know who's familiar with the actor model or not So Few people so the actor model is essentially no shared state. Everything's done through message passing. So You can't have any of my information. The only thing you get is the messages. I send you everything's asynchronous and It's think of it like and one of the measure before is it is mailboxes So I send you something and it goes into your mailbox. I Don't know if you've got it. I don't know what's going on. I Send you a message you eventually get it and you're gonna act on it and you're gonna act on it at some point that I don't know about So the actor model in Erlang is implemented in processes So processes in Erlang are cheap. They're isolated. They have no shared state. No, nothing if you spawn up a process All that process has is what it has They communicate through message passing you send messages to another process There's links and monitors it allows you to know what at a higher level what's going on with the process And if a process dies, you can be informed of that So if this process dies, I can have I can monitor it and know that it's died and get feedback I can be linked to it and if it crashes I crash and then Processes the garbage collection for processes is pretty cheap and easy to because there's nothing shared that process Has only its stuff so you can do essentially garbage collection against the process really quickly because you know that Everything in that process is either reachable or it's not and you're not having this huge Object craft or memory graph across anything else that this state could be touched anywhere by anything No, it's pretty much isolated that process And when that process dies or hibernates or whatever you can know it's clear because nothing else can be referencing it So processes, they're cheap. They are not platform threads. They are not platform processes They're not like Unix processes, and they're not threads They are essentially green threads they're implemented by the Erling VM They're managed by the Erling runtime as well So Erling runtime will do the allocation about when to let each process run and give it so many times before it gets popped off Popped off the work queue and let something else go and it can take advantage of multiple cores and say Okay, we've got a bunch of these processes We've got a bunch of process workers that allocate and they can go back and say these processes They're going on this many cores if you set it up They're very small and if we can get to the end I will show you how small that they can be They are in the kilobyte ranges if that When you first start up a process now they can get big depending on how much state you want to hold But they're in the manner of I Think I've heard something about 500 words of Memory is the size of processes people have run millions of processes on a raspberry pi, which is not a huge Not a huge thing and they are very quick to create it does pretty much takes up no time at all to go spawn a process They are cheap I went to Erlang camp and I saw another presentation Martin Jay Logan who runs Erlang camp and wrote one of the co-authors of Erlang and OTP in action he gave a talk at Lambda Jam as well and He won he was he was he had a mantra that he was reiterating. He's like processes are not threads Processes are cheap and he said just repeat that to yourself over and over Because these are not the processes and threads that you think of When you think of processes and threads and other languages So they're isolated they have no shared state They can only access what they have access to anything that they have they can get they can't get things elsewhere. So Because you pass through the only way they can get something is if you pass it through and as a message So message passing You've got a concept of mailboxes So every process has its mailbox. That's where it gets its messages So messages must provide all the information that that actor needs to work on that process needs to work on It has only access to that message in a shared state So if something's not in that message that it needs it can't do its job This includes things like return addresses if I'm supposed to reply to you You better be sending me where I'm supposed to reply otherwise don't expect to reply because I don't know where to send it They have no view of outside world. This is pretty much sure there's a few small exceptions under the NVM But things as like a process registry just so it knows how to communicate to other processes, but that's pretty much it and They're all asynchronous communication. You send it it gets delivered at some point in the future You're not necessarily waiting around as in the peer process sense to get a response What you have to do is you have to say I want to explicitly wait and receive a message and do a pattern match on that receive So links and monitors monitors monitor just as of another process. Is it alive? Is it working? Links I'm dependent on this if this one dies I die too like I can't work without this so We're both we're both related Garbage collection they don't share a state they live on their own So it's easier to reclaim and especially when it's no longer used that process is done. It can wipe out everything So then we get back into some of the early behavior the OTP behaviors Behaviors are send it essentially a set of interfaces that are defined that you implement the stuff So OTP one of the nice things about OTP is It's taken all the common stuff that you would ever have to do in relation to building a server and a client Pretty much abstracted out everything that's generic and left you to implement the specifics via callback mechanisms So it stands for generic server It is the base behavior that OTP get all the other OTP behaviors are pretty much built on It takes care of the things that you would have to write yourself all those All those generic server behave all those things that you do when you write a server It takes care of that and all you have to do is your process So it also handles maintaining state in your processes And then gives you a way to do synchronist style communication over asynchronous messaging So you say I'm going to implement a behavior for a gen server and then here's my set of callbacks that I need So I have behavior of gen server And then the callbacks I have a callback of init handle call handle cast handle info terminate and code change So in it, this is the essentially for the next few of these This is what these are the specs that are that are defined so and it takes a series of arguments and It returns either okay ignore or stop and What you're going to do is it's going to reach if it's okay. It's going to return the state So this is how you start your process. You start it up. You give it set of args. It's going to do its little Little stuff like you might think of it as a constructor and object orange in the band But it's going to do what it needs to do and then it's going to return. Okay. I'm ready to start and here's my initial state Or you can specify a timeout or say hey, no, sorry you passed me something stop and here's why Ignore just says to ignore what ignore my words like you started but ignore whatever I'm sending to you Handle call so handle call you can do a couple different things, but what it takes in is it takes in a request? so essentially some kind of message Where it's from and the state you have so far What is your what is your current state? So to generally the most common returns are going to be a reply the atom reply they Whatever your reply is that you're going to be responding to And your new state so you're either going to take in a message and do compute states and return a reply That says okay. Everything's fine or you're going to say Do a query and you're going to go look something up handle reply and then pass in your new state again Whether or not you made a change or not You can give a timeout so if I reply and I don't get a response. I can do some other stuff Odds are sometimes there's some cases where I don't need a reply But generally if you don't need to reply you're probably not using a call You'll be using the next one and then sometimes there are messages that you get that say hey So I've received a message that says I need to stop and then you can say here's why I'm stopping Here's the reply I get I'm giving you back and what the state is or you can just stop with a reason and return the state Handle cast handle cast is takes a message and your current state So handle cast is when you don't expect to reply so I call I do a cast and I don't expect a reply to be issued so Call is for synchronous communication. So I Call you and I expect a response coming back from you and I expect you to respond to me Whereas if I do so if I do a call I'll call you if I do a cast I might do a cast and send a message to you and say here here's some information I don't care. I trust you to do your thing and I can go on and handle myself So generally we send it. So it's a no reply in the state We can also do a stop and a reason and then return the state as well handle info You're gonna send me something. It's gonna be some some kind of information message and I can handle it Probably until you get really advanced you're not gonna be really using handle info too much so just it's one of the callbacks you have to do and In a lot of cases you'll get some templates that implement that and you can leave that template behavior there for you Terminate if you need to get a terminate message if someone says hey, you need to shut down This is essentially your cleanup. So go take care of all your state if you've got a if you're a process that's responsible for a database connection then maybe If you're so you need to close out the connection when your process stops Don't leave that connection open to the database because the database won't ever think you've closed So terminate your cleanup area and then code change Unless you're doing something really complicated and really high availability You're probably not going to use the code change. You can get really complicated But code change what code change allows you to do is As I mentioned earlier, you can have two copies of the module loading We're at loaded at any time and so what your code change is is when you move over to that new copy Implement your code change you take your old version so you get an identifier you get your state and you can get some extra information and Code change is your chance to say hey I've got to transform my old state from this version into the state I expect now so maybe I've got to add some extra information and some metadata or some extra fields to this object So it allows that's how you can work to get some very high availability high up dime without actually and do a live code upgrades on Machines without actually taking down the server and stopping it But it can get really hairy depending on your especially with some real data So you may not use it that much because most of the stuff people do It's like okay If we need to take a couple minutes of downtime to go stop it and start it again We can do that but if you need to code changes how you do it and they've given you that ability As part of a gen server to be able to handle that case, which is nice. So you don't actually have to write That toggle yourself you just have to write how you're going to transform your state into your new state And so when you're done you just return okay, and here's my here's what my new state looks like from changing it You would do that Yeah, you would do that when you define when you create a new version of your module they're gonna have some module and version stuff on it and so in your code change you're gonna be rewriting your code change and Then you can do some pattern matching and stuff against what previous versions look like and how you get from one version to another and that So gen server when you're calling a gen server, we're gonna call it asynchronously and we're gonna Do the call function so it's gonna take a server ref which is Usually a pit you could be do some things like naming it and then a request and it'll give you a reply You can also specify a timeout if you need a timeout on it If you want to do it asynchronously only I did you just do a cast and so you give it the ref again And you give it a request and Request is really your message that you're sending so Jen server Really good practice is create an API for your Jen server Don't so why so 90% of the time how would your client know your server if if not more You're gonna give server if they're gonna either have to do a look up of the pit on their own and figure out how it gets mapped or a name or something else and That's not That's not nice form Also, if you change your server ref if you're running What if you change your request format? Giving a public API gives you something a little nicer that you can say here's my parameters And I can change my undercover implementation that Jen server uses and then if you do that all All consumers need to know it and all consumers need to have that update instead of having a common interface that kind of You can control your breaking changes with so Jen server an example is I'm gonna have an public API that says add following word And so that's gonna take in a word and a following word and what we do is given a word We're gonna find the process for that word because in our in our application every process every word represents its own process because it has its own state about all the words that follow it and And then we're gonna do a Jen server call With that word with that word process at the pit is a process identifier So we're gonna find the identifier for that word and then we're gonna call the request the request is going to be add following word with the following word So at this point the public API is like I can say add add a following word to a word So this here's a word and here's a word that follows it How it handles under the covers with the Jen server it doesn't need to know because that word really represents a process Yes, thank you That should be slashed to you so code time So we're gonna generate a reference for a process for a word So every word is gonna be its own process processes are light We're gonna have a bunch of words and a bunch of text. They're light. They're cheap We're gonna spawn them up each one's gonna hold its own state about the words that follow it So we're gonna then we're gonna start a profit process for each word seen and Then we're gonna add a following word to the state of the process So Inside the markup word URL. We're gonna have a fine process for word So we're gonna get a registered name for the word and that's gonna be our key and Then we're gonna do a where is which is the helper which is a little function built into Erlang that says Given the site that given this identifier given the server ref Where is the process for this? So if I come back with undefined I Know that I need to register this word This is a new word. I haven't seen before and here's my key and When it if I get back a PID and this is part of the pattern matching with guard clauses that I kind of touched on Glossed over earlier so I can get back a PID win is PID this value So I get I match on something I get a value and when it's a PID I match So if it's of a type that's a process identifier Then I'm gonna return that process identifier if I get back something else Something else has gone horribly wrong. I don't care and we're gonna we're gonna get an unmatched clause And we're gonna blow up with a pattern match Clause match error. You don't really throw exceptions There are there are a couple of cases where you might need to throw an exception But generally what happens is you don't throw an exception you code for the happy path and We'll get to that in a little bit when we talk supervisors because I don't know how much Any very good he's heard of Erlang or heard people talk about it. You'll hear people talk about letting it crash and if you follow some of the Erlang people I think Francesco Cesarini were one of his Erlang let it crash shirts when he was hopping aboard an airplane one time and realized the irony afterwards but The concept is you let it crash and we'll get into supervisors in a little bit But a supervisor just click essentially watches another process to know what it goes on and then can hint knows how to That's where you put the logic for what happens when something goes wrong Yeah, right now. We're not completely Managing concurrency, but I but at this point. I don't think this is in This is at the higher level API level so this is really gonna be only this is gonna kind of be single threaded anyway But yeah, we're we could have we probably could have edge cases There's a testing framework That helps find some of these things which is advanced it's But it'll try and run through Concurrency it'll like do concurrency tests and try and break your system for race conditions But there may be a small race condition here, but the odds of us hitting this especially for just demo purposes It is gonna be very unlikely So where'd you get so we're gonna find that name and then we're gonna look up that name So if we have a bunch of things going on at the same time where you might have multiple We might get into a case where we try to register the word, but it's already been registered But uh, and then we'll crash with it and saying hey, sorry, there's something already else with this ID When we register this ID if I'm remembering my next couple of slides right then that's right I may have messed that up, but yes Because everything's an expression this gate statement return something so PID When it's PID it's returning a PID when we do a register word That's a good catch that register word is gonna return the PID as well So we can when we call a fine process for word one way or another we're gonna get a PID back so We've got add following word So when we add the following word here's another again another public API type of thing We're gonna take the word and we're gonna do the following word Did I just duplicate that I? Shed this example a little bit ago. That's what it is So we find the process for the word We get that PID and then we call that with the word PID so that identifier That's our server ref and then our message is add following word If I can actually select it and Then the word that's following well PID is a way of having a server ref So it's a very common one. You can use a server ref by a name So in one of these exam if you look in the code you'll see okay I think did you ask about the someone asked about the question mark module or question mark server? Okay That is that's the name of the module and sometimes if you it's a very simple one You can have like if you're only gonna have ever one of these processes You can kind of allocate it by you can say my server ref is just my atom of the name You can do it and I think I might not have had us implement that one The get registered name for word that might already still be implemented So if you look at that you can actually say hey is this local is this global and here's an item Here's a name for it kind of thing and you can register it that way or And so if you know that you can use that if not you can get just say go find me by the name and get a PID back for the reference so server ref is kind of the generic if you look at the Documentation on the Erlang site. You'll see it server ref is Of one of these kind of types You get to go on so here's another one so pick next word after So we've got a word We find the process for the word again And we get that PID and Then we're gonna call Do a gen server call passing the PID we found for the word and its message is going to just be pick your next work pick the next word So inside mark off word so here's our handle call method So you can see we're sorry. Here's our handle call function. We you can see we've got a couple of different Handle calls so we have a handle call that looks at Add following word word. So this is one message. We have And then we've got a handle call that handles the request of pick next word So these are internal API so that we call And so it's the request if we remember back to that other slide that talked about the API that this invokes its request From in this case. We don't need the from it's gonna kind of handle it back because we're gonna just do a reply and We have a state we have a state which is a record of which is a list of following words so we're gonna say following words and Pattern match on the variable following words So and then we get create a new state Which is our state? with the following words to be equal to Adding the word to a list So we add word to list that method we implemented earlier that took the word and put it as the head of the other word And so ours new state just represents the following words. That's the list with that item added before and Then we return our return value For our function clause is reply Okay, a new state So we're gonna reply you told us to add a following word. Okay. We've done it So when you come back, you like if I'm calling that The response you should be getting expecting to get back is okay. Then we've got the other clause which says pick next word So we take our state we match our state So here's a pet. Here's a couple nested pattern matching here So we say state equals the state that we've got passed in the state record With the following words also pulled out and destructured So we've got the list of following words pattern match against the following words in the state and We're matching the whole state object the whole state record sorry Against that pattern match as well. So we're able to get both state and following words in the list of the state record via destructuring and pattern matching and So our reply is we're gonna call pick next word From the following words that we'd get and we haven't actually done anything here to our state and So we keep our state the same so we don't have really a new state our state is the state we got and Then our reply is gonna be that word we picked so we're gonna reply with the word we picked and The state is anybody still following along typing this in or Do you want to should I continue Continue so and then we can start a process for a word. So we have a start link function That takes a word key and when that word key is an atom We're gonna call gen server start link and here's where we kind of see how we register it So we register it as a local process With the word key. So the word key is gonna be the key We have and then we're using the module and some other arguments So our state and again our initial state is just an empty list So here's where we let it crash. So supervisors So supervisors are another behavior. They monitor their child processes They can supervise either worker processes or other supervisors They know how to handle the restarts So there's a couple different restart strategies if one dies Do we just restart that thing or do we restart everything else that we've started or they all dependent? and then Kind of what happens if a child process it manages that what happens if a child process dies So don't throw you don't necessarily throw a lot of exceptions. It dies You let that logic handle in the supervisor that says what happened this crashed. What happened? So the supervisor The supervisor just has a single callback which is an it and we implement the behavior of supervisor The init function looks like this It takes a set of args and it either returns an okay some flags and a list of child spec objects or child spec tuples Ignore or some kind of error that we got So this is the supervisor for our Markov supervisor So the restart strategy is a simple one-for-one. So if something dies, we just restart that one thing We don't need to restart everything all these words are independent if we lose a word No big deal You have max restarts and max seconds restarts max seconds between restarts So it'll say hey go restart these children if I get a more than this threshold of a thousand restarts in 3600 36 30 3600 seconds Then I'm done. I it's a problem bigger than me. So I'm gonna I'm gonna crash So the supervisor is the restart strategy the number of restart the maximum number of restarts and the seconds between the restarts And this this goes against this goes across all the children What kind of restart strategy is it? It's a permanent Shutdown so it's a timeout of how long it should wait for this child process to shut down and take care of its cleanup The type of the process this type is the worker You can have supervisor types as well for if you're if what you're starting as a supervisor And then the child spec is it's a Markov word It's in the Mara Joel Markov word. We're gonna call start link. Here's some arguments And then we're passing it out the restart the shut down the type and some other stuff to it And then we return okay with the flags and the list of child specs in this case we only have one kind of child we're managing and The simple one-for-one says these are all gonna be dynamic as well So you can either start them all up at the beginning or just they can say hey This is dynamic start them as needed So we got one-for-one Here's if something dies it dies, but we expected to be done pretty much at the beginning One for all if this thing dies everything else that the supervisor does dies as well, and we're going to restart everything Rest for one is anything that was started after me Gets restarted if I if I have a couple things anything that started before me That's fine. I've got those as dependencies But anything else that started after me probably has me as a dependency and get it gets initialized with my information If I die if my process dies those are all invalidated and those need to be restarted with the new thing And then simple one-for-one And simple one-for-one is what we use here where it's these are going to be dynamically allocated They live on their own if we if a word if a one of our word states dies It doesn't really cause us too much problems. We can just go reprimand or do something else So code time Pretty simple one for this is start a prize start a process for a word scene So we're going to register a word And so we do Markov word supervisor Soup and we're going to say start child and we're going to start it with the word So we're going to take the word we've got passed in and we're going to start it And we're good and we should expect to see a response and this is the tag tuples that I was talking about earlier We're going to respect to say hey your send me a response that says hey everything's okay And here's the PID that you got back So here's how you reference that thing in the future and then we return just the PID so Super that's the soup is supervisor it's local to a node and so what Erlang does is you can create a bunch of different shells and Those can all be nodes. So every shell you start represents its own node and all the nodes the nodes actually support a gossip protocol, so If this node knows about this node and you have another node So if you have no a and no day knows about b and they're kind of connected and they've been informed of each other You can do global which says when I register this is registered across all the nodes in this thing and nothing else Can I have this name? it allows you to kind of have a bunch of these asynchronous things that are communicating and split across multiple machines or instances and Local is just if I'm on this node. It's just local to that. It's a local to this node It's not outside of any other running instance of the node Yes Yes Acca I haven't really messed with Acca. I would say one of the advantages of Erlang over Acca is It's been around much longer and they've hit those processes They've hit those problems Acca is trying to solve years ago So they've probably got a bigger handle the other thing I would say over Acca is Acca or Acca.net there are processes that Acca is managing and the actors are still not as lightweight as the Erlang processes and So these are very very and I'll show it we got some time so I'll show you some of the size of these processes and one of the new tools There's a tool called observer that and one of the other things of Erlang over any almost anything else That doesn't match it is I'm guessing you're Brian, right? Are you Brian? No, okay Jared Okay, as Jared was kind of talking about is you can actually connect to a live running system You can have I can pull up my machine I Can get to a box and either on that box or from my machine if it's configured, right? I Can connect and watch what's going on on another on another process on another machine in another node And so I can inspect all that stuff. So there's a whole bunch of other stuff besides just the active model That Erlang gets you that I Haven't heard of many other languages that even kind of come close to some of the features that you can get for the Availability if you need it or knowing what's going on in a running system The other thing that Erlang does that's kind of an advanced topic is you can turn on tracing dynamically at runtime So you don't need to turn on and off live debugging. You can say hey I'm on a machine again power, but the great power comes great responsibility very Very useful, but also potentially very dangerous is Hopping on this live machine I'm going to turn on this trace and watch what happens and I can see when these things are getting called and what's going on And what the arguments to a method are sorry, what are the arguments to a function are or not? Yeah, you can kind of see that The other thing with a process graph that I wasn't sure if you were going with was supervisors So the other thing you do and if you're familiar with aca or aca.net You're kind of familiar with as you get a process hierarchy where you have a root process and what you'll see in the code is so we've got We've got a Markov supervisor. So this supervises a Generator so it's got a generator supervisor underneath it and it also generate and also supervises the word supervisor So you can build up a whole process hierarchy of Things that when they fail they bubble up and they command and restart and accordingly So you need a process hierarchy in that way, but then you can also kind of see Messages and things like that. So this is the complete one. So I'll do rebar So I take the tail the first couple paragraphs of I take the first two paragraphs of the tail of two cities So it was the best of times. It was the worst of times. It was the age of wisdom. It was the age of foolishness It was the epoch of belief Etc etc So I've loaded that into a variable and then I will start the Markov chain So I'm starting the application right now. The application is not running until I hit start. So now I started it so then I can do a So I can prime it with two cities And then I can do a generate chain and I'll pick I start with it and take 25 words and so The chain I get is it was the other way and short the epoch of hope. It was the best of belief It was the spring of times. It was the And I can do that over and over again and some of these get really crazy Because then there's times I've seen it where it was there was a king with a queen with a king with a queen with A king with a queen and it kind of gets looped in this catch because 50% of the time it's king and queen so it's just like randomly it's like so yeah Actually, here's one of them in general there were a king with a queen with a king with a plain face on the season and so That's the but what I can what I was going to show you with starting it off is I've primed this I could do observer start and This becomes pretty fantastic and you can do this on a product on a production system as well So we can see Here's the total block size of everything. Here's a bunch of information. What's running total memory usage is 23 max Processes I have 8391 kilobytes from processes So the max processes that are configured the number of CPUs that's using So I can go you can see things like loads applications. Here's the So here's what the application hierarchy looks like the process hierarchy. So I've got the Markov Supervisor The Markov Generator Supervisor Markov Generator, and then I got the Markov word supervisor and then it's got a bunch of children And so I can look in I can look at all of these processes And you're not necessarily gonna be able to read this fall cuz I can't really I get it So we can see all of these products. Is that semi-readable? So I can see a bunch of these words. So every time I create it I Instead of having evil or crystal or degree or despair as its own ID I Prefix it with Markov word to know what it is and then I can go into it and I can inspect this process itself And I can see what's the status it's waiting How much memory does this take up? It takes up two kilobytes of memory What's the heap size 233 bytes for my heap so messages I can turn on messages and watch what messages it saw Dictionary State so here's the state so I can see the state the state was was okay. That's not super interesting. I Can look at So I can look at the and I can go look at the state of the And I can see the state and open up the state and I can see the state looks like here's the state So it's got state Lord's throne throne Superlative present period other winter spring season season epoch epoch age age worse than best And so I can see what that I can go and inspect this live running as it's running on this system So if I go prime it again with some different text, I can go re-see what this looks like Yeah, you you turn on tracing and you can it allows you to manage all of that stuff So there you go questions and then Resources if you need the complete app if you didn't get through it all the way You can go to Steven Proctor slash Markov dash Erling on GitHub, and I've got the complete thing couple books designing for scalability with Designing for scalability with Erling and OTP. That's from O'Reilly That's by Francesco Cesarini and Steve Vinoschi and Then Erling and OTP in action as well as those are both good resources By Martin Logan Eric Merritt and Richard Carlson Erling and OTP in action is more of a prescriptive guide of here's how you do Erling and OTP Trust us. This is what you want to do designing for scalability with Erling and OTP is kind of a Here's why you want to use the OTP here's the in-depths and like What everything goes like here's all the cases you would probably not think about that Jens over it takes care of you and supervisors take care of you So those are some good resources right now. There's ending up. There's a massively online Open courseware from University of Kent by Simon Topson. That's in like week three But it's a pilot and now you can find the videos on YouTube, but he's gonna open it up and do another one for For later on so this is his pilot one so it's gonna be an introduction to functional programming with Erling So it's starting from scratch. I think it even covers complete setup No, so that's another thing that could be useful Contact me you have any questions. Drag me down here Twitter at Steven Proctor Steven dot proctor at Gmail. Feel free to email me. My personal site is proctor it comm so find me there Share your learnings if you get into or if you're liking any of this you start learning Erling Let me know planted Erling. There's a planet site that I talked about I've got everything that so far and if you are doing stuff and you're not on there. Let me know and I'll get you added It's a giant RSS feed From a bunch of different sites that I have that covers the Erling ecosystem LFE elixir Erling so you can kind of subscribe to this and subscribe to everybody else's blog instead of doing a many and doing a bunch of other stuff and Then challenges so limit to 140 characters for a tweet So limit to characters words and then maybe you'd say I'm going to limit pass in a tag tuple What's my limit? Do I do words of 25 or do I do characters of 140 and tag it as a tuple and then pattern match against that? Read the text in so prime it from a file. So pass it a file name and primary text Add an ability if you want to take this take what I have and you want to go play with it Add an ability to clear the prime text. So we prime it go wipe it out Find the list of process IDs to stop nicely Could do we could be another process which handles everything ETS table amnesia. Those are other storage things Maybe kill the supervisor and not nicely and make everything restart Try the different things and see what happens Kind of mention Twitter you want to full-on challenge is limited to 140 characters for a tweet Hook it up to the Twitter's Dave API consume in all incoming tweets and put it out Now that's a big challenge for you if you're feeling really Really aggressive, but want a longer term challenge project And most of all go out there and have fun with Erling and that is it