 So who am I? Actually, this picture came from the Ruby Midwest that I met PJ at. A lot of people started saying that I look like 10-10. And since that Ruby Midwest, my ad is on most websites and Twitter is 10-10. On Twitter, it had AdKron. Had that since I was like 10 years old, which according to PJs, like a couple years ago. Is that better? I own and own is a big term. I freelance mainly for four years. I really love it. And I also have been doing a podcast called Disaster Life for four years. And it's been a whole lot of fun. I invited conferences to interview people, but not often to speak. And kind of like Johnny. Johnny has seven kids for you guys that don't know. I like to spawn lots of processes too. That's my family. I have five kids. If anybody else has quite a few kids, you understand that this picture is kind of a magical feat. Everybody's smiling. Most of them have their eyes open. In about five seconds after this, it was like Dante's in front of me. No one poked me. He punched me in the face. And so it was great. But I realized after spying lots of processes that we really are a relapse. This talk's not about relapse though. I just thought it was fitting. And that's actually what most of our family pictures look like. Somebody's scared. People laughing. But really this talk is about being a hippie. And what does it mean to be a hippie? Well, festivals, obviously. And that kind of conferences. So we're all part of the way there. And a lot of it is living free of constraints and allowing yourself to just enjoy life, stay away from the man, do your own thing. And making your own things. Like clothes, jewelry, full-tolerant applications. So I guess full-tolerant applications. Let's start out. I'll get to the meat of this now. You might have kind of an idea of where I'm going now that I'm talking about living with freedom. But recently I was working on some application and I had to write a password reset system for the application. And this is kind of what I had. It's pretty simplified. But I have a supervisor with a password reset and it talks to the database. And I started thinking about that data and what you store when you go to a password reset. So someone comes into the site, they ask for the password to be reset. You set a token, a token to be temporary. And I didn't want to pull in another dependency, a key value store. So I just threw it in the database. But I wasn't really happy with it because I knew that that data was just going to go away in a short period of time. I didn't like that column sitting around like Josh said. I don't, null columns aren't great unless you have a reason for it. And this was going to spend 90% of its time null, hopefully. And it's very short lived data too. And the database null is that fast. I'm not saying it's going to be that slow, but I kind of did a kick out of shaving just in the middle of seconds off of things. So if I could get rid of that, that would be really awesome. So I think Josh already left. Is Josh here? Oh, Josh, I put this slide in for you. Josh, the database is not your friend. I'm not saying to get rid of your database, although that might be an idea. So I started learning Elixir on and off a long time ago. And throughout, I saw some talks by Sasha and he often has picture of blue background, yellow squares of processes. So I'm going to make that with some sticky notes on my wall to give off this. And because this to me is what I thought of when I thought of using Elixir or lanes, all these little processes talking to each other. But I think Sasha has something missing whenever he puts this picture up there. There's nothing here. They might do things, but there's no data. And I heard you can store data in processes. Does anybody know what these are? Anybody have a car that you could put these in? No, nobody's going to admit that. All right. I don't believe all of you. So this is what I thought of. These are still all much processes that have some data in them. And you can use that data, but really, I mean, it's worth putting all this work in. People have made it in cash. They made Redis. They made a lot of different things that you just pull in and use. It wouldn't be that bad. So why waste your time to put this into your Elixir program yourself? And I said, because of TDOS, I hope you guys can read that. I don't mean distributed denial of service, although I might have played off of that intentionally to try to build up some emotion in people. But it's an acronym that I use for dependencies. So you have libraries and other things that you have to pull in that may or may not be being maintained. So I wanted to reduce dependencies. Deployment, which Paul talked about deployment. I didn't want to figure out how to configure something else. I didn't want to make two new things talk to each other. And then something else to monitor and pay attention to all the time. Is it up? Is it down? Is it taking up too much memory? And I needed less knowledge and less experts if I could stay in one location. Another option is optimization. I mean, what's faster than going out to your database? And I frequently get in-memory key value store. Remember GTIs, Redis? What's faster than that? How about not having a wire protocol? Not having to change data from one format to another just so you can pass it around. So you can get rid of that. What's faster than an in-memory database? Your current application having it in memory? That works out pretty well. Security. And this is where a lot of people are like, well, what are you talking about? For every dependency that you pull in and every hole that you have to poke in a firewall and everything that you have that you're pulling into your ecosystem, you're increasing your attack footprint. So I have a report open or I have this new software and we're already going to hopefully keep up with the Elixir releases, the early releases after moving along in our application. And a lot of these external dependencies get forgotten. So you miss the security output on Redis, which actually, I'm not trying to knock on Redis, but when I looked up some stuff about this, Redis had a security problem that they put out that they had when they said check the change log and there was a link and you click the change log link and it went to Reddit and that's where their change log was and it just made me feel a little uncomfortable that Reddit is where they said it's where their change log. It's not in their repo or anything. And I also looked up Shodan or shodan.com is a search engine for Internet of Things. It's also a really great search engine for finding open databases. And if you type in Redis in the US alone, recently I did a search and it came up with 6,550 insecure instances of Redis and a lot of them had a key store that was called Crackit or something similar to that. I should have written that down. And that came from a problem in Redis that was a couple of years old. So people still had broken Redis instances from a couple of years ago and it just pushed back to me and said well I hope I didn't install it on a base because it probably did. And then the closer that I can get to not having these external dependencies, the closer I can get to a Unicurnal system. So a Unicurnal is a machine image and when you deploy it's a lot like deploying to an embedded system and it's a pared down Linux kernel that has nothing really your app is the OS. Sometimes you don't have a lot of system calls and Elixir and Erlang really live themselves well in this. You can hop in remotely and get into IES and have kind of time on your system. So you don't need many services. You can be more secure. And last but does anybody know this guy? Speaking of that conference. Who knows who this is? Somebody. Uncle Bob Uncle Bob gave a talk at Ruby Midwest in 2011 and it's called architecture of the lost years. And as I was thinking through this I popped back to that talk which for the most part I just thought was entertaining more than informative. But apparently it's made me learn a few things over the years. And he said put off persistence as long as possible. You're probably not going to need it. And I'm paraphrasing a little bit about it guys. He's nuts. I still think that. But so I was like well okay I've got a database I don't really want to reach to it. I didn't go for that external dependency. I really wanted it for you. I want to see if I can loop it out of the database. So I started talking to a friend about it and he said that doesn't sound like a hippie. You sound like a hoarder and I thought oh what? That makes no sense to me. He's like you know what that kind of hoarder? This kind of hoarder. I don't want to maintain that application. No way. So we settled on it. And really it's this kind of hoarder. Those are old video games. I think I might be comfortable living from there. But the thing is to be this kind of hoarder you have to figure out what you're dealing with. You need to classify your jump. You need to make sure you're putting it in the right place. And then you need to protect it because I'm sure there's some people that might break into that house too. I'm going to make sure it's good. So first thing I wanted to think about was what are the types of data? And I came up with two. There's permit and there's temporary data. So permit and data I came up with two ideas. There's static permit and data which is config files or things that you would actually store in code. Things that in order to update you would probably do exploit. They're not. They're rarely changing. Hopefully every week you're not changing where IP addresses are or anything like that. So those things are, they're not good for a game base. They're not really good for res. They might as well stay in some static files. And then there's permit changing. Medical history, comments on a blog, users last name. It does change, but probably not that often. So those are pretty decent things for you to store into a database. And then temporary data. And I was actually asking people like, with temporary data what does that mean to you? And I kept getting these three words. Temporal, transient, ephemeral. Well, at least all the same thing and people would argue back and forth. So I decided to look them up. So the first one, temporal is relating to time. And that seemed like a pretty good thing to store in the process, like weather. I actually thought that the password reset token is semi related to this. I mean most people don't ask for a password reset token and then come back six months later and reset their passwords. It's usually a fairly immediate thing. Unless you're like me and you remember your password right after you request the token. Transient is lasting only for a short time. So at first I thought the word transient was like transporting. But then for a short time a lot is cache. Cache would be a great piece of transient data. And I thought this because when I looked up ephemeral it said lasting for a very short time. People argued with me over these being different. I was like, well they're pretty much the same me. So we decided that ephemeral was things being transported. They're very short periods of time. So, alright. Got our app. Got some background information. So how do we actually store data? And that's agent to the rescue. Agents are, if anybody's not used them, it's based on a GIN server. And it's a great for storing simple data structures. No conversion required protocols. Right inside. And it's pretty quick. So that's what I pulled out. I have a password reset. And a password reset horde and a supervisor. And I probably should have swapped these so that the horde is where the data is to work. That's our agent. I probably should have swapped them so I could start out first. But this isn't software. It's a picture and it's too hard to swap. So now the thing is about this is we are likely to have failure somewhere in this system. And with a single supervisor like this, a password reset has a business logic in it. Which is where we're most likely to have failure. If it fails too many times, then that supervisor will end up shutting everything down. And that's not really what we're left with. Let's go ahead and look through the code of an agent. So the agent, whenever the server starts, the function up there that's passed in, the return value of that function is your starting data that is inside of your agent. And so we're going to get an empty map. Because a key does storage the map. I don't need anything big. I just need a key to the user. So in ad, somebody had an agent up here yesterday and they had an update there. And that's what I started with too until I learned about cast. It's so update and an agent, you tell it to update the data and then you wait for it to send it back to you. It's a call. But cast, the second argument there is function to put the token and the user that's passed in into our state inside of our agent. So it's saved. It's set in forget. I don't care. Just go ahead and put it there. And with a password reset, even if there's a problem and you lose it, somebody can come back and click that button again. So that's not going to heal. And then we have get user. So again, we just fetch out of the map based off the token that's passed in. So a crash from the state. And then a remove. Because after they reset the crash from the token, we don't want that token to be reused over and over. Somebody else gets their email and clicks that. Change the password. But like I said we could have crashes. So what about crashes? Does anybody know this book? Alright, so this is all about the agent. I wanted to be able to live that. Let it fail. And so I had to find a new hero. Agent wasn't going to do it for me. And this little guy down here in the very bottom he's the protagonist in this book. And his name is hero protagonist. So I just had to put it up there because we got crashing right here. So my hero is Jen Silver. And I changed the supervisor patterns here. And this is really important on how this works is that I have some failover data. I have Horde and I have password reset. So Horde here is what is about to change. And failover data is something new. And you'll notice that we have a separate level of a supervisor in here. And since we have a separate level supervisors by default have a certain number of resets in a certain amount of time. And it's three resets in five seconds before they just fail everything. So in our case now we have three resets in five seconds from the supervisor for this. And if it dies that top supervisor will restart this one and do it again. So now I can have nine resets in 30 seconds. And we still from that password reset I'm always going to return okay to the user anytime they ask for anything. So I can cast everything. I can put everything in the background. And there the selection of the user. So response times when I tested this went from 120 milliseconds, which is very good to 56 microseconds. So that's way better right now. And the thing about GenServer is it allows you to have custom termination logic. But let's show the failover. It's a lot like what we had before. My failover data starts out with an empty map. I'm still using agent here, cast to add state. The function for dumping the state normally takes the state in but we don't care what it is we're just going to override it. And then loading the state back out. But here's the secret sauce here is the GenServer. So the GenServer I'm leaving most of the code out. There's a whole lot of boilerplate in there and then the add remove and get user are based off of the agent that we had before because there's not a whole lot of change. But in the init, instead of setting data in this guy to an empty map when he's restarted we instead use the failover storage that was back here and we say hey I want your state so I ask for it. I get that state but then I'm terminating and this is what GenServer gave me over agent is on terminate I get the last state that happened if there is an exception if the supervisor shuts it down for whatever reason and I dump that state in the failover storage so when the supervisor starts me back up I get that init again and he gives me that back. So the big thing here is you can do many other things in this terminate callback and munch that data you could write it off to a file so that if this is failing over and over you then have something to use hey maybe it's failing because my data is bad let's go see what that data is. But again I had failure and I really didn't want that hoard even though it's going to dump its data off I wanted to give an extra layer and this is really the crux of all of this. If you want to start living like a heavier order and being free of these constraints you still have to watch out for yourself and watch out for yourself you want to keep things that are going to fail often and push those to the bottom of your supervision trees. Whenever you're designing the things near the bottom I should have told you earlier that we can reload over and over based on the supervisors so I have more chances of failure before I have a problem and I ran into this and the reason why this has saved me is a database where he said shut down the server and came back up and I ended up shutting down my whole application and I didn't know for a little while but after I had this in it gave it enough time that that server could come back and tested shutting it down some chaos monkey type stuff and it worked out really well. So you have more chances for failure and then the only thing you have to deal with is catastrophic failure so what happens when somebody unclubs the server or something else on the server just shuts down so I don't get the term a callback and that there are ways to handle it but I'm not going to go over those today because there are some other connections here we have expiration so I wanted the password reset token to all of my labs for 24 hours and I've since moved on to the password reset token generated as a signed key, an encryption key and I don't even have a story but this is the bandit for other data that I've been using but expiration wise too if you wanted to stay within Elixir itself there is a library called cachex that I found it builds a cache with time to live functionality and it's very configurable it's been pretty fantastic to use I haven't been able to dig into everything that it does but it's got it's got some really good stuff I suggest checking it out the other thing you can do is distribute your data so using Amnesia or just another Elixir process on another machine like if you're already low balance you can push your failover out to other machines on a regular basis and that way if your whole machine goes down when it comes back up it asks the other machine to give me that and it does add a wired protocol back in but it's it's all still a regular Elixir and you're not pulling in new things and really if you want a good understanding of how to deal with really distributed data and deletes last year at Elixir Conf EU Chris McCord gave a talk on Phoenix 1.2 where he discusses Phoenix presence and how they implemented all that and it's a very good list to dig into it's probably way more complicated than the data that I wanted to store so I might keep all that in the database because and then binary data so binary data garbage collection is pretty well documented this can be an issue within early it's done differently if you there are 64 bytes of magic number anything under 64 bytes is stored on the heap just like every other process and whenever that process goes away that data goes away it's just cleaned up if it's over 64 bytes it gets stored what's called PROC bit and it's reference counted and the reason why is whenever you try to break binary part instead of it copying a data over it puts the pointer to original 64 byte or larger binary data to a section and so since it's reference counted has a completely separate garbage collection from the rest of Elixir garbage collection and it will shut down your system pretty easily I've read into that garbage collection doesn't kick off until you have a memory pressure is what it says in the early documentation I found out it's about 90% of my system memory and usually before the garbage collection can finish my system crashes it doesn't have a memory so be careful if you try to store any binary data at all make sure it's under 64 or really pay attention to what you're doing with it store domain objects instead of storing binary you convert it into some domain knowledge and store that to work well better for you so the next step in order to be a little more fault tolerant is ETS and DTS, Erlang Term Storage has been brought up a few times I've heard people talking about it and disk Erlang Term Storage so you can still store native Elixir things in there but the great thing about using those two in combination for your failover is that they use the same API so I can take the same module and inject whether it's using ETS or ETS and use the ETS for failover or more more data right into this and ETS for my regular memory and just write it off to disk which also gives us that if I crash I have a known state somewhere on the drive that I can look at Amnesia, I don't haven't dug in a whole lot into it if you go Elixir school has a little section on the top that has four questions that tell you when you might want to choose Amnesia over ETS and so that it's more distributed and has some different failover issues built into it that you can deal with and then really my next step that I think is I want to write an application with NoDB and I really think that everybody should try that you should try this I have an invoicing application that is all Elixir and I think I'm going to rewrite it with No Database hopefully I don't lose my invoices if I do have some really happy customers so let's go over what it was, how to live like a quarter, so agent's pretty good for temporal data things that you don't care about you can load back up pretty easily current weather so it would be a fantastic place to put that and the reason why I put it the only reason why you wouldn't want to use an agent is if you need some kind of termination callback it does not give that to you even though agent feels like a great thing you can just hurry up and jam a bunch of stuff in and I run it in too very quickly that it's better for prototyping than having to let it crash and failover mentality so GenServer is your friend because it gives you that ability to do failover you can feel more confident that your data is there and then it's going to be taken care of make sure you push logic to the bottom, logic is where your problems are going to be you're probably not going to have as many crashes because you're trying to store into a map or your data so push anything that's going to fail to the bottom of your super engine tree to give you more research for those hyzen books failover data up tops, keep data stored in the top of your tree that's the safest place for it to be, the bottom of your tree is the stuff that fails all the time stuff at the top is the stuff that's going to stick around whenever you run into problems keep data in its own process so throughout all of this I had put I moved the calls to the database out into that back end process so that the friend could respond to the user's right way and I found out that when the database goes down I was losing data even if it was restarting and that's because it was restarting in that single app but if you push the process or the business logic out to its own process and not have that stored data using tasks is a fantastic way the restarts will not affect your data it's completely boxed in on its own so if you have DTS you can back up to your database Josh you should probably do that because the top thing here is probably the only resource that you really need because all will be listed here github.com such binary noggin slash live like a hippie there's an index file and it will have all of these links and errors that will link to the architecture of the lost years that you can talk by and I've heard this a few times because I already have slides for the zen of Erling this article is actually the inspiration for me to say why do I need to be able to talk to people like every time they bring up a database they're putting into this and I figure if him and Uncle Bob will say why don't I need a database, why don't I need a database for that and then I'll link to Cachex which would be beautiful and then I'd like to thank the Lipser days for having me come out and this great conference I think all of you and these guys help me out, I look for Erling, he's been writing Erling for a really long time Adam runs the St. Louis group and Craig will check out my slides over and over but I'd like to thank him so I guess I have time for questions or you can just come and see me and talk to me afterwards I don't know why they had to put the thing at the top or is it past the address and I thought it was pretty obvious but any questions? yeah I told you that you guys see that? alright so so quarter so I had to set up Amos at X Maple because I typed for Erling so that means that I can't type so I asked you to put it in the password reset and anybody see what's going on there? oh thank you here we go so you can see it is using Ecto and it's storing the reset password into the database there, there's the reset password token and this is the using database first pass and I mean if it crash here I don't care if the date is in the database it's pretty nice when I want to complete I can't remove the order reset, complete and give it a new password, a new password that's super secure and so I will we should mod program so this is the database version and then this is all actually in that repo so you can pull it up and complete it yourself we do this again there's sys.getState so we start out our failover storage is an empty map, not doing anything bigger and our password reset is also an empty map when we start out so it's hard to spell X Maple so I got nothing returned back if I check the state now this is a password reset and the failover storage is still empty now because I only place it in failover storage when I fail so I actually store the entire user, I never have to go back to the database again whenever they come back to the reset password thank you so our reset now if we look at our failover storage it has that data in it and if we go back to our port when it started up it should ask that failover storage for it in time so now I have a process that can get killed, we start it