 So my name's Nikom Halich. I'm a developer at Dockyard. And as Chris said before, we're a consultancy. We do consulting and training for Elixir and Phoenix. So if you need something transitioned to Elixir or you want to build something in Elixir for you, drop us a line. So my talk is on, it's titled, Leveling Up Your Phoenix Projects with OTP. And so Elixir and Phoenix have a lot of tools to augment the traditional web application stack. Specifically, today we're talking about GenServer OTP as a basis for building stateful subsystems in your application. That's useful in a lot of different examples, but we're gonna do one specifically today live. And this is my first conference talk, and I did the recommended thing, which is to live code Elixir from scratch, which everybody told me would be a good idea. So we're gonna do that today. The talk we're gonna define what we're gonna build, we're gonna code it, and then we're gonna use it in a Phoenix application with zero pickups. So backstory, one day me and a coworker were working on some OSS stuff, and we had a Rails add-on or a Rails gem called PartyFal which would automatically log your exceptions to GitHub as issues, and it would do some nice things where it would find duplicate issues and it would add comments to that, it would do some nice assigning, things like that. So we said, well, why don't we do this for Phoenix? So we were thinking, trying to think of a name, and we said, well, Phoenix is like a bird, right? So PartyFal went out to PartyFal, so that's what we're gonna build today. So the design goals. We wanna support synchronous and asynchronous reporting. We wanna have some sort of load shedding back pressure management so we don't completely destroy application with this. We want it to be easily integrated into a Phoenix application. We also want it to be well-tested so we know our stuff works. So now we're gonna get into the actual code. The more detailed plan of how we're gonna code it is we're gonna set up a project, we're gonna set up a error reporting system to GitHub, we're gonna tweak that, make it a little bit better in some ways you'll see. We're gonna do async reporting, and then we're gonna implement the load shedding back pressure. So that's the plan, and the only external dependency we're gonna use here is something called Tentacat which is a wrapper for the GitHub API and it just makes this whole thing a whole lot easier. So we're going to throw this up here, and put this up for me. Can you see that? Is that a decent size? More? Okay. More? We can keep going, more. Okay, so the first thing we're gonna do is we're going to generate our project here with mixed new partyfowl, and we're going to, the other thing is your typing gets way better when you're in front of a group of 130 people. We're gonna pass dash dash subs, we can get a supervision tree automatically. We're gonna go to partyfowl mix test to make sure everything is okay as we expect. Let's test. Okay, great, so that worked. The next thing we're gonna do is we're gonna add Tentacat in here, which we're just gonna add as a dependency, so we'll open up our project here. Okay, in our mixed file, we're gonna add Tentacat as a dependency here, which is the worst part of this, because it has to go over the internet. And this is just straight out of their documentation, and the other thing we're gonna need to do is add it as an extra application we're running. The other piece is in our config file. We're going to need one piece of configuration which we're gonna set up, which is gonna make the rest of this easier. We're actually gonna set a config for partyfowl called GitHub API, and we're gonna make that Tentacat. So if we wanted to swap this out for our own version of this later, we could do that, and it would be a problem. So this is the worst part of this whole thing, where we're gonna get the dependency, and if it takes too long, I'll just, that might as well just rip it from the other backup I have. Yeah, we're not gonna wait for that. So we're gonna copy, I think this should work. Is that still gonna work? Yeah, I didn't wanna have to hit the wire, but I think theory this should work. Does anybody know if that's gonna, perfect, okay. Okay, great, so we're still good there. The other thing we're gonna implement is in our tests, since we're gonna be, we wanna test this thing not against the real GitHub every time, we're gonna make a support directory, and implement a fake version of GitHub. So we're gonna call it fakegithub.ex, and it's gonna be, oh, it's fantastic, it's talking over the wire, that's great. Okay, so our fake GitHub is gonna be module, def module, partyfowl.fakegithub, and the way this particular, there's an issues module here, and it's gotta create method, owner, we don't care about repo, we don't care about the content we do, and the client we don't care about in this case. And all this is gonna do is return the title out of the content, and body, because that's really all GitHub issues are, title and body, in this case, for this beginning part. And then the next thing we need to do is we need to go in and tell Elixir to compile this stuff, so we can add in here another path here, which we'll define as a function off the mix environment. And I'm gonna, this is one of the sections I'm just gonna rip from a, I'm not gonna type everything because there are some functions that would just take too much time. So for tests, we wanna get this test support, and then for everything else, we just want the normal lib. So now we can start writing the test that we're gonna use for this, and we're gonna need some configuration stuff, which again I will rip from here and explain what we're doing. So we're gonna put in our environment instead of the actual Tentacout, where it's this fake GitHub thing we just created, and then this is just configuration that we'll use when we put it into Phoenix, and this stuff doesn't matter, but eventually it'll be your GitHub, you just name your repo name, and then any access token for that. And our test is gonna be pretty straightforward. We're just gonna make sure that the, for right now, we're just gonna make sure that the issue that we create with hello body world comes back as hello world. There's not too much here, we're just adding the dependency to the project, so if we're on mixed test again, I spilled something wrong here. Yep, this would be plural, because I'm not just faking the test. Am I missing something here? I've got this exactly as I have it here. Sorry. Let me step this thing. Test support, the fake GitHub is here. Yeah, I put that in the mixed file here. Oh, this is paths, I'm sorry. There we go. It's always one character that kills you. Okay, so now that we've got this fake GitHub working in a test or something, we can go in here to actually writing some code to do some things and report to GitHub. So in this application here, in our party file application, we're gonna define a function called report, and that's gonna take the OTP app so we can log errors per application, the kind of error, the reason, and the stack. Eventually these will come from Phoenix, which we'll see later, so that's why the, I'm operating with a little bit of special knowledge here on how this is gonna work. So we're gonna get our configuration stuff here out of the environment, and so we're gonna get the GitHub API, concatenate together here, the GitHub API.issues, because we know that's gonna have the same convention there, get the configuration we need, make a client, and then make an issue, and the issue title and issue body right now are gonna be fairly straightforward. One thing we know we want is we want the issue body and the issue title to have something about the kind of error, so we'll get error, colon, something, and then a little bit about the reason, just so you can see from a glance what's going on. So we're just gonna inspect it and grab 100 characters off that, build the body with the issue ref, which we'll define in a second in the stack trace. And the issue ref is gonna come into use later when we have a unique identifier for the issue. So we're gonna get the module, the function, the erity, and then the file and the line. So that way later, when we make this a little bit smarter, we can say we've seen this error before, and so you don't get a million issues opened up for one particular issue, one particular bug in your code. So this is really just constructing strings and things, but we're gonna write a test for that, and we're going to test that our report function should create an issue with title, body, and stack trace. So the kind, this is a little bit of cheating also. We're gonna make a fake error here, which is more in the stack bit. This is like a fake error stack with the module function erity, and then we'll say it's on file one, line two. So we'll call our report function here with our OTP app of whatever the test was on the kind reason and stack, and then I'm not gonna type out these assertions because the way those strings work out, there's a lot of new lines. But we're basically gonna make sure that the title has the kind of error that we passed in before. It says something went wrong, and then we have this fake stack here, which is constructed over here in this exception.format. If you pass it a well-formed kind reason and stack, it will do that and give you a nice looking issue. A nice little stack trace with something went wrong on this module dysfunction, this line. So we should better run mixed test and mess up our amount of ends. But everything's still good. So everything we've done so far is we've just passing in a kind, the kind of error, the reason, the stack trace able to construct a title on a body which looked somewhat interesting for debugging purposes. So that's everything really we need to report an issue to GitHub, but that's not really too useful because say you've got 100,000 users and they all hit your index page and they all hit the same error, you're gonna get 100,000 issues on GitHub open instantly, which will get you in, you don't want that. So we're gonna change this a little bit to be a little bit smarter and look, try to find an existing issue. We're gonna take advantage of that function with this issue ref. We're just gonna assume that if the module function already file in line or the same, it's the same problem, which is close enough for this. So not too much is gonna change here, but instead of opening another issue, we're going to leave a comment. So we're gonna make another, the comments module here, which is in the Tentacat case under issues comments and we'll make a fake one very similar to that. And instead of just building a straight new issue, we're gonna case on a function we're gonna write called find existing issue. And there's a issue.list, which we're gonna write as well in our fake GitHub, which just lists the issues for a particular owner on a repository repo client. And we're also gonna pass in the stacks so we can build that ref again. And if we don't find one, we're gonna do the same thing we did before where you build the body. But if we find an issue, we can make a comment and we can make the body. Another function we're gonna write called build comment body. Issue, kind, reason. Stack, similar to the other one. And then comments, which is that module for the API, create owner repo issue number comment. Client, and this is just the signature of Tentacat, which we're going to replicate. So we need to build a, we're gonna grab a function here to find an existing issue, which will take the issues that we listed and then our stack. So we're gonna build an issue ref from our stack. And then we're going to find in our issues an issue that string starts with issue body. Body is going to be our issue ref that we made. And that's, all that's doing is saying, is that the right line? We know we're forming our issues in a certain way. If we don't change that, we can assume that if we find something with the same start of the body, it's the same issue. The other thing we need to do is build a comment body, which I'm gonna grab as well from a snippet. And it's just gonna say occurred again, and then we'll get into what this get reaction is later. And then we'll build an issue body again off the same thing. So also just for some fun, and because the guy I was building this with is just as hip and trendy as I am, we're gonna throw some emojis in here. So if you're, it's the first time is probably okay. You're a little bit mad. The second time you're disappointed. The third time, second, third time you're probably worrying. Fourth and fifth time, and eventually you get up to the skull. So that's all we need. We'll find an existing issue if it's there, we'll build a comment with the appropriate reaction and post that to GitHub. So now we can write a test for that. So, I'm not gonna type the whole thing out here, but it's gonna be very, very similar except we're going to, when we call report, this is gonna leave a comment instead of an issue and we can assert that it starts with. And this will, for this will probably fail because I don't think it will match the unit code, but we can fix that. In the, so the only other thing we're missing here is this won't work because we don't have anything in our fake GitHub as a comment service. So we can just drop in the two things that we need which are listing issues and a comments module. So the listing issues, we're just making this totally contrived returning existing with the same module and function that we've been testing with. And then we've got a comments module here that lives under issues, which just returns content body for creating. So if we run mixed test again, what's going on? Oh, is that one of my, what am I missing here? Oh, yeah, yeah, okay, thank you. That's, okay, there we go, thank you, appreciate that. So yeah, so that leaves, finds the existing issue and then leaves a comment on that. So that's not, you know, now we can get it actually something a little more interesting where we do things asynchronously. So what we're gonna do is in our application now we can start to do some gen server things and OTP stuff. So we're going to have a new supervisor and a new worker here. We're gonna add, and it's gonna be error reporter. And this error reporter is, we're gonna write in a second which will handle the, do the gen server stuff. So we're gonna define another function here called report async, which we'll call the error reporter dot report with the same signature as report. So eventually you can just swap in and out report and report async and it will just, it will work the same way for your application. So we need to make a file called error reporter and that's gonna be module party file dot error reporter. We use gen server, our start link. This, we're just gonna grab this. And our init is gonna look sort of boring right for right now but we will use this state later which is why I'm gonna leave this here for now. And here's where this is, we need to find that report function in here. So def report is gonna take that same signature, the OTP app, the kind, reason, stack, report, OTP app, kind, reason, stack. And then we need to handle a cast for that. So what we're actually gonna do here is call back into the party file that application. So everything's really just using the same function. So we're just gonna start a child here with that task supervisor of party file application dot report which we'll call that function that we just made. So that's really all we need to do for async. Really just call our call itself which makes this really sort of nice. So we can make a test for this which will be slightly more complicated but it's not too bad. So it's gonna have a similar config as before except we need to do two things differently. The first thing is we're gonna make a different GitHub called GitHub async and you'll see why we need to do that in a second. And then register on party test here so we can test that we're actually doing the things that we wanna do. So first we'll do this, we'll write this fake GitHub client and that will look like this. So it'll have the same signature as everything else that we have except instead of like returning a function it will send these, it'll send messages. So we can detect that we receive these with assert receives and make sure that the right things are happening asynchronously when we don't have return values. So this is the exact same stuff returning just the straight up content because these are just titles and issues and titles and bodies. So we'll go into our async test and we can actually write a test for this now. So report async for should create an issue with title, body and stack trace. It'll be the same error, reason this fake stack that we have before and then we're gonna call into partyfowl.application that report async config test the kind reason stack. And now we can say assert receive create and then what this, what we wanna receive which is really the, remember in the other test here we were just checking that we're getting this new issue back and then we're making sure that the issues title and issues body look like we expect but in this case we're just going to assert that second argument of what we receive which is really just the content or in getups case the issue, make sure that that looks the way we expect. So we know that our async version is building the same exact kind of data structure that the synchronous version is and so we know that it's gonna report to GitHub in the same way. So that works fine and just to prove that I'm not faking this, we can, there is, this is actually testing something. And the other thing we need to test is make sure that the, it's leaving a comment if the issue already exists. So we can test that report async for should leave a comment if the issue exists. Then we're gonna grab the same stuff here. We could, this could be dried up for sure but application report async config test kind of reason, stack. And then we can assert receive that there's a create here and it's just returning a body because we know the comments only have a body. So we can assert that that gets received when the issue already exists. I think I need to create one, we don't need two of those. Okay, so that's the works. We know that our async is doing the exact same thing that the synchronous one is doing and it has the same signatures. We can just swap that out in our application depending on if we want synchronous or asynchronous reporting to GitHub. The next thing we're gonna do is figure out the problem where if you've got 100,000 users and they all hit your index page and you've got thousands of comments hitting your, getting posted to GitHub when they're all the same error. Like there's a point where you don't want that to happen anymore. So the way we can get around that is in our error reporter, we can start saying we only wanna see X number of issues in this certain timeframe. So let's make our timeframe something. And then say the max errors we wanna see is like 100. Which is I think an okay. So now we're gonna use this state. So we're gonna define a clear after time which we can either pass in in the opposite of timeframe or default to the timeframe at the top. Same thing for max errors. And then here's the reason we've been passing the ODP app everywhere is we wanna do error, we wanna do this sort of like low jetting stuff per app. So we can keep the apps that we're working with in addition to the number of errors per app that we have. So now in this, our handle, cast here is gonna be slightly different in we're gonna call case on this function where I called ink error. And that's gonna take the current state and the ODP app that we are working with. And if we match with okay state which means we incremented the error, we just do the same thing we did before and report the issue to GitHub. But if we get this error and we're gonna say rate limited, we're gonna do no reply and the same state again. Yep, thanks. What is it not like there? Oh yeah, ink error, we haven't written that right. So ink error is going to be fairly straightforward. We will copy it though because I'm running out of time I believe. So what we're gonna look is in the, if that, in the state we keep the OTP apps, if the count keyed by that OTP app we're passing in is greater than our max errors, we're gonna turn that error rate limited. If it's not, we're going to just update the count by one. The other thing we're gonna do is we need a way to clear the history after the time frame is passed. So we're going to define a function here called schedule history clear with the state which we'll send after that clear state time to itself, clear state, and then that will just replace in that map with the blank map there. And we need, the only thing left is to call that schedule history clear up here from a knit. So that should do everything we want there. There's really not too much, just keeping track of your error counts and then clearing them when you want to. And the last thing we want to do is write a test for that and say do nothing if the rate limit is hit. So I'm running out of time here so I'm doing a little more copying than I want. But to test, if the rate limits hit, do the same kind of reason stock before, for 120 times we call report async. And we check that only a hundred times we receive the list because we're checking if the issue exists and then that create. And then we refute, receive anything else. So if all goes well, run that, of course it doesn't work perfectly. I'm missing one thing here but this would, I said max error is 100 in the frame frame. I don't really know why that's happening but I don't have time to debug it right now. Oh, am I not? Oh yes, perfect. This is actually a test and you won the prize for the rest of this. See, there we go, congratulations. Yes, so that's how that works. And so the very, very last end of this is I have this wonderful API here and I love it, it's my pride and joy. And if I run it, once this runs, I will show you how great this is. Cool, so hit my API and this is my great API. Oh darn, okay, it's not match, not okay. Well, good thing I have a way to add party file to this. So in the router, it actually makes it really simple because Phoenix has a, it's almost cheating how easy this is. But once you add the, it has a dependency, there's a plug called error handler. So you can just use error handler in your router and define a function here called handle errors. So it takes the connection and then this is the kind reason stack. So you just call party file.application.report or report async with the OTP app, the kind reason in the stack. And if we go to, this is the repository that I made for this. Oh, we have an issue there. Okay, not okay. And if I go in here and I start spanning my API because first you have to run the API. It's like why isn't this working, why isn't this working, why isn't this working? And then I have a bunch of comments left on here with increasingly bad emojis. So basically the takeaway is you can use stuff that GenServe and OTP provide you to make these little, you know, stateful subsystems that can work in your Phoenix application for, you know, useful ways. And that didn't take too long. And, you know, hopefully that is a, you know, you can use that in some of your applications. Oh, and thank you to Doug Ewan and Chris McCord for helping on this because they both contributed to the development of this. And it will, this will be on the Docker GitHub at some point probably next week if you do want to use this in your applications or just reference the code, it'll be up there. So yeah, thank you.