 I really hope it works. Code is all not so clear, but I'll start with the demo and then I'll move to the presentation, and then we'll come back to the demo and hope the code executes correctly. This script here runs about 800 lambdas, we're going to run 800 lambdas on the Cloud. Today we're going to talk about lambdas because I really love lambdas, and we just won't watch this anymore because of all. Hi everyone, let me just start my presentation. It works. So lambda from functions to apps back to functions, because lambdas are really, really flexible and we're going to talk all about lambdas today. So this is a slide everybody's seen it. If you look at lambdas, the right at the edge here, completely zero operation and they're completely very, very light. That's why they're so great. But this diagram is difficult. So the way I think about lambdas is like this. A lambda is basically under the hood, it's a container, it's running some stuff in a container. So there is hardware and nothing runs on fairy magic dust. You have to choose. You have one parameter to choose what hardware you run. That's how much in memory. So 128 goes all the way to three gigs, increments of 64 megabytes, but you got to choose that. So you choose one hardware setting. You don't have to worry about the operating system, but you actually do. We'll talk about that in a bit. You choose your runtime Python 36, Python 37, Java, a whole bunch of runtimes, and then the idea is that you just write the application code, because then you as a developer, you can just write the application, you can forget about all these OS nonsense, and just go ahead. But that's not entirely true. Because no one just writes an application. If you went to lambda and you wrote this very, very simple Python program, this is three functions. There's a hello function which uses the request package. If you don't know what the request package is in Python, it's one of the most popular external packages. It's as close to an official package as you can get, because even recommended by Python, but it's not an official package. So if you run it and you say, hey, lambda, run this hello function, you will get this, it will fail, because request is not packaged into lambda. You need to package request into lambda for it to work. And you just can't dump files inside, because underneath the hood is a container. You need to know it's an operating system that runs Amazon Linux, and you need to package a Python package for Amazon Linux and upload all of that code. My God, it's difficult. So you don't write code, because sometimes you all need to dependencies. And if you have to worry about dependencies, you have to worry about operating systems. So fortunately, if you serverless, there's a serverless plug-ins called serverless Python requirements, and you just do that, and you just deploy, and it will work. I promise you it will work. There's a couple of problems though. What it does is it basically spins up a Docker container, packages the request, dumps it all into the code, uploads all of them into the code bucket, and it will work. It'll look like this. So if you go into the console, it looks like this. Look at the stuff on the left, or your right, I don't know, on your left, all of those folders, right? So all of those folders get packaged automatically by the plug-in, and you can see request, and no URL at three, and all that kind of stuff. Now it works, now because you package it with the dependencies that it needs to run, right? But then if you look at the console, again, it looks like this. So I had three functions. Two functions, absolutely nothing. One function used request, but they're all the same size, right? Because underneath the hood, what serverless does is it packages up with your whole entire application into one single zip file, uploaded to an S3 bucket, and then that's the code that runs all the functions, and the only thing that's different is the handler, where it actually starts the code, which method it runs, right? So Sayonara and Hello, those two functions, no, Sayonara and Goodbye do absolutely nothing, but they're 857 kilobytes because they're packaged with request. Request is a big library. Well, it's not that big like 857, but it's not very efficient, it's not very effective, and now these three functions, which are not so identical, are three megabytes of lambda space. You have 75 gigs, so that's not a big problem, but as time goes on, it's gonna be an issue. So how can we use layers to solve this? But before that, you gotta look at it. So we split it up, right? So application code and you writing it, that's great, but there's another layer between the runtime and the application, and that's your dependencies. All of these packages, right? We talked about F of MPEG just now. F of MPEG is the binary, you can compile it into a layer and consider that as a dependency. So now it's like package that, but use layers. So if you look at underneath the hood, all of these lambda functions have the same code, they're all big and they have the same IAM role, right? Okay, never mind. So let's use layers. Everybody knows who Gordon is, right? Yeah, if you don't know, you're probably too young. A lambda layer is nothing more than a zip file that gets extracted into the op directory of your lambda container. That's it, full stop, that's all it is. But the op directory is in your path, so you can put stuff there and it will get extracted in, right? So what you can do is you can create a layer in even a different account, because layers can be shared publicly. You can create a layer in a different account with request and say, you know what, lambda? For this particular function, so this is YAML. It is the, and this is serverless, yeah? But you can look at that, say it says handler.hello uses this layer and I use fngoing just so that I can solve, but it was basically just one line you can plug in the whole AR end and it will work. And right at the bottom there, Klayer's Python 37-request version one. We'll talk about why you have to have a version there in a bit, but basically you can package a layer, you can package a dependency like request into a layer and just say this function uses this layer. And what you get is not all the functions are 503 bytes, it's small, because they're not packaged with all the dependencies. And if you go back here, you can grep your serverless YAML to figure out which function are using which dependency. You got a lot more control over which function is using what. It's kind of cool, because now your functions, you put it into a layer and then it looks cool, right? So there are some notes about Lambda layers. Your entire function has to come in within the 50 megabyte zip, 2050 megabyte zip unzipped limit. So you can't just package everything you want into it. But chances are if your Lambda layer is 250 megabytes, you probably want to split that into two Lambda layers and we'll talk about why that's cool and can only happen in Lambda. So when you create update a function, you must include a layer and you must specify a layer version. A layer version is immutable, you can't change it. Once you upload version one, the next time version two. I tried to delete every single reference to the layer, but the next time you upload the next layer is version three. And the reason why is I think underneath the hood, Lambda isn't really in runtime packaging the layer together. What it does is it packages the package and that's the one that runs. So you're not getting a performance hit by using layers because it's pre-packaged, right? And that's why you can't change the layer. In fact, if you delete the layer, this function can still run. It will still have a copy of the layer inside. And that's how we sort of know that, yeah, it's not really, it's not really on runtime, they take a layer and build it, it's actually pre-built. That's what I think. Layers and other accounts can be imported in. So if you're hitting some 75 gigabyte limit, you can create another account, create a bunch of layers and just grant permission to this account to get it or make it publicly available. And I have some publicly available layers that I can share with you. FFMPEG is a layer, for those who want to use, there's a binary layer and then you just package maybe your Python library layer and you can put those two layers together and use FFMPEG in your Python code. Pretty cool. So if you want to go all out, all out, within serverless, you can have the serverless IAM roles per function plugin. Each function can have its own IAM role. Really cool. Then you can use native serverless framework stuff to package up each difference that you can specify. This function, take only these files, this function, take only these files, this function, take only these files. And what you get is lambda functions that have their own IAM role, they have their own dependencies, layer 1's request, maybe layer 2, FFMPEG or whatever. So you know which function has which dependency. You can package up different source code into these functions all in one nice YAML file for serverless. All looks very nice. But these things don't look like functions anymore. They look like full blown applications, right? Because if you have a function, you sort of think that all functions in the same code have the same dependencies. All functions in the same code have the same permissions. But you can, with serverless and layers, split them out so they actually share nothing in common other than they're in the same serverless function. Same serverless, whatever you call this application. So you can take lambda functions, you can make them almost like applications, right? But the cool thing about lambdas is you can go all the way back. You can go back to the function. Because lambda functions have one feature that Azure functions and Cloudflare workers and all the other functions don't have. You can invoke a lambda function. So there's a difference between trigger and invoke. A trigger is file hits an S3 bucket, trigger a lambda. Something comes to the API, it could be trigger a lambda, right? Something's happened in the real world, trigger your lambda. Okay, invoke is, from code, call the lambda. Pass some data, call the lambda. And it's not like some hack, you know? It's not like someone is in the background putting a file in an S3 bucket or whatever. It's a real invoke this function with this data. It's the same thing as you would call any other function in your code. It will look exactly like local code. It will look a bit different, but you know, it looks something like this. So the line that says response equals the client.invoke. I'm invoking a function called function one. And I get the response and I can continue going on my code. So at some point in this main function, I'm offloading some compute capacity AWS, getting it back and running it all in the same program. But this, this is boring because if you look at the invocation type, it's request response. So I request before response, get it back to. What happens if Amazon allowed you to invoke a function asynchronously? Well, actually, they kind of do. But because they invoke it asynchronously, I can call up to a thousand per region by default and just let them run like crazy, right? You don't have to worry about the response. You can get it some other way. So what would you do if you had a 1000 core three terabyte machine, three gig per lambda, 1000 lambdas, three terabyte machine? There's a lot of stuff you can do, but here's what I did. I did this. It's called potassium 40. It's a bit of a tool that I did. So what I do is I invoke 800 lambdas asynchronously. They go out there and they scan one million websites. They get the robots.txt file. And the only reason, because robots.txt file is supposed to be scanned by robots. They dump all of the data into a street bucket as a lambda function, takes all of that data, makes it a zip, sends it all back. So within one invocation, I get the robots.txt files of a million websites. And now, let's see whether this worked. So let me escape this. So that, let's hope that worked. Magic. So within 214 seconds, we scanned one million websites invoking 800 lambdas, 1.25 gig per lambda, downloaded, consolidated all of that. So one million websites, but only about 400, 500,000 have robots.txt files. Consolidate all of that into a single zip file, download that zip file into the locomotive machine. Let's take a look. Is it there? Yes. So you can see this zip file, 851, yeah? So you know I'm not bluffing, unzip it. Yeah. So let me clear this up. LS. Okay, so now I got it unzipped. I'm gonna cap that, but it's too damn big. So let's see. I use jq, jq pretty flies the JSON when it comes out. Takes a while because it's half a gig of data. And when it comes out, you can see this is basically the robots.txt file of about 300,000 websites. I downloaded it while I was giving this presentation. Actually, it's about three minutes of scan time. So it's pretty cool. Yeah, thank you. So, yeah, well, let's stop this, yeah. That's basically about it. But basically, well, that is not that great. But the point is that you can go from applications all the way to functions anywhere here in the spectrum. You can do your Lambda, man. You probably don't want to do it at the edges, but anywhere in the middle, you can do it. So with Lambda, you can do all this cool stuff. There's a whole bunch of cool links here. serverless.com, of course I use the serverless framework. You can use it. SEMCLI, it's up to you. There's a bunch of plugins. LAMCI is what they use. So there's also something we shouldn't forget. So OWASP has a serverless top 10. So there's the OWASP top 10 for web applications and then there's the OWASP serverless top 10. And there's a whole bunch of layers there and the GitHub post for the project that I just showed you. And that's it. Thank you. Any questions? Yeah, sure. This might be a dumb question, but why not just have a big function or what's wrong with having a function instead of using layers? I didn't quite understand. Oh, so you can share that across. That's number one. Also, by default, within the serverless framework, all the functions will share the same dependencies. But when you separate out into layers, you can actually have each function have its own dependency. So if tomorrow somebody says there's a CVE for this particular Python package, it's massive, you have to update it. You can figure out which functions within your code actually are impacted or if they are impacted, first of all. And actually, under the hood, it's all just one big function because I think AWS, if anybody knows how it works, I think they prepackage it. I use Golang, which creates some static binary, which is the same. I was going to say, surely executing a static binary is going to be so much faster than messing around with the ZIPs and the layers. I mean, just the assumption. A static binary? Yeah. It's not crazy fast compared to... Well, I would assume. No, no, so you can put a binary in a layer. And actually it doesn't have any performance impact. You put the binary in the layer so that you can share that binary across multiple functions without having to redeploy it every time. Plus, if the binary was like 15 megabytes, that's still okay. But when you are troubleshooting and you're uploading your code, you're just uploading that bit of code that uses the binary instead of the whole binary all the time into the function. And that's like one kilobyte versus 50 megabytes. It's not a big deal, but it's going to save you some time. You can put a monero in my binary, yeah. But so basically someone has, so for example, the FFMPEG binary already. But they only make it available, I think in Northern Virginia. So you can basically, there's two ways of sharing a binary. Either I share you the ZIP and then you upload it as a layer in your account, or I give you the ARN and then you just input that into your CloudFormation stack and it's good to go. But someone has the FFMPEG binary. But if you want to use it, sometimes you need to have the Python library to use FFMPEG. So you can have that as one layer, the Python package is one layer, and then your code on top of all of that without having to redeploy it, et cetera. It also allows you to share across AWS, not just on your account. Was the layer ARN work across regions? No, so I think I have it here. Also, I'm gonna show you the monitoring. Oh, it doesn't work anymore because it's, I think, too long. So yeah, you can see, doing full disclosure, three of them failed out of the 800. So it's okay. But, so this is a list of ARNs that I have for Python. But I deploy in all regions, but you have to specify a specific region in your ARN here. All right, thank you.