 Mary had a little lambda. S3, its source of truth. And every time that lambda ran, her bill went through the roof. You'll notice my Twitter handle on every slide that I've got. I'm currently live tweeting my own talk. We'll see how that one winds up going. The director's cut for this. As mentioned, I do run last week in AWS. If you look at the back of the program, you'll see a print version of it. So my apologies in advance if it offends anyone. Other than that, I'm winding down my consultancy and looking forward to being gainfully unemployed. It's can't imagine why. Okay, who here is familiar with AWS Lambda? Cool, who here isn't sure what it is? Who here pretends to know what it is until about two months ago when you start building a talk about this and have to learn it a lot better? All right, at a high level, it's effective. Instead of getting away from the idea of instances or spinning up virtual machines, now you just hand the code off and the provider handles everything for you. It's serverless to some extent. As an aside, if you ever try and pitch a serverless conference, prepare for the waitstaff to spin in your food if they don't have a lot of context to go in there. So it's an interesting model. The economics of it become fascinating. It's generally a decent fit for small projects, very, very small glue pieces of code, components of pipelines. It excels at tasks that are highly concurrent. And that's the sort of thing that it starts to be used for. Think of replacing cron jobs, we'll get there. The entire premise is that in one of the provider's blessed languages, you upload a pile of code. You can write it in several different versions of several different languages. If your language of choice isn't on there as a general rule, it kind of sucks to be you. So it's a little prescriptive. And the way that it gets invoked is generally an event source or a trigger that fires off whatever your function is. The idea here is that it changes the way you start thinking about these things. You wind up with small components tied together again and again rather than something monolithic or trying to shove everything into one terrible function like some of us who claim to be developers sometimes. It's if you're not very familiar with AWS, there are similar offerings from GCP with their cloud functions or Azure functions from Microsoft, which are very similar except people use this one. So that's why this talk is here. So before I get into what Lambda is or isn't, let's talk about first what it's not good for. Using some machine learning technology that I've licensed from hot dog or not. If you have your code and its dependencies for your function and that winds up being larger than 75 megabytes, that's not a good candidate for a Lambda function. There are ways around it of grabbing more code when it runs, but that's pretty much a bad pattern. If you can't break it down smaller than that, Lambda's not for you. If your code has a lot of state or you're dealing with large quantities of data that need to be involved locally to run a function, you absolutely don't want to use a Lambda function for that. The idea is that a Lambda function should be idempotent. You can run it multiple times and not break things. Exactly once execution is according to modern computer science for distributed systems theory impossible, you can have at most once, at least once, or who the hell even knows how many times it's going to run, but exactly once is very hard to prove. And lastly, if you want an architecture that a freaking human being can understand, you definitely don't want to use a Lambda function. So now that we've covered a little bit about what a Lambda function isn't, let's talk about when you do want to use a Lambda function, namely what it is and what it looks like. Under the hood, what Lambda's infrastructure is, is a bunch of containers that inside of a container gets spun up, containing your function and it gets run. And this container does have some interesting characteristics. You generally don't get this stuff exposed to you and some of this is published, others if it isn't, we have some sneaky tricks to find out more about what's under the hood. For example, the way it sets up and gets built for is you configure a sliding scale of the amount of RAM. It starts at 128 megs and goes on up to a gig and a half. And the way it's built is amount of RAM and how long the function runs for rounded up to the next 100 milliseconds. So it's usually dirt cheap. As an example, if you have a pile of functions throughout the course of a month that takes one gigabyte of RAM and you've run it for 10,000 seconds, that'll cost you one penny. So millions of requests generally break down economically into the tens of dollars range. And the CPU power is never explicitly defined, it's sort of hand-waved over, but it does scale with the amount of RAM you allocate for this. There's never been an explicit statement made of exactly what CPU you're getting or how performant it's going to be. That does mean that your function behavior can be somewhat non-deterministic. There's also a hard lifetime of 500 seconds. So after five minutes, your function terminates whether it's done or not. You should probably have a plan to handle that. And if you run a function a second time, it can reuse the old container. It's not guaranteed that it will, but it might. So if it's connecting to a database, there may still be a database connection there the next time it runs. So testing for these things sometimes makes a lot of sense as well. And you also wind up with half a gig of scratch space in slash TMP, which is the only writable thing on the entire file system. So if you're just trying, if you're not paying attention to where you're writing a file and you're trying to write something to store it somewhere, it'll fail and it'll be very confusing as to why until you start learning under the hood what this looks like. And I want to apologize, because this slide is just awfully designed and badly put together, but add some color and maybe a better font and maybe a background image. And there we go, here in 2013, this is a very forward-looking talk, let me tell you. Now, a lot of this is stuff that we know because Amazon has told us this, but a lot of it they didn't. And we've learned some of it because Eric Hammond in his wonderful blog have built a number of functions that lets you run arbitrary shell commands to explore what the environment looks like. And other people have built functions to copy the entire runtime environment into S3 and then people have built Docker containers out of this. And there are local projects you can use for this. And Clay Smith of New Relic in a truly marvelous, terrible idea, wrote a Lambda function that opens a reverse SSH tunnel to a host you have running. So for five minutes, you can now play around in the file system and take a look at it firsthand. Don't do that. So if you're still unclear on some of the fundamentals behind Lambda and you wanna learn more, there's a developer's guide that AWS makes available. And you can flip ahead to the Getting Started section, which you'll find on page 174. What the hell's on the first 173 pages? Well, actually, it's very simple. And if you don't understand it, maybe you're not equipped to handle it. Yeah, this stuff is complex. There are entire companies and projects built around simplifying a lot of the nuts and bolts around this. This is not an exhaustive list. I've missed a few that have launched in the past 20 minutes. It's almost like JavaScript frameworks. It takes a lot of the manual work out of building these things, but it's good to do it at least once, to understand what's going on and then never do it ever again. And I sat down to build a bunch of demos of highlighting terrible ways you could use Lambda. And I had some horrible, awful ideas that I was in love with. For example, one of the ways you often see this run is something triggers the Lambda function to run, such as an object appears in a given S3 bucket. That will invoke a Lambda function that operates on it. And maybe it then goes ahead and writes an S3 object somewhere else or back to the source. And now you've built yourself an infinite loop. And this will work. Well, it won't work, but it'll do what you think it would and run forever. And that's a bad idea. When you're trying to go to massive concurrency, it turns out that database servers don't like 10,000 things, all try and talk to it at the same time. It smells a lot like burning if you've never tried it. And there's always the idea of transpiling between different Lambda functions again and again and again, just to translate hello world and have a 15 second loop that winds up completing. And yes, you can use bash. In fact, some people's handlers are simply a Python call that shells out to bash and then runs whatever command you want. If you're writing Lambda functions that are primarily grep, awk and said, I salute you. And you can do a lot with go as well because you can statically compile a binary with go and then shove that into a Lambda function as long as below 75 megs. And yes, Lambda functions can call other Lambda functions. It's like a game of telephone only with a lot more added misery. And my personal favorite, demonstrating large scale projects with by breaking into a lot of small different pieces. Here's a fun fact. For a long time, elastic search default password has changed me and I can test for that programmatically. Another completely unrelated fact, there are 4.2 billion IPv4 addresses. I can test that with a calculator. So you could probably see where that's going. So if I try and scan this with four billion scan targets running a second apiece, I probably should have started writing this talk a little bit sooner. Incidentally, running this would cost you just over $9,000 for the Lambda piece, which is why it's very important that you run this in someone else's AWS account. And that 1,000 at a time is not just an arbitrary number. By default in an account, a safety limit is that you're limited to 1,000 concurrent executions of Lambda functions. That's been raised from 100 a few months back. And you can request limit increases because I didn't have 46 days to build this talk. So yeah, I opened a support request. Yeah, can you set my account to run four billion concurrent Lambda functions, please? I want to scan the entire IPv4 internet for vulnerable elastic search installations because I think it would be funny. Now, Amazon has 13 leadership principles that they hire based on, but one that's noticeably not in that list is a sense of humor. So you can probably guess how that conversation went. And there's now a shoot to kill order out in place for me if I ever try and make it into Seattle. And I originally thought that all of this would be funny. And I was wrong. It wasn't funny. It was hilarious. But more than AWS just being spoil sports about turning their flagship serverless platform into a wonderful attack platform to actively make the internet worse, I had a bigger problem here because a lot of people are using Lambda for use cases that are very interesting. It's like they looked at my ideas that I just went through for this talk and said, oh, you're a adorable child. Here, hold this a minute. I'm going to need that back. So the challenge I ran into was that I fundamentally got upstaged here. And the stuff that I found when I was looking for this makes my examples look an awful lot like contrived nonsense. And ha, this is funny as a joke, pales in comparison to we did it and we open sourced it apparently with a straight face. But it's mean to make fun of people for their work. So this is now instead a talk about various proofs of concept that people have done with Lambda. Does anyone know what this is? I'll give you a hint. It's usually wrapped in this. That's right. It's a 6502 CPU from the 80s. And it's normally wrapped in a Commodore 64 or equivalent when you see them in museums. You might see where this is going. Someone built this. You can now write code in 80s assembler and run it through a Lambda function. My infinite loop doesn't really compare to that level of lunacy. Now, perhaps some of you have heard of Clam-A-V, the open source virus, scanner. That analyzes files to make sure that they're safe. And perhaps some of you run it. Perhaps some of you should get out of Boston and go back home to 2013. Because we're talking about serverless here. Airbnb wrote something called binary alert that's up on GitHub. And it serves the same function. You give it a binary. It runs a bunch of heuristic tests on it. And then it outputs whether it's clean or dirty like hot dog or not hot dog, same approach. But instead of installing it or getting it installed just via a yum install Clam-A-V like I'm some kind of ancient caribou, instead we wanna go serverless and web scale. Because why have a single binary when you can have three Lambda functions, S3, simple queuing service, CloudWatch, simple notification service, DynamoDB and several more components all bolted together. But hey, at least this way you don't have to run instances anymore. Getting the files into the S3 bucket in the first place, you guessed it, is left as an exercise for the reader. This is a helpful reminder that everyone's primary project is, has been and always will be their resume. This one's fun. This is, don't be fooled. This is George Porter. He's an assistant professor in the computer science and engineering department at UCSD. And I did some checking. He's not a Lambda function, but he is going to be relevant in a second. John Emmons is a PhD student at Stanford and he wrote AWS Lambda face, both as a proof of concept, as well as an amazing insult to call people on the playground. And earlier this year, he went to a conference, an SDI here in Boston, as one does. And he spent the day at the conference with a GoPro strapped to his head as one most assuredly doesn't. And it's fun because you see Facebook in the background there of that slide and they're looking at the guy with a GoPro strapped to his head, just going, amateur, we'll show you how to surveil properly. So at the end of this conference, he has over six hours of video. So he had a Lambda system that he trained with a machine learning algorithm to analyze each frame individually of that six hours of video to see if it contained George. And he stitched that using something called X camera that those frames were stitched back into a montage. And we learned some things. We learned that in six hours, George Porter is willing to spend only 17 seconds talking to someone at a conference with a GoPro strapped to their head. And the point being here is that when you have a project that is massively parallelizable, suddenly you're in a world where Lambda becomes an awesome and efficient tool for handling these things. It was also not expensive, breaking through each one of the frames of a 30 frames, I'm assuming 30 frames per second, of an entire GoPro, a six hour GoPro movie, only to cost $8 to do, as well as the machine learning components. And start to finish on that six hour movie, it took five minutes. It's kind of an amazing thing when you think about it. And this starts to become an interesting idea, especially when you're as there are now ways you can run Lambda functions locally, do pre-processing for shoving video over a skinny pipe, for example. This may interest some of you. Lambda is a platform that is now certified to handle credit card information through PCI and personal health information through HIPAA. And that's great. But back in 2015, when Lambda was released to the public, it was announced is AWS Lambda is generally available. And I love that phrasing because it's also their SLA statement. Do you guarantee three nines of uptime? No, but it's generally available. Lastly, I want to talk a little bit about using Lambda to make people regret ever letting me near a computer in the first place, let alone theirs. I feel like I should have a disclaimer first. Specifically, this is for humor, more than a thing you should do. Please don't hate, sue or both me. So almost anything could be a Lambda trigger. And AWS in their demos of things you can do in serverless does, gives a wonderful pile of examples about all this. Warnings when you hit spending thresholds. You can use it to clean up remembrance of auto scaling groups. And obviously, because they are trying to drive adoption across their entire enormous suite of offerings, most of their examples include using the AWS APIs themselves for different services as a target for what the function does. Unrelatedly, did you know that you can open support tickets with an API? And you pass parameters to Lambda, this is functional programming 101. And there you go. So theoretically, every time you encounter an error thrown by something in the AWS environment, you can automatically open a high priority support ticket with Amazon over it. Your try catch box can do thousands of these an hour. And when the support API itself starts rate limiting you and throwing errors, you could open support tickets for that. Who hears an Amazon customer? If you ever wonder why support sometimes is a little slow to respond, yeah, it's me. Some boys cry wolf, I breed them. This is a relevant thing you might want to take a picture of in the next slide. There's going to be a pile of links that reference all of the things that I was talking about in this. Yeah, there we go. I'm also tweeting out the link for it as well and I will make my slides available. But everything I've discussed is a real thing that does exist and I have links to it. I'll leave that up for a minute. Does anyone have any questions, comments, criticisms? We also accept what does the matter with you as a perfectly valid question. Does it matter with you? Yeah, exactly, many things. Long name and expensive to fix. Yeah, Lambda is an interesting way of thinking about different problems. And this is the challenge. We saw this with virtualization back in the early 2000s. Then we saw it with cloud, cloud, cloud for a while which we're still living through. Then we saw docker, docker, docker, docker, docker, docker. And now we're seeing it with Kubernetes but serverless is the next thing that the hype evangelists are jumping on. And as a direct result, it's fantastic. It is a wonderful technology. It's not fit for all use cases. Not every problem is the sort that your Lambda hammer is going to be effective at solving for. Are you finding any anti-patterns to the terrible use cases? So basically, have you found good things to do with Lambda? I mean, I've implemented it for some, but... No, great question. I'm in the process of working on a blog post of it now but there's I think four or five distinct Lambda functions that I use every week to publish my newsletter for one. It's one of those fun problems where I interact with a bunch of different APIs. I want to make sure that things happen a certain way and if I do it all by hand, it would take me far longer and I don't have that long of an attention span in case that wasn't obvious. But yeah, other than that, you also see it being used as a lot of glue code between different services. In my last job, we had one where we would copy off a... We'd use it to wind up, inspire to do copies of AMIs between regions. We'd have it to spin up different creation using Packer of new AMIs to be created in the first place. There's a lot of maintenance functions around there that wind up being used. It's one of those areas where if you don't want to have persistent infrastructure or you want things massively scaled, it's a great plan and a lot of the use cases you'll see are, ah, instead of running a web server, we're just going to run it serverless. In many cases that particular pattern instead winds up being more of a proof of concept than something that makes sense to do. We're still seeing a tremendous growth in emerging best practices around how you wind up deploying, maintaining and versioning Lambda functions, especially with more than one person using it. So until that solidifies a bit, I'm somewhat hesitant today to say, ah, this is what you should do and how. So that winds up being a constraint on it. That said, it's a fun toy. You don't have to worry about infrastructure in the way that you used to. There have been outages of Lambda, but they're generally not common. Again, despite the fact that it is certified for sensitive data, I wouldn't use this as something that I'm losing huge amounts of money or people's lives if it doesn't fire off within a given window. Use your head. Monitor for success, monitor for failure, make sure that things are doing what you expect them to do. I've learned some fun things too. If you have multiple versions of a published Lambda function, they'll all run so you can build your own race condition. That's kind of awesome. These are things you learn by doing. So my question is, you showed a lot of terrible things that you could do with Lambda function. I'm assuming since they're running in containers that everybody's containers are shoved on to the same instances in the back, should you be concerned about people doing terrible things while your functions are doing things that are not terrible? Well, I'm assured by a giant number of angels that Docker is perfectly secure. So I can't imagine that we would have anything whatsoever to worry about. Yeah, today at least, there's no such thing as dedicated tendency for Lambda. There's no guarantee what workload you're working next to. And despite it being approved for HIPAA and PCI and several others, if it were up to me and it's a bet the farm on its story, I would find a way to have whatever is running as a part of my function not contain sensitive data. There are ways to securely pass encrypted keys into it using parameter store or KMS, but it's still not a terrific plan for every use case. I wanted to comment on another actual use case we found, which is if you're trying to bootstrap a small company, the nice thing is you can host a web application that doesn't cost a thing until somebody uses it. It's pretty handy. The question I was gonna ask you though is, do you use anything for deployment frameworks? You put serverless up on the slides, but... Serverless Apex, Zappa, Claudia.js. I mean, there's a lot of ones to choose from. SAM, the serverless application model now has, is from AWS itself that starts speaking to a lot of this. People have a bunch of projects on GitHub that speak to the same thing. With respect to your first point of economics, you're absolutely right, especially for bootstrapping, especially when you're small scale and building a thing where it's a labor of love to some extent. Once you're past a certain point and you have a company that actually, I don't know, has money, suddenly your engineering time is not free and re-architecting your blog to be a convoluted series of Lambda functions every time someone browses to it probably isn't the best use of time. Why are you doing this? Because we can save $75 a month on our hosting infrastructure. So we're gonna have probably $50,000 of engineering time going into doing that. Yeah, break even is only 200 short years. Yeah, it tends not to be an economic case at outside of certain edge cases. It's one of those understand that people's time isn't free and that this stuff is still evolving and complex. They also end of life one of the early JavaScript runtimes. So at this point, you're now, those functions will no longer run. You needed to upgrade them to newer versions because of course you do. I've actually got a quick question myself. Thank you for the security focus of the talk. You mentioned a couple cool projects and I think binary alert and stream alert and every of these other security open source projects are super awesome. I think there's a gap in Lambda security projects, for example. So I was curious what you thought was a place that somebody could fill a gap with an open source project to detect bad things in Lambda like other people running Lambda's on your account that you didn't even notice because you're not actually tracking it, for example. Yeah, there's a large contingent of Lambda functions for various one-off functions. And again, that's the entire point of them. You may have picked up a paper in the last six months and seen things about different data breaches from S3 buckets, not the Equip Accident. That was something far stupider. But the idea was that, oh, we're just gonna have improper permissions on S3 buckets, which is easy to do. So having a Lambda function that fires off on invocation every time that a bucket is created then scans your buckets to validate that permissions are set correctly or setting that on a schedule makes an awful lot of sense. But I would be hard pressed to identify the business value of a single function that winds up then going forward into a world of, oh, this thing fits in 75 meg. This is a single idempotent function and I'm going to build a business case around that. I'm not saying it doesn't exist, but almost by definition, it feels like every Lambda function itself does something fairly trivial. It does one thing and does one thing well. Selling that, I don't know if there's necessarily a story there. Maybe there is. I never claimed to be a business case person. I have one question. Sure. Okay. So regarding the Lambda functions and its life cycle management, have you explored several less framework that not only can manage Lambda, but it can also do Azure functions and also you can put security as well as the whole deployment and rollback portion of it? Yes. Serverless Zappa Apex, a lot of them do the same thing too. And fortunately, we're starting to see a consensus around the proper way to manage those life cycles and keep them agnostic between cloud providers. The unfortunate part is that the consensus all of the others are wrong. So it's still an emerging field and I'm personally am somewhat hesitant to pick a horse just yet to bet on. They all work. Don't get me wrong. But it's one of those areas where it's not quite like standing up a web app on a Linux system, where yes, it's horribly insecure and a terrible idea, but I can leave that thing running for 20 years and it's probably still going to at least work. Whereas with a platform like this that's evolving as rapidly as it is, I may have to go back and reduce significant portions of that work in a year and a half to two years, once fundamental precepts that are baked into that start to change. Perfect. I want to also thank Randall Hunt, who is the senior technical evangelist at AWS. But I had a suspicion about him and sure enough, he is in fact 30 different Lambda functions inside of a hoodie acting like a person. He's gonna kill me for that. Yeah. Thank you all very much for your time. Enjoy the rest of the show. Thank you.