 Thank you. Thank you. And yeah, I'm happy to discuss any of this stuff. Catch me outside my Twitter's at the end. There's a couple calls to action at the end, too. So the end of the talk, hopefully, I'll have motivated you to do some of that stuff. But we're going to start with a little trip down memory lane. So this is a lot about application developers, folks whose job it is. Wow, that's a little loud. Can we get it down just a tiny bit? OK. Folks who are mostly not worried about the stuff we're doing here today, they're mostly worried about solving business problems and producing business value. So looks like it's a little loud still. I've been told I'm loud. Thank you. So when everything started, you started with main. We experimented with functional languages and object-oriented languages. Along the way, we figured out different ways to organize our code. And so we've been doing this for 40 years, 50 years. So if you were in the late 1970s and you were going to build, well, you didn't build a web service then, but a network service, you got all of this setup code to write before you actually got around to doing anything interesting. And then it was kind of painful to do interesting things because you had to worry about buffer overflows. And yeah, maybe C isn't the greatest language nowadays. But in the 80s, we figured out, hey, we can make this way easier if we get someone else to open up that socket for us, and we just connect it to standard in and standard out. And so that's what InetD does. And you just need to write your stuff in there and pass it to standard in and standard out. And so your network programming gets a lot simpler. Here's where I made my introduction to programming. CGI Bin did the same thing for HTTP. So you put your Perl script on a path and you got all of your arguments, like all of your HTTP headers, passed in as environment variables. You got the body passed in as the request body. And you spat back some HTTP headers and then a body. And I think that even works. I haven't tried it. But that made things a lot easier for writing programs. And so lots of people got started writing web apps with this kind of stuff. In the 2000, Rails came along. And the code doesn't look a lot different except if we look at that middle block there where it says Rails applications routes draw do, we've removed mapping all those HTTP paths from files in the file system to something that's under your programming control. So you can actually choose without having to lay out your code in a particular way. Oh, URLs like this go to this thing over here. And that opened up the space a lot more for more creative web programming and nicer URL paths and things like that. So then Spring Boot came along. And leveraged Java annotations. And so now you can actually stick that, hey, this code is for slash right next to the function that actually provides that output. And this is pretty nice from a productivity point of view. You can see there's no main anywhere listed in here. Somewhere there's a class that's annotated with Spring Boot Controller, I think. And then it just starts that up. And wires everything together for you. And then I put FAZ in the title here. How does all have to do with FAZ? Well, this is an actual sample from the AWS documentation for Lambda. And you can see that it's basically extracted out all of this handling. And you get a package set of data. And you can just pull out the pieces you need. Hello, first name and last name from the event. It's already unpacked the JSON for you. And because this is Node.js and they're not using async and await in this particular model, they hand you a callback that you call at the end. Because that's just what you do in JavaScript. Everywhere else you would just return data. But this allows you in JavaScript to chain your callbacks and so forth. And then the real reason for doing this is that before we were mostly talking about programming models changing, but you mostly just had a binary that you ran. FAZ takes advantage of the fact that you've got these small units that you know you're not going to store state with. Stuff that we kind of learned in the 2000s when we wanted to scale out web applications. And we wrote the 12-factor manifesto. And we said, hey, why don't we design this programming environment to make it easy to write stuff that just scales out horizontally? And then FAZ said, OK, so we're going to assume autoscaling is a basic primitive. And we're just going to tear down the startup containers whenever it's needed. Sounds a little bit like Kubernetes. So if we want to do the same thing in Kubernetes, well, OK, we have to wrap it with Express. That's not too bad, but it's not quite as pretty as this other thing, was it? This is just export to function. And now we've got to worry about our ports again. We're serving on localhost. Oh, and then we've got this extra little file here. It's in a special language. It's not your main programming language. It's got a bunch of special weird commands. There's a bunch of good ways to use it wrong. If you want to, does anyone know why there's two from lines here? Raise your hand if you know why there's two from lines here. This is so, for all of you, you're not the target audience for this talk, except if you're building tools. But there's two from lines here, because the first one is setting up an environment where you can fetch all your dependencies and so forth. And then it's carrying the built product over into a smaller image so that you have a faster startup time and your image is smaller. And we're asking every developer to get this stuff right. We've gone wrong somewhere along the way here. And Docker files were a great way to bootstrap an ecosystem. You can do a lot of cool stuff with Docker files. But we shouldn't be exposing them to end users on a regular basis, because I kind of think of it like a Swiss Army knife. It's great when you're in a jam and you just need a screwdriver or you just need a knife or something like that. But if you're using this your everyday tool, you're going to get cut a lot. And your screws are going to get stripped. So the first step is, when we're building one of these, we need to know what that good interface looks like. And so for function as a service, you can sometimes just look at what someone else has done. That's not actually a bad place to go. If there's already a place where people say, oh, this is great. Hey, I'm a Java developer. I love Spring Boot. Well, then we should make our stuff look like Spring Boot. Maybe you could build your container just out of the metadata and the data that you have in your Spring app. And there's a great tool called Jib that will actually do a lot of that. So you'll get one of these minimal containers. And it just feels like you're building a regular app. And ideally, it will go in and optimize things. And maybe it'll figure out it can do native compilation even. And so instead of packing the JVM in there, you're actually just packing a native x86 image or an RM image or something like that. And we should be building more tools that fit in with developers tool chains. So once we figured out what that shape is, what that good function signature looks like, we need to figure out how to take that user's code and turn it into something that's a container that matches the outside shape. Because we still have that outside shape. It's still Kubernetes. Or in my case, I'm often thinking about Knative. But Knative is really just HTTP serving processes on Kubernetes. So it kind of looks like Kubernetes again anyway. And so in this case, we've got that nice little function that we just pulled off of the AWS web page. Someone can copy and paste, use all the knowledge they've already got. And then we're going to slap Express.js around it. And we're going to slap the cloud events SDK around it. And we'll get something that's pretty close. We will write the main for them. And we will figure out what their function name is based on the fact that it's exported and it's named Handler. And we will build a container that has Express and cloud events and that little bit of glue and their function. And that's our scaffold, and that's how we'll build it. And to package up all this stuff that's in that Docker file in a generic way, we can use a tool like Cloud Events. If you're doing Java or if you're doing Go, there's some nice tools, Jib and Co, that will do this based on other stuff. Like Co uses your actual Kubernetes YAML manifests. And Jib uses like your pom.xml. But in the generic case, Buildpacks lets you encode all of this knowledge that basically looks like a weird shell script with capitalized commands and pack it into an image that you can point someone to. And that URL becomes this is how to build me a container. And so you can just say pack build, give it an image name that you want, and then build or such and so. And that gives you an infrastructure where you can actually build things up with Buildpacks. And you can actually compose multiple layers the same way you can in Docker. So you can say, hey, I need Python and I'm building a Flask app with a requirements.txt. So I need pip and pip install too. And so Python becomes one layer and Buildpacks becomes another layer. And then your app gets layered in there and then you get a complete image is how the underlying infrastructure works. And because you've got a tool that canonicalizes this and says, hey, I'm going to run a container x and I'm going to run container y and I'm going to run container z and I'm going to take the output and I'm going to pack it up into an OCI image. You get a lot of features out of the box. They've thought about things like caching. That's one of the things when you're building a Docker file, you may think about, oh, hey, I want to copy in my package JSON first and then run the right MPM command to fetch in all my dependencies. And then I'll copy in the rest of the code. So if I change any code that's not my package JSON or my package lock, then I'm going to get, I'm going to only have to do the build steps after I fetch my dependency. I don't have to go back and fetch my dependencies again. Minimal images, again, is something where you get one expert to sit down once and figure out how to make a minimal image. And then rather than cargo culling that into all your Docker files, you have an image that just does that. There's a cool feature in Build Packs where they've got a detect phase and then a run and then a build phase. And so you can actually build an image that can build both Java and Python. And you end up with two different Build Packs, one of which raises its hand and says, I can build this. It looks like Python. One that says, I can build this. It looks like Java. And if you have both Java and Python in there, it may get a little confused, but what are you doing? This may be a time when you need to pull out the Docker file. You can also make sure that things are reproducible. And because it has those layers set up a separate strata, you can actually rebase things moving forward if you know things are compatible without needing to actually go through and rerun builds on all of your images, which doesn't sound like a big deal until you have 10,000 images in your enterprise and you just discovered that there is a shell shock vulnerability in Bash. And you're like, OK, it's time to spin up the build farm. And we're going to chug through these things. But I don't know which ones are important. So we're going to hope and pray. You can instead just swap out that layer with a Bash that's maybe has a few less features, but doesn't have the feature of giving the whole world access to your machine. And the other thing is this is an ecosystem. So there's integrations. And you can borrow stuff from other people. And I've actually done that for the demo. I'm going to show it a little later. So this is roughly what my build pack looks like. But you can see I've just added one more build pack to the end of the list of build packs to run, which is to add in the function part. So we're getting CPython. We're getting pip and pip install. And we're setting environment variables and stuff like that if it's needed. And so that's another nice feature of build packs here. There's other ways to do it. This is not build packs are the only way. This is use a tool like build packs. So that's great. Now we at least can build containers without inflicting Dockerfiles on everybody. But there's a lot more to having a good developer experience that we've learned over the last 40 years than just not having Dockerfiles, not having Bash scripts or make files or step up from Bash scripts, but not having make files everywhere in order to get stuff to run. So some of the other things you should think about as you're building a happy developer environment are templates for getting started. So the React project has a great create, react app that gets you started with an actual thing. You can just start running, and then you can start poking at it and changing it and so forth. And there's a lot of stuff in there. But you don't have to do it all at once. Spring Initializer does the same thing for Java. You can also set up a bunch of conventions where you say, oh, hey, we know that we're building a Python container. Usually it needs about this much CPU and this much RAM. We'll spit out a manifest for you or store something that gives you some default recommendations on how much resources you have. So you don't end up with, oh, look, I've got not 500 pods sitting on this node, but I've got 100 pods sitting on this node because they all set resource requests to zero and CPU zero. And so Kubernetes thinks it's free to put them on there. Hint, it's not free. You can also auto-wire up health and liveness probes so that, hey, we know we're building a web app. Maybe we should check that the web app port is open or that get slash works or something like that. You can also wire in guardrails. So you can do work in these tools to generate an SBOM or to apply other supply chain policies scanning and so forth in a way that users just run the tool and they don't need to think about, did I wire this up? It's already there. And the last thing is if you're building the container for them, you don't just have to put Express in there. Maybe you put Express in there and you put in the open telemetry integration. Maybe you put in a profiler, a non-stop, a non-interruptive profiler, so that you can later on, when you discover your app is really slow, go in there and discover that you're spending 25% of your time in reg access. True story. So I'm going to be demoing a tool that I think is headed in the right direction that's part of the Knative project called Funk. But it's got four basic commands. Create is basically that template. So hey, I want to run stuff in a certain language. Give me an example of that. And then I will go and expand it from there. Build basically is just build a container for me. I'm going to take it from here. You've given me a container. That's good. Deploy will actually wrap your container in a Knative service and run it on the cluster for you. So that's actually what I'm going to demo here. And then run lets you run the container locally because as great as it is with tools like telepresence and so forth, sometimes it's easier if you've got the thing locally to poke at. And you can do a mountain or you can attach a debugger or something like that. So let's get so we're just going to get into, whoops. OK, so we've got a payload that I'm going to send to this program later. But otherwise, we've got an empty directory. We'll say create. And we'll say that we're going to use the AWS Python language, which is a special version I made that knows how to package up the samples from the AWS Python. If you take a function from the AWS stocks and you copy it in this Python, this should be able to run it and wrap it and run it on Knative. And we're going to have it handle HTTP. And we're going to call it a demo just so that we pick a totally different name than the one I used earlier. And ta-da, it's a created one. And if we look in there. So we've got two files right now because I wasn't feeling very creative. We've got a funk.yaml, which doesn't have a lot in it. But I could use this to set things like the namespace. I could use this to set things like volumes or resource requirements and so forth. And we're still working on it. But ideally, over time, we'll default a bunch of things in. So if this is Python, maybe I say, oh, I probably need a half a CPU. And I probably need like 150 megs of RAM. And then my actual function is pretty small. By default, we're going to look for something called lambda function. And inside it, a function called lambda handler. So here we go. We've got a little function. And we are going to just. And I'm going to point out one other. So this is prompting me for information that I didn't provide earlier. And then I don't know if you can see it here because it's kind of small. But there's a little stopwatch spinning around with an ASCII animation. This is really good when you've got a stage that you don't necessarily want to spit out a lot of logs for. It's just let the user know that something's happening. You can do dots going back and forth. You can do spinning around. But that ASCII or that Unicode art really helps people know that it's not just stuck. So right now, it's building the image. Since I picked a new name, it's not going to have any caches or anything that it can reuse. So it's going to take a little bit longer. The second build will reuse those caches. And so if we go to change the code, it'll be faster. And so we've built it. And now we're going to push it to the registry. Again, this is using my existing ambient credentials for Docker. So it's another thing, if you can, try to reuse those ambient credentials for developer environments rather than taking things in a special format. And now it's been deployed. I don't know why I printed that twice. So we've got this. And if we curl it, we get a stack trace saying that I didn't pass any data. And I tried to subscript none. And that doesn't work so well. So remember how I said I had this payload earlier? So here's a first and the last name, which is what the function wants. So I can just do. And we've built a function. You never saw a Docker file. You never saw any YAML. I promise you the YAML is there. I promise you the Docker file is not there. And so if you want to see what that actually looks like with kube control. So we have a resource. It knows its URL. This is something else I'm going to plug from Knative. If we actually look at the object, we print out in the status all kinds of nice information like the URL. This address URL is how you can reach it on the local cluster if you want to avoid going off cluster. So there's actually two different addresses here. But we repeat a lot of the information from above with what we've actually observed. And we do a lot of work to make sure that when we say ready, you can actually hit that URL, which makes your tests a lot easier because you can just wait for something to hit ready and then know it works. So another thing to think about when you're building developer friendly environments is when you go to do a test, are you going to have to wrap things in retry loops? Eventual consistency is great, but it sucks in tests. So I talked a little bit earlier about how we were trying to build this whole experience and make it open and extensible because the answers I've got from my organization are not going to be the answers for your organization in terms of developer experience. My guess is about 50% to 60% of the time. So I can just keep flipping that coin and hope I keep getting heads. But it's better to make some of this stuff pluggable. So particularly templates and that getting started type of experience, we want to make it so that you can bash together your own template for what a function looks like and distribute that to your developers as an image or in this case as a GitHub repo. And so this code's not quite done, but when I ran that funk create earlier and I told it my language and my template, it'd be real great if instead of saying a language, we said this GitHub URL or this OCI artifact that you can unpack and you can see inside there what those templates should look like. And then being able to run locally. So right now we have something that wraps up and deploys the manifest, but it'd be really great to also have a dump option on KNFunkDeploy because it's awesome to be able to get to a URL with like two commands, create and then deploy. And then you go and try to cram that into your GitHub's pipeline and there are tiers. And so it'd be real great if you can take, you know, KNFunkDeploy and say, you know, dump manifest over here. I'm going to put that in GitOps and from now on I'm going to work off of that GitOps copy rather than off of what we sometimes affectionately call YOLO mode. But it's great to have that YOLO mode for experimentation, for demos, for I'm just trying to figure this out and figure out if it's right. And then when you go to production, you need to straighten up and, you know, put on the pants and go outside of the house. It's fun to hack in your pajamas. Don't go to the store in your pajamas. And so I've talked about all this in the sense of Knative, but this doesn't apply only to Knative. This applies to anywhere where you're trying to up-level the developer experience beyond Dockerfiles and Kubernetes. And if I've managed to convince you that that's a good idea, the first thing to do is to think about what is that higher-level abstraction that you want your developers to think about? So this is Functions. Knative actually works for web apps, too. So maybe we need a slightly different thing for web apps, but it's okay to have two faces on the underlying technology as long as it's making people's lives simpler. If it gives you the feeling that, oh, man, I always wish I'd picked the other one when I picked something. Maybe you got it wrong. But figure out that runtime contract. So for Knative, you've got a web server. It listens on the port environment variable. But maybe you're using Kata and you're streaming stuff that pulls off of Kinesis. And the answer is, hey, you pull off of Kinesis on a well-known environment variable or something like that, or a secret. There's a secret file that's a file at a certain spot. And when you start up, you start reading from your Kinesis stream at that spot. Whatever that is, standardize that, and then you can start building containers that do that. And the great thing about standardizing it, you can start with Dockerfiles. Dockerfiles are not a bad tool, but they're not the tool you should be using all the time. I was trying to make analogies on Twitter the other day, and I said, yeah, it's the Swiss Army knife, and sometimes you want to scalpel and you want to do really fine stuff. You don't really want your Swiss Army knife for surgery. And sometimes you are putting together an IKEA set and you don't really want your Swiss Army knife because the screwdriver doesn't lock and it'll keep twisting and you'll be saddened. So you really want to go get yourself a real screwdriver. I'm encouraging build packs here because I think having more build packs and being able to build off of... There's a lot of patching involved in keeping build packs up to date. So there's a vulnerability in Python. We need to package a new Python. Yeah, you can put yourself in the business of packaging new Python every day, or probably every three weeks, or if we as a community can all come together and say, hey, here's a place where we're going to maintain some baseline pieces. I think that we can save a lot of effort. NSBOM and tools like that will also help us detect those vulnerabilities. But having 100 organizations each rushing out to patch Python feels like a waste to me. And then try it out. Try it out yourself. Putting together this demo was awesome. I found two bugs that I'm sending PRs for. And do user studies. Get your users to try this stuff out. Sit down with them. Say, hey, does this actually make your life better? If I told you you had to use this, would you be angry? Meh? Ecstatic. Like, you know, I used to do it this other way. This way seems about the same. It takes about the same amount of time. It's all good. Hey, this seems like it's twice as fast and I don't have to copy strings from here to there. I love it. Or, you know, hey, I had these five tools that I built around the old workflow. Those tools are part of your developer experience, too. If you break those tools, you're breaking that developer experience. So try out new things with your actual target customers. Ask them questions. You'll probably be really excited to know that you care. Also, try things with different languages. Maybe you'd use Java all the time. Try Python or Node or Ruby, where you don't have a type system in the same way you do with Java and you don't have a lot of the other tools. And you've got other conventions instead. They're not bad languages. But they don't all work the same. And if you're a Python person, you know, try Java or try Rust or something like that, try Go and see how that changes the way you think about things. So thank you. I apologize that I could not find a good font color to stand out that didn't look terrible. But my Twitter is E underscore K underscore Anderson. Yes, I was late to that game. My getup is Evan K. Anderson. And the big thing I wanted to call out was if you're interested in this function stuff, particularly for Knative, we are just getting ready to do another round of user studies on it to see how people use it, what they really wish it could be. And if you're interested, join us on slack.knative.dev. That will give you an auto invite. If you turn those two things around and you put the slack part second, then you get to an SSO login and everything's terrible. And yeah, join us in the function sandbox and let us know what you think.