 Hello everyone and welcome to our session on serverless computing. My name is Belle McCoy and it's my pleasure to introduce two of our Red Hat rock stars. We have Steve Tran and John Keem with us today. A little background on our presenters. Steve has a master's in computer science from American University. He has been with Red Hat for over 10 years and is currently a principal consultant. He is also a Red Hat certified architect. John has a degree in computer engineering from Virginia Tech and he ran his own consulting firm for over 10 years before joining Red Hat as one of our solution architects. Just a few logistics before we get started. If you have any questions during the session, please submit them in the chat window and we'll try to cover them at the end of the session or make a point to follow up with you afterward. Also a recording of this session and all the sessions today will be available after the event on YouTube so you can have an opportunity to benefit from all the useful information being shared today. And after the session we encourage you to join us for a live chat during the break on the main stage for some live dialogue with some of our Red Hatters. And with that I'll turn things over to John to kick things off. John. All right I'm actually going to turn over to Steve and then we're going to bounce back but nice to see y'all. Steve I think you're muted. Oh sorry can you hear me now? Hey can you hear me now? Hey let me know if you can hear me, anybody. You're good. Good, okay. All right sorry about that little technical difficulties but you know welcome and thanks for tuning in today to learn about serverless computing. We are really excited to talk about this because we think this is a tool that everyone needs to keep in their toolbox because for the right type of problems serverless is going to be the best solution but this presentation is not about convincing you all to change everything to serverless or trying to adopt a serverless first mindset. No we're here to give you enough information to make your own decisions as to when and how you should apply serverless. So for today we've got some slides of what the technology is, the pros and cons of it and we're going to share some lessons learned from the field when implementing it with customers. And then in the second half we're going to get hands-on with a live demo to actually build and deploy an application completely from scratch. So and then whatever time we've got left we'll do some Q&A. So a little bit about us. So my name is Steve Tran and as a principal consultant I get to work with a lot of different customers from all different types of industries and I get really deep into the trenches with customers getting hands-on on keyboard with time with them to solve whatever problems they're facing. And just like most people here I'm a developer I'm very passionate about things that improve developer joy and I just love learning about new and open source technologies. And John you want to do your quick intro? Yes all right hello my name is John Keem. I think the intro at the top was pretty good but ideally thinking about where organizations are and how they can use or get to the cloud or have solutions that are both spanning the cloud and on-prem that multi-hybrid cloud scenario. We look to see how we can or I specifically look to see how we can map that to technology and how we can use open source particularly open source technology to solve that. Yeah so put things in context today like what is serverless what are we talking about really? All right I'm glad you asked Steve. All right so serverless is a word that we use and kind of throw around a lot and depending on who you're talking to can mean different things so it's for today I wanted to lay it all out there define it what it is what it isn't and really kind of dispel some of those stuffs. So first a system with no servers so an analogy I like to use is the wireless analogy wireless internet. You think about you for the end user using the internet there are no wires but it doesn't mean there's no wires everywhere right there's wires somewhere that are making it all work for you but for you the end user no wires so it's the same thing for serverless for you the end user the developer here in this scenario you are not concerned with the store the server you are just simply deploying your app and the infrastructure those concerns are moved elsewhere so with that in mind is it the best architecture for everything we'll know it's not you're not going to write all your future apps using this approach and you're not going to go back and rewrite all your other existing apps and refactor it into serverless but for certain use cases that we're going to discuss today it is a great fit and solves a really nice problem really well so again what it is is a system where the server concerns are moved to the infrastructure okay so we can see how we even came up with this concept of serverless where did this come from how did this evolve you know if you look back in the older days you know we had a lot of monoliths monoliths aren't necessarily a bad thing but we just had a lot more of them back in the day and as monoliths got bigger more complicated the industry at large said well you know what can we do to address the difficulties around this and some patterns started to emerge including breaking them up to more loosely coupled services and as those services then started to rot and get older and bigger we kind of did the same thing we took those services and we broke them up into even more finer grain services maybe this time around particular business domains maybe more stateless maybe we think about independently scaling them and automating the deployment of them and as that continued to progress we looked to break them down even further maybe even single functions to solve a very specific type of use case so if we take a step back and see all the various approaches that we did we can see a pattern emerging right every architectural decision you make comes with its own set of pros and cons but see if you can spot the trend here we're trying to deliver faster results by breaking up our delivery units into smaller more scalable units so that way our applications are easier to fix build and scale okay a lot of words here but we're going to walk through this this is a bit of a little small history lesson but I think it's very interesting to see how the serverless paradigm has evolved over the years so let's briefly take a look at the history so starting with one point oh the first version of aws lambdas was released in 2014 now it wasn't the first version of serverless you could get it elsewhere but it was surely one of the most popular and the reason is because they offered developers the ability to just ship code and they would scale for you handle all the other stuff now this is so compelling for developers and despite its many limitations at the time like execution caps tight coupling with aws infrastructure poor local development experience developers still loved it still were using it so we moved on to one point five where aws lambdas now weren't the only game in town there were a lot of other vendors offering these types of functions as a service and they were all a lot of them were building on top of kubernetes so what google did is they kind of saw those similarities between all the approaches took a step back and said okay well how do we bring serverless onto kubernetes so google tried to standardize it by creating a common set of components and objects to do this and thus released the k native project now realizing the opportunities that this could bring in this new momentum behind this standardization red had joined as a very early founding member but what's exciting about k native is that it's more than just the implementation of serverless on kubernetes it's really an open standard that a lot of this very vibrant community is contributing to and this standard actually breaks away from the vendor lock ins of each of these specific cloud providers so k native essentially works by spinning up new containers on demand so because of this this not only enables functions but also enables microservices and basically any container that can be spun up quickly now this brings us to today 2.0 so k native came out in about 2018 and is still continuing to push the boundaries of k native and the serverless paradigm and with our customers you know with all the people that we're seeing that are using k native and serverless we're seeing them evolve their architecture into really sophisticated architecture that are truly elastic and really using the cloud for its ability to scale up and scale down quickly and in this phase we're seeing serverless essentially become another feature right another way of developing applications since in practice most enterprises will be running a combination of serverless and non serverless workloads so microservices monoliths functions all running as containers giving you know customers giving organizations giving groups giving companies the ability to choose the best tool for the best job so steve wanted to ask you about some of the benefits of serverless yeah so i guess it depends on who you ask so for us developers it's all about faster time to market we get the focus on writing code that provides actual business value to solve problems to enhance software to fix bugs and so on serverless is going to help us abstract the way a lot of the deployment complexity reduces the need to write all that boilerplate so that we can just really focus on providing value of the code for the ops teams it's going to allow us to offload a lot of deployment concerns and to the platform things like worrying about auto scaling fault tolerance thinking about capacity management patching servers and so on these infrastructure teams they could be freed up to focus on you know bigger initiatives because they don't have to worry about all this overhead and then finally for the business it's really cost savings we don't need to pay for servers that sit idle anymore it's a pay-as-you-go model as you mentioned you spin up and spin down based on demand so it really just maximizes your resources by minimizing waste so you know conversely like where the gotcha is like what are the trade-offs for serverless um you know testing applications used to be easy but with serverless it's a little bit more involved to replicate um you know the infrastructure locally like you've got third party dependencies you've got databases event streams and other types of eventing systems you've got to mock out and stub out to really replicate for local testing for operational teams there's a little bit more complexity all this new functionality it doesn't just magically appear out of thin air you're gonna have to install the k-native components for all this to work and it's pretty big but you know when things go wrong um these op teams they're gonna need to understand how to troubleshoot these pieces and then um you know the last one I want to mention here is not every application is suitable to become a serverless container you know the first invocation of a serverless function it's going to be the slowest because it's a it's a cold start a pod needs to be spun up the actual image itself needs to be pulled from somewhere and the app needs to initialize from from cold so even though you might have some good candidates to be serverless in mind if the application is too slow or has like long running processes or intensive processing it may not be a good fit so John you know with that said what are some good serverless candidates yeah actually let's take a look at a few of the characteristics that all good serverless applications have so let's start with the upper left here stateless independently and easily scaling things up and down is done much more efficiently and effectively if there's no state to maintain things can go away easily and specific single purpose and loosely coupled again meaning we can independently scale specific functionality as required by the dynamic and real-time requirements of the system next short and sweet simple now in terms of complexity in my complexity I mean two things simple for a developer to reason about and simple in terms of asymptotic complexity and execution limits closely related to short and sweet the application should ideally run briefly and free up those resources it's using when it doesn't need it anymore for other purposes so Steve in your experience what have you specifically seen that are good yeah you know there's there's tons of good use cases but here's a real example that I worked on so I was with an airline company and they've got these things called unaccompanied miners you know like kids traveling alone without their their parents these these kids they get wristbands that get scanned every time they move from one area to another or they change custody's like gate agent to flight agent and so on you know this type of event doesn't happen too often but when it does like maybe you have a spike like maybe you've got a field trip and you've got a lot of kids coming in you know this traffic like you said it's it's it's specific you know all it is it's just tracking location of you know where this this person is it just sends it to a database it's very short and sweet there's no state to it it's versatile it's not constantly being invoked so you know that just makes it a really good candidate to be a surplus component awesome yeah I like that so conversely let's see what characteristics where a serverless approach would not be ideal and a lot of these are kind of the opposite of what you saw so stateless we have state full now so if you have something like a server side session or something right where you're maintaining state server side then you'll probably use things like sticky sessions or something but that becomes very counter to the serverless approach of applications quickly coming and going or the next kind of bad pattern is maybe an orchestrator where if your application is tightly coupled to other services either a lot of other services call yours directly or your serverless application calls a bunch of others right this means that the tight nature of this coupling is that many components actually need to now be scaled together and this is also counter to the serverless paradigm next long-running and constant notification I'm going to talk about those together long-running applications or applications that are evoked often would not be able to take advantage of that serverless nature of rapidly scaling up and scaling down because these type of applications actually then start to look like non-serverless applications which might actually be a better approach for them so this really underscores the point that the serverless architecture isn't for every type of application it's simply another tool in the toolbox some use cases fit really well and for other use cases like the ones here they don't so Steve what are some real world bad real world examples of some bad serverless use cases yeah I think the most famous example right now is from amazon prime video we've seen in the news that they recently converted back to monoliths and that was because they were using a lot of step functions which is a lot of orchestration they were you know saving things the s3 buckets transferring state something else was picking it up and that was very costly because you've got no work overhead you've got to serialize these serialized and so on so you know that was a really good example of an orchestrator you know trying to do too much and you know they realized that it probably wasn't the right fit for for serverless yeah definitely so um this is what I'm sort of calling the wall pane kind of based on my experiences collected from the field and each of these bricks here they're kind of sized relatively to how often I I see and become pain points on the left here you've got air handling and logging and bright red and really big bricks here you know in in most cases um air handling is really important to understand and get right because what you do in non-serverless applications is pretty much the same that you do for serverless application but you have to be a little bit more defensive to anticipate problems so you have to do things like build and retries to back end systems or set up bulkheads to to kind of isolate problems when they you know when they arise or you know circuit breakers to you know close open and close um logging is really really critical here because the ephemeral state of these um serverless functions they will spin up and spin down so you have to log correctly you know verbosely enough um it helps a lot if you use structured logging so that you can forward this to a log aggregator and then you can filter by different dimensions and slice and dice that data however you need to so that's really important to do testing is another thing just because we're writing smaller pieces of code doesn't mean that we you know we ignore testing we still got to follow good testing practices writing unit tests um integration tests you know you you can still follow the test pyramid like those are all still foundations of writing good code and the last one i want to call out here you know every one of these bricks could be like a you know deep dive but just understanding the configurations there's a lot of things you can change and configure for k-native but a lot of it out of the box will probably work um for the majority of cases but if you start tweaking things and trying to get ahead and fine tune things too much um you're gonna kind of shoot yourself in the foot so you know those are just some of the ones that i just wanted to highlight but like i said all these could be deep dives for like you know our sessions yeah definitely agree i also wanted to underscore there the dark uh red ones there error handling and logging those are the two kind of key ones that you really want to start with you're going to get the biggest bang for the buck so don't ignore those and really shift that idea left get it right right when you're writing the application instead of after the fact later when you're hunting through all the code and trying to figure out how to instrument it and add error handling and things just do it upfront okay now time for the demo the awesome demo we've got here so hoping we can switch uh the screens for the powers that be here that control the uh the screen sharing and steve what do you got for us today yeah as soon as the screen uh comes up i'm trying to share my desktop oh there we go all right so we are going to pretty much write something from scratch and there's a there's a couple caveats here you got to have a cluster obviously so i've got an open shift cluster and and the second caveat is the serverless component needs to be installed which i've already installed so the very first thing i'm going to do is create a namespace i'm going to call it dev nation and that's pretty much it you know for the cluster side the next thing that we got to do is you know we got to write code right so rather than writing code from scratch i'm going to scaffold a lot of this um so that it all needs to write the boilerplate so what i can do i'll go to corkis i o i o and um click on the start coding button and for those who are familiar with like the spring initializer it's the same concept you come in here you can set your your maven coordinates so i'll just call this um serverless uh dev nation as the artifact name um and what we're going to do for this demo is just build a simple rest service so you can pick whatever flavor of rest of the rest implementation you want i'm just going to pick rest easy here and because i know that i'm deploying to open ship there's a really good open ship dependency that's going to streamline a lot of deployment and and develop uh deployment uh commands that i usually normally have to manually do so i'll just search for open shift i'll just add that extension and just like that i've got a project that's already set up you know i would download this as a zip file and you know unzip it and open up in my ide i've already downloaded it so i open it up in vs code this is exactly what you would get um there's a couple of scaffold dot methods for you already you know if you go down to this folder there's a greeting resource and then this is like the sample that they give you uh for the sake of the demo um i'm going to copy and paste some sample code because you know with live demos anything go wrong and i don't want you to see me uh type a million typos so let me just copy and paste this and then i'm going to import my packages here take that that and don't worry too much about the code i'm going to dig into this a little bit uh in a little bit and for the um the open shift plugin there's a couple properties that we need to set and i'm going to copy and paste that as well stick these into the application dot properties file and that's all we need to do to deploy um a serverless application but before i dig into the details let me just kick off the build because building these are going to take um you know maybe three or four minutes and let me just do that so that we don't have to wait and while this is building in the background let's go back and look at the code so i think the first thing i want to call out here is if you look at these packages there is actually nothing corkis or nothing serverless um it being imported right now these are just the standard java apis you know this is all these jacartas that rs that's just the um the jacks rs api um and then if you look at the methods themselves like these are just standard java methods like you know it's just a regular rest service so coming from um you know spring boot or whatever framework you're you're used to you know it's very easy because you're just writing standard java um a little later when this is deployed you know we're going to hit this endpoint and we're going to get a little hello from rest easy reactive um i've got another method here to simulate a little bit of delay in processing um you know i can make this thing sleep for a set amount of time and then um i've got a headers endpoint that it's just going to pronounce some debugging information you know the point is there's nothing special here this could be anything this is where you would put your business logic to do whatever you need it to do and then this is um this is almost done it's going to take maybe another minute or so but what's happening here and if i scroll back up um i i did the maven package but i passed in a profile for native so what that means is i'm building a corkis native executable file um and when i do that it's going to build um a really compact a really fast executable um using you know ahead of time compilation um cork is gonna is going to scan through the tree figure out where all the dead branches are exclude those classes that it doesn't need um and and really you know it's just going to give you the fastest runtime possible so when this thing boots up it's going to be you know sub second boot times yeah i also wanted to quickly jump in and add that you talked about jack's rs i mean what's really great about cork is too is that the fact that it follows the micro profile specification so it's really about those following a standard and using the pieces that we need to get java onto the web using micro profile so i think adhering to that standard i think is such a cool benefit to allow developers to be able to jump in and start writing regular java code just like you said awesome so this thing is doing quite a bit of work the compilation it does a lot of ahead of time optimizations analysis to be able to really get that binary that it produces against machine code as small and as lightweight as possible and so we're seeing that now now there's a couple modes we can run which is in native mode or also the traditional open jdk mode but i see it's not done so rock on yep so what happened was you know i built an image i built the native binary i also built an image it's already on my cluster now because of the open shift plugin so let me go back go back to my cluster go back into my my project dev nation and just to show you here is my you know my image stream which is my image and because the plugin was configured the open shift plugin was configured to build that route for me if i go over to the serverless tab here and click on serving it created a route for me so if i click on this route it's going to show me the application but before i do that let me just show you the pods that are running on first deployment you know it's going to deploy it and you just saw that it's terminating right now because it didn't see any traffic um and you know it's spinning down automatically so let me go back and actually hit the service now and all i gotta do is click here and notice how it's taken i don't know maybe that was a second a second and a half because that's the first invocation of it so if i go back really quickly and look at the pods oh look you know this is the new pod that's running and the old was terminating if i hit this um hello endpoint remember that was the that was the um the scaffold method that came from the starter code if i do headers this is my test endpoint that just prints out a bunch of debugging information and you know what i want to show you now is the autoscaling feature of serverless because this is a really you know a really small really quick app it can handle a lot of traffic so there's two things i need to do to sort of artificially bottleneck this the first thing i'm going to do is you know i'm going to pass it in an artificial delay so i'm going to make the application sleep from between one and two seconds with my load um my load testing script um that'll slow things down a little bit but it's probably not enough so the second thing i want to do is i want to go modify the serverless configurations itself and i want to um i want to change it so that it can only process fewer concurrent requests per container right now it's set to zero which means that it can process as much as it can handle but i want to artificially bottleneck it so that it so that the um the machinery thinks that hey i'm under load i need to spin up so what i'll do i'll just set this a two you know two is pretty good number i can only do two concurrent requests per container and i'm going to go back here and i'm going to actually run my load test here and there's nothing special here i'm just using the work to container here it's just gonna curl essentially it's a curl i'm gonna send it 500 requests you could use whatever tool you want to here you could use like a jmeter or a load runner postman or whatever you want to use so let me just kick that off and before i kick that off let me go back and look at the pods so we have one pod that's running right now as soon as i run this command it's gonna run for 30 seconds flip over here and it's automatically scaling um but it looks like a dozen maybe 18 or so pods but it automatically calculated that hey i'm under i'm i'm getting a lot of bursts of traffic so i'm gonna automatically scale up to handle this load and if i were to you know send another load test it would continue scaling out until it was able to handle it uh so my tests are done and you know these containers will stick around for probably you know 60 seconds or so make give or take a little bit um once it starts sitting idle the you know the the k native component will figure out hey you know i don't need all these serverless pieces so i'll start spinning them down to save resources um while we're waiting for it to spin down that i want to show you one other thing when i change the configurations here let me just click on uh this you know this 0001 was the original i modified it the configurations to set the concurrency to two um so it created another revision for me so every time i modify my serverless configurations i get a new revision which makes it really really easy to roll back if i need to or if i was testing maybe new functionality and i want to do some sort of blue green or canary deployment i could do that with these revisions as well so let me show you that real quickly while we're waiting for the pods to spin down um actually some of the pods are spinning down but we'll we'll come back to that in a second i could set the traffic distribution here via the um the ui i could also do this command line as well but if i wanted to route maybe you know 50 of traffic to the new service and 50 back to the original first service you know that's where i could do that as well so i'm not gonna do that but i want to show you you know let's look at the pods again um yep you know they're they're all gone so we spun down we spun up you know 18 or so pods and we spun back down back to zero and this one is just the build so that's not the running container um and i think that's all we had for the demo so let me go back to the slides awesome so i might be a bit biased but i thought that was a wonderful demo steve uh if you'd like to play along at home and see all of that code and have access to that um this is the github link here um so you can check it out now the one that you're going to see here is going to have a bit more code uh we just steve stripped out just the important bits to show today but on this github repo you'll see a lot more varied functions that do all sorts of interesting dynamic things like that so recommend everyone to go play with that um next some resources if y'all are interested we have a whole slew of things i'm going to put it into chat here where all of you go to these urls play around and click around get your feet dirty if there's nothing else that you walk away from this is to try it you know go to corkis.io go to one of these links download the code set it up run it locally and get a feel for it see how easy it is to get off the ground all right now q and a how are we doing on time we have 10 minutes left so yeah if there are any questions um feel free to put those in the comments window and uh sounds like you guys are ready so uh we do have one question so far um let's see this is about serverless on open shift if it's running on kubernetes uh we already pay for the node does the serverless model save infrastructure if it is a low usage case sure i'll jump in we both i think we're both able then wanting to answer that but the idea is yeah as you start to scale and as you start to spin the answer is always it depends but let me say that as you scale up and you start to get more applications that are serverless as your workloads potentially are you know up and down kind of sporadic then you're able to save the cost as you're able to spin down however as we talked about earlier with the good and the bad candidates if you are however having a steady state application it's always running and have a lot of these applications then the overhead of the k native machinery and the added kind of complexity potentially that that brings we have to learn k native and learn the configurations that steve showed today then it might not be worth it right so there's there's good sides there's good candidates and there's bad candidates so answers it depends yeah i i totally agree it it depends but um you know it's really about optimizing your your capacity you know so that you're spinning down you're allowing other resources to be able to use those resources so you know it's going to depend nice thanks guys let's see uh if there's any other questions feel free to post those now we'll be able to get to them we were so thorough and complete no other questions yeah yeah so if that's it then you know thank you all for tuning in you know we we're really excited about this technology and and like we were saying before throughout the demo and throughout the presentation this is not a one size fits all solution this is not a silver bullet there's going to be very specific cases for when it's the right fit but um you know there there's always different deployment strategies and and so on so yeah it's just another tool in toolbox we uh looks like we have one more question here any uh more tips on air handling uh yeah air handling there you know there's a lot of patterns that you can follow already like you know i mentioned retries uh and john mentioned micro profile earlier you know there are a lot of um libraries and dependencies with micro profile that'll let you automatically build in circuit breakers the bulkheads the retries so you know i would leverage those apis and implementations versus trying to write your own like yes you could write a try catch for loop to kind of do some sort of um retry but there's you know there's libraries and apis that will help you and make that so much easier yeah i want to add on to that too where adding in those air handling pieces you could add them in definitely into the code base as you're developing but also if you're using a platform like open shift open shift has the ability for you to orchestrate some of that's in the platform also closely relates air handling you didn't ask about it but log aggregation as well collecting all the logs having them index index and searchable is another key aspect of that where these really good practices um kind of like are even more amplified uh for these serverless apps yes well it looks like that's all our questions so um unless anybody has anything else i think we can wrap things up there okay well thank you john and steve for that great presentation and thank you everyone for joining us today we hope you enjoyed the session and got some valuable insights