 Hello, everybody. Quick housekeeping. I will post the slide deck on SlideShare and share it by end of day Monday. So feel free to take pictures or not take pictures because the slides will be updated. And please ask questions as in when you have questions. So yeah, housekeeping, please feel free to ask questions when you have them. I'd rather get the most out of your time here than run through a slide deck. This talk is going to be super technical. So please do ask questions as in when you have them. So quick agenda. I'll give a little bit of introduction about myself and go over the infrastructure landscape of application deployment paradigms in the past, present, and future and why we need serverless. I will not go very deep into the motivation for serverless because Kenneth has covered it pretty well in the previous talk. And we'll dive straight down into what are unikernels and why unikernels could be relevant in the context of serverless. And we'll do a quick demo of unikernels and we'll end it with a Q&A. So myself, my name is Madhuri. I'm the founder and CEO of a serverless platform startup in San Francisco. Prior to founding the startup in July of 2016, I was the tech lead of one of the first startups in container space called ClusterHQ. ClusterHQ, we developed the first storage solution for applications that are stateful that need to run in containers. How many of you are familiar with containers? 50%, yeah. So yeah, so we had the first storage solution for apps that need to run in containers. And prior to that, I spent three years at VMware. I worked on the management infrastructure part of VMware and I also contributed to the hypervisor layer. I did some work on the memory management parts of ESX. Yeah, pretty nerdy. And prior to that, I spent 12 years at Oracle. I worked on pretty much every part of the Oracle database server. I worked on the first distributed database called RAC and worked on the transaction layer, replication layer, and eventually in memory database layer. So prior to that, I was dropping out of a PhD program. So I'm primarily a systems person. I'm very much interested in how you can take an application and run it on the infrastructure platform of your choice in the most efficient way for the app and in the most cost efficient way for your employer. So that's where my interest areas are. So let's look at the application deployment paradigms of the past and present. So the goal for running apps has always been the same. You have an application, let's say a jar file or a Node.js file and you have a platform of choice which could be a private cloud machine or a public cloud machine or what's emerging right now is IoT Edge devices. So the goal is to go from this point A to point B in the most cost efficient manner and in a manner in which your application is happy. The happiness is defined as the application is getting all the resources it needs. The resources happen to be CPU and memory resources and network resources for it to deliver your customer requests in a manner that makes the customer happy. So which means that if you type www.google.com if the threshold for your happiness as a customer is a response in one millisecond, anything that delivers that website in under one millisecond is the right way the application is being hosted on the infrastructure platform. So traditionally you would take the app and you would shrinkwrap it. How would you shrinkwrap it to run on your platform which is a bare metal machine? You would just run it as a process. But starting from 2002 until 2012, you would take the still monolithic heavy weight app and you now have a private cloud, say VMware cloud that you can run your app on and the shrinkwrapping model has changed from running it as a process to running it as a virtual machine. So the shrinkwrap now includes an entire operating system and all the add-ons that will come along with the operating system and then your app is sitting on top of it. So that was the shrinkwrap model of choice between 2002 until 2012. So what happened along the way is that people started wanting to move from these huge heavy weight monolithic apps to microservices applications. You want to break up your app into several parts that are self-sufficient and they do one thing and one thing only and they do a really good job of it and if that one instance crashes, you can bring up another instance and you should be able to pick up the work without any interruptions. So we started moving from these monolithic apps to microservices apps and the microservices apps, what those led to is a lighter weight shrinkwrapping need. So now you wanted to move from running on private cloud to running on public cloud mostly and nowadays people want to run on IoT edge devices. For example, if you are a service provider for an oil and rig company and you have these devices that are monitoring the temperature of the oil being drilled, you do not want to wait for the temperature reading to go out to AWS or Azure to figure out whether the temperature is a seriously high temperature and you need to shut down the walls. So you don't want to go through the satellite connectivity to your cloud and come back and you might result in an explosion if you don't have satellite connectivity. So these edge devices, these small little computers that are mounted on edge nodes in IoT are getting smarter and you need to be able to run your applications on your edge as well. So as of now, private cloud is kind of becoming a legacy. It's going to stick around, but it's still becoming legacy and people are moving to public cloud and IoT is up and coming market. So now how would you shrinkwrap your, let's say JavaScript web server to run on public cloud or to run on the IoT edge device that people are moving towards containers from virtual machines. So instead of provisioning an entire VM that has an OS layer and then your app sitting on top of it, you provision one Linux server and you have several containers which are fancy processes with namespace isolation and you run your app as a container on this large Linux server that's provision. And container is going to be much lighter weight than a typical virtual machine that's been provisioned in legacy workloads. Does that make sense? So what are some of the issues with running a microservice application in a container that's always on? So even though most of the applications are pretty lightweight when run as containers as microservices, you still need to provision these large Linux servers to be able to host these containers that come and go. So you are paying the cloud provider a huge amount of money if you're on AWS, you have to provision for production workloads M4.2x large instances which cost you about half cent per minute which is a lot of money which adds up over time. So why should you pay money and waste resources on your cloud when your app is not actually running? If there is no customer workload, why should you be paying for resources? So cost is a major problem. And the second thing is there are orchestration frameworks that have to start these containers and figure out where to place them, which server do I need to place my container on and is my container getting enough CPU? Is my container getting enough RAM? Is my container getting enough network bandwidth for it to be able to service all the customer requests? So those are complex problems which you're better off not dealing with them if you are not having any customer requests, right? So why pay for an orchestration framework? Why pay for a load balancer? Why pay for an auto-scaling product if you don't have any customer workload? So what if you could just pay for them only if you're actually using the resources based on your customer demands? Does that make sense? And also the last concern is that with the IoT Edge devices coming up the size of the on-disk image of your app when your application is stationary and sitting on-disk the size of the image is an issue for these Edge devices because the Edge devices have limited bandwidth for storage. So the smaller the size of the application that is stationary, the better it is for it to be able to be transferred back and forth from Edge device from the cloud. So serverless paradigm basically says that if you want to reduce the cost of your private public cloud usage or IoT device usage and only pay for resources if you have customer demand only then you should be able to provision the shrink-wrapped application otherwise no need to provision the shrink-wrapped application. So most of these serverless platforms available right now in the market are backed by containers. So when you say you want to provision an AWS Lambda function, the function is actually a container spawned on an M4.2x large machine or any other kind of machine and the container is when we talked earlier about the cold start it's basically the container is how long is the container kept active if there are no connections coming to it, right? So you still, if the container is stopped you still pay the price of the container start of time when you need to use the function again but even if the container is shut down you're still using up the M4.2x large instance. So the server is still provision and the server is still running and Linux is running on the server and it is still consuming CPU, RAM and network resources. So even if the container is, even if your function is shut down the underlying resources are still warm and running, right? So what we have been looking at is that if we look at serverless platform it has certain needs for the shrink wrapping. The shrink wrapping has to be really lightweight. It shouldn't have to consume a lot of on disk storage space and the shrink wrapping should be such that the start of time should be super fast. This again addresses the cold boot question that we talked about earlier. And the run time security of the application should not be compromised because of the way the shrink wrapping is designed or because of the way the shrink wrapping functions. Does this make sense? So all of the existing shrink wrap options right now in the market are based on containers. So Lambda, everything else is based on containers. And let's look at sample applications. So instead of taking just a function this benchmarking metrics have been collected using Nginx. So if you convert entire Nginx into the serverless manner in shrink wrap it how what is the size of the on disk image? And when you look at it as a container the numbers look pretty good. The on disk size is under 200 MB which is not too bad and the run time overhead is 274 MB which is not great but it's not terrible. And the start time is 1.13 seconds. So this is like if you want to do a cold start 1.13 seconds is not terrible but it's not great either. And security vulnerabilities is an interesting metric. So you would think that if there is no customer if your app is running and it's sitting as a function which is shrink wrapped as a container you would think that the security vulnerabilities of that app are just limited to the code that you wrote. But if your app is exposed to all of the security vulnerabilities in the shrink wrap as well. So when your app is shrink wrapped as a container a container would have a base Linux layer and then you would have all the additional package add-ons you would have on top of it and then your app is sitting on top of it. So basically you are exposed to all of the underlying Linux layer security vulnerabilities and you do not have any choice in it because your serverless platform has made that decision for you. So if your serverless platform has decided to use Ubuntu 14.04 as the base image for your container for provisioning your function so any security vulnerability in Ubuntu 14.04 is going to be exposed to your app. So let's talk about are there any other ways of shrink wrapping your application that's running as a function that actually meet all the needs of the shrink wrapping model that serverless platform desires and demands and that's where we get to Unicernals. How many of you have heard about Unicernals? One. All right, so Unicernals concept has been around for several years. So let's look at the evolution of the shrink wrapping. So initially we had the entire virtual machine where you have an entire operating system and then the app sitting on top of it. Now we come to a container where you have a Linux base but you also have a Linux base for the container. So if your base Linux is Ubuntu 14.04 but your Linux base in the container is CentOS 7 you still have the CentOS 7 layers on top of the Ubuntu 14.04 and then you have your packages that you add on say you want to use Vim as your favorite editor or you want to have network debugging tools, et cetera you would have all those packages and then you have your application Node.js file. So what Unicernals says is that as far as the app is concerned you do not need an entire OS for the app to run. Most apps do not need a floppy disk driver. Most apps if they're stateless they do not need a file system. So the way Unicernals looks at it is like the minimalistic approach of what exactly does my app need to run? So if your app is a stateless app and it only needs a network stack to communicate with the outside world Unicernals would take the app and pick network stack of the operating system and give you an on-disk image which will boot into a virtual machine but the virtual machine will only have the app and the network stack it won't have any other OS parts in it. So it runs as a single process and a single address space model and it would boot into a virtual machine. Does that make sense? So this virtual machine will be from the host OS or the example you're going to accept OS or this will be going to only. So it would run on any hypervisor that exists. So for example, if you take, if I just have a Linux server and I have KVM on it then it will run as a virtual machine on top of KVM. If you are running on AWS it would run as the demo I would show you right now is it's an AMI that boots into an EC2 instance. So basically what you are avoiding is provisioning a large instance that is sitting and waiting around for starting and stopping functions. So you're still paying, someone is paying for that instance whether it is the end user or the actual platform provider someone is paying for it. Whereas the concept of Unicernel you look at it from the other point of view, right? You only provision resources when you need them and you don't have resources, provision and waiting to start applications. Does that make sense? So let's look at demo of it. So we have, I have an AMI that I have created for Node.js web server. So this is the source code of the web server. So it's a basic web server that prints hello world as the response and it listens on port 8002. So it's a, this is just like a container for the actual customer code you want to insert into the web server. So I've created, I've taken this code and I've turned it into a Unicernel image. So the Unicernel image is now sitting as an AMI on EC2 and this is the AMI. So when I launch the AMI, what I can pick is the smallest possible instance that is available. The reason for this is because a Unicernel contains only the parts of OS that it needs and you don't need everything under the sun in the OS for your app most of the time, you can make do with the smallest possible instance that's available. Isn't none of the smallest? Yes it is. So I want to run some performance tests. So I want to take, I want to give it some resources. So, so what I want to do here is I want to select a security group such that because I have the web server listening on port 8002, I want to open up port 8002. So I will select a security group that exposes port 8002. I apologize for running the demo on AWS at Microsoft Office. So you'd see that it's a T2.micro instance and if we look at the details of the security group. So I opened 8002 because I wanted to address the other question that the other person had about how would you debug your application? So I want to show you how to debug it. So we actually exposed the debugging console through port 8002 by default and the management platform that sits on top would collect these stats from port 8002. So we have 8002 open for management and 8002 open for our web server. So this AMI contains your application along with the operating system there and it's just a lightweight version of that. So There's enough resources that you would require around your application. Is that what it is? Yes, so the AMI is basically, so we have a few open source tools. One of them is called Uconvert, U-K-O-N-V-R-T. I'll share the link for it. I'll upload the link for it in the slide deck as well. So basically what Uconvert does is it takes the source code of your app and it analyzes the source code to figure out what are the parts of OS it needs and it automatically gives you an on disk image. So the default on disk image is of a certain format which can be converted to your cloud backend of choice. So does that- You don't even have to pick a generic OS. So you give Uconvert- Just the application? Yeah. So you give Uconvert the file location of your app and Uconvert will give you an on disk image. So that's the whole point of having Uconvert is you as an end user do not have to figure out what are the parts of OS that I would need. Right? What OS is that? So the OSs that we pick are based on the source code of your app. So for example, like right now, except for if your app is written in Golang then we might pick bits of Linux but if your app is written in Node.js, for example, we pick an OS that's custom designed for Unikernel. So the OS that we picked in this case for generating this AMI, it actually has a completely rewritten network channel, a networking stack based on Van Jacobson channels that offered up to 90% performance gains based on as compared to Linux stack. So it's like, think about it as if we had to rewrite Linux, what would be, how would we rewrite it now knowing what we have learned in the past 10 years? Right? But we have Microsoft, if I had a C-sharp.net program, that would work. So far, like of the customer conversations we have had so far we have had 40% Node.js, 40% Java and around 10% Golang and Golang is starting to take a lot of Node.js space and some CC++. So as a startup, like we are a really tiny startup. So we would totally support C-sharp if we had a customer use case. So as a unikernel framework, the unikernel framework supports certain runtimes. So the runtimes that are supported right now are Node.js, Golang, CC++, ELF and Java. And some functional programming languages like OCaml which we haven't seen any production workloads in. So if there is a runtime that comes up as a customer use case where folks are interested in exploring serverless for that runtime, then we'll totally support it. Does that answer your question? Okay, so let's look at our instances. So this is the one that we just provisioned, I think. So let's look at the screenshot for this, yeah, yeah. So if you look at the screenshot, what did this AMI boot into? You'd see that it actually booted into your app itself. So you don't have the AMI boot into Linux or some other OS and then you're running an app on top of it. So what the AMI booted into is a teeny, tiny, easy to instance and it's actually running the app itself and it doesn't have any other parts in it, any other operating system parts in it except those that are required by your app. So let's verify that this is actually running what it is supposed to be running. So if we curl the public IP at port 8002, so that's what the app responds to. So your instance is running your app and nothing but the app. And no other ports are exposed except the port at which the app is listening to and the management port. So let's look at the management port to see some of the diagnostic data that are available for the app. So you can see the resource utilization of the app. So it's pretty minimal. So even though the total is one GB, you have majority of the memory available because the OS is taking very little part of your memory because it's minimal OS. So all of the 90% of the memory is available for the app's utilization. And similarly, the CPU utilization stats can be gathered and the storage stats and all of this data is available via the serverless framework APIs as well. So it can be programmatically queried. And you can, the Unicernel format automatically gathers stats. So you do not have to custom configure logging or stats or traces for the app. They're automatically collected. And again, this data is gathered from the analysis of the app that is performed itself. So you do not have to add any specific extra information to collect traces. So you can look at TCP state and page table entries like you can go pretty deep into the stats. So this is what your company provided on top of all the open source? Well, so what we do is, there are several frameworks available for Unicernels. So this one that you're seeing right now is based on a framework called OSV. And because your app happens to be written in Node.js, the tool that we used under the covers, we picked OSV as the Unicernel framework of choice. If your app is written in Golang, we would have picked something else. Try to understand what your company is actually doing. Oh, okay, okay. Sure, yeah. So what we do is we have the platform built on top of the different Unicernel frameworks. So you can go from the source code to your application running in one step instead of trying to create, yeah, yeah. Does that answer your question? So- What is the, let's say OSV, one of the components like the monitoring concept is what is offered by the framework. That's, is that one component of the framework? So there are two parts to it, right? One is how do you, what paradigm do you use to go from your app to the on-disk image? And the on-disk- That's why your company- Yes, but when we convert it, we pick a Unicernel framework that is best suitable for your apps. So what we're saying is that you have your app and you have certain requirements from your application that's running. The requirements are, my application performance need to be really good. Like by really good, I mean, it's like tangibly, you can measure the performance gains and the image size of my app on-disk should be really small and my app needs to be secure by default. So what we're saying is that we'll go from your JAR file or Node.js file to a running app that satisfies these three formats. Whether we use OSV under the covers or whether we use Rampran under the covers or whether we use Mirage OS under the covers, that's something that is abstracted out as an end user to you, if that makes sense. So you can optimize it. I picked the right kernel, Unicernel framework. Right. And make sure it satisfies the requirements. Right, yeah, exactly. Does that, yeah. So, let's compare how Unicernel's perform when compared to containers. So the motivation for this talk is not to say that one model is better than the other. The motivation for this talk is, so far all of the serverless platforms out there use containers as the shrink wrapping of choice. So our question is, is there another alternative and is that the only alternative that satisfies all the requirements that we have for the shrink wrapping for serverless platforms? So if we compare the metrics for container versus Unicernel for the four metrics we visited earlier which is on disk image size, startup time and the runtime memory overhead and security vulnerabilities. Unicernel performed better than container in three out of the four metrics. And with respect to security vulnerabilities because you're not provisioning a server that's sitting around waiting to provision the functions what you get rid of is that your attack surface becomes really small by default. So for example, there was a Venom attack. Is anyone familiar with Venom attack from last year? So basically Venom attack, someone hacked into an app that was running on an EC2 instance based on Linux using the floppy drive virtual driver. So why would you need a floppy drive device driver for an EC2 instance, right? Like you don't physically have a floppy drive to insert disks into. So just because you are sitting on top of Linux you have this humongous overhead of this large operating system vulnerabilities waiting to be exploited. Whereas if you go with the Unicernel because you're only picking parts of the OS that your app needs you're reducing the attack surface by quite a lot. And another question that comes up is like hey, this sounds really interesting and promising but how is my app performing, right? How does my application performance compare? So the comparison metrics for the Node.js web server we just looked at when we ran Apache Bench with concurrency set to one the request per second serviced by the Unicernel were about 91% better than container. So it's pretty promising. I'm not saying that it's a definite alternative. What I'm saying is that so far it looks very promising enough to be considered as a potential alternative for some time in future when the technology is mature. As of now the technologies are not as mature Unicernel technologies are not as mature as container technologies but there is a lot of progress being made in that direction. Any questions? Yeah. So from what I understand I probably don't understand Unicernel but from what I see and from what I hear from you the Unicernel looks like an alternative to the server based platforms and rather than the container platform because what we call as functional service is probably a different part than what we're doing with Unicernel where we are running probably a dedicated service in a sort of a mature way. Is that what it is or have I'm missing something? So for me it looks more like a much optimized version of the server based architecture. Right, yeah, yeah, yeah. So there are two ways to look at it, right? The first is, the first is yeah, it's a replacement for server based architecture but it is also a potential replacement for function as a service. So if we look at the function as a, how infrastructure deals with function as a service, right? In one of the customer cases we are working with is the customer has an M4.2x large instance provisioned and they are a serverless platform company and when a client request comes in they provision container, they start a container so it's always a cold boot. So their main concern with that model is that their AWS costs are super, super high because they have these M4.2x large instances sitting around waiting to start functions, right? So what we are doing for them is that instead of M4.2x large instances sitting and waiting for them, why don't we create the EC2 instance on demand? So that's option A, right? So you're not even, there's like absolutely nothing running if there is no customer demand. When a customer demand comes in your function is spun up in an EC2 instance that I showed. If that start time is also too much for you to handle then the other option we are exploring is have the EC2 instances running, these G2.nano instances running and insert the customer function into it at runtime and that's going to be significantly faster than container startup time. So it is still a comparable alternative option as compared for function as a service. Does that make sense? So the cost savings when we did the evaluation for them is they have 1500 apps running in six data centers and the cost savings for them when compared to running their function as a service in container model versus unicolonial model was 40% cost savings per data center per year. So it's still a large amount of cost savings for you to like evaluate it completely. Does that answer your question? Yeah. Someone else had a question there. So container tech. What is the container technology? What's the container technology? So the container technology that, so I come from container space, the container technology that we have evaluated has nothing to do with any specific vendor. So all of the things that we talked about with container space are not inherent to a vendor specific design decision. It's inherent to the way like, containers run on Linux, right? So the security things I talked about are coming from Linux and container is a process with namespace isolation and it is still using Linux's networking stacks. So it's not using a specialized networking stacks that's custom designed for performance enhancement. So it's all of the, all of the red boxes that you see here where unicolonial is performed better are not because of a vendor specific decision on container space. It's because of what a container is basically. Because in the latest talk account, they are releasing. Linux Kit, yeah. Yeah, Linux Kit is actually a step in the right direction. Linux Kit will optimize like, it'll bring down the on-desk footprint, for example, of your containerized application. So. The difference between Linux Kit and unicolonial. So Linux Kit is still Linux Kit. It's not a unicolonial kit, right? It's, so basically you're getting the, you're getting. Can you say what that is? Linux, is that a smaller version of Linux that I'm from in there? Right, yes. It's safe for you to view these open source by Docker. From what is the difference between the Linux Kit and the unicolonial? So Linux Kit is a small form factor container that you can like, if you form a regular container for a web server that's like, say a Java web server, a regular container on disk size is 800 something MB. With Linux Kit, you can customize it to be much smaller on this footprint. So it does address the image size issues of it, but it's still end of the day, you're still, it's still running as a container, right? So it's still running on as a Linux container. So it's not running on a hypervisor straight as a VM. So does that make sense? Not yet. So I was at DockerCon 10 days ago. So I just learned about Linux Kit as well. So we do not have, I'll, yep, yep, yep. Any other questions? This is from Q&A. So a couple of takeaways, but yeah, we can move on to Q&A. If you deploy, let's say, container, I can use ECSR and later service. If I can deploy a new image, more like, but then I don't know what you call it, or unique color. So do I need to start a new EC2 instance with your AMI, or I can install a new image in the existing EC2 also? So there are two ways of running a unique kernel. The first way where it would be most performance efficient is to actually boot an AMI with the app itself, of brand new AMI. That's like, the products that we are working on are towards that, to enable that. And the second version is if you do not, if you want to take a safe route towards unique kernel, you have your Linux server. You can have QMU on it and start up your app as a QMU virtual machine. But you're running an emulated virtual machine. You're not running on the hypervisor at that point. So your performance will suffer. So your performance will not be as good as when it's run as a standalone EC2 instance. I'll get good performance when I start a new EC2. Exactly. I might need to wait for some time for this image to come up. So that's the thing, right? So if you choose to cold boot, but what we are saying is that you have the EC2 instances up and running and you're only injecting your function in. So they are actually up and running all the time, just like your large instances up and running to start your app as a container, your function as a container. You have the smaller EC2 instances up and running and you warm boot your app. Does that make sense? Yes, but like there are many departments that we generally follow. I understand that this is not a finished product. Let's say sometime when I deploy my container, I want four set of containers to be deployed at the same time. It would be a same functionality part. Yes, yes, yes. Can I prove something like this? Yeah, yeah, yeah. So the product that we have right now is exactly for that, for the provisioning part of it. So let's talk afterwards. One more question, maybe. I have a last minute request for a very short introduction, so if you have one more question and yeah. You said that most of the service functions are using containers. I may not be made up of this, but the Azure functions that we've seen earlier, I actually doubt that they're using containers. Oh, really? Because what containers do they use to, do we know? Kennedy. Kennedy? Yeah, he's from here. Oh, really? So the Azure functions, are they using containers or how is that actually, is it just an application on IAS that does something? So Azure functions running like the type of service behind, basically running on the Azure sites, happening through the open. So they are not running on containers. They're not? So the running does. The running does last. I'm not sure where they're going to do it. Really? Even my guess is it's a container. If you look at even Lambda, it started in 2014 after the Docker code. Right, I try. That sort of business may not have been closely, but my guess is it's pretty much a container. Yeah, open this also. Actually, you can build your own platform or Lambda like service with Docker. Right. I think the Azure functions are much more live based. They just have a, maybe an application pool in IAS that does something. It's much like that. So you don't need the rest of it, but I don't know. It's when you're sitting in the mic, it's okay. I'm sure. I'm really interested in the answer to that. I'd be really interested to hear the answer to that. I know that IBM opened this, but I'll see you soon. My right guess is all this business I did. Yeah. Cool. And thanks to everybody for giving me this opportunity and questions, please. Also, I'm in town, May 14th and 15th, and I'd love to meet with folks that are interested in talking about this in general on 14th and or all 15th. So please let me know if you're interested and if you have that. Tell me, say just some new stuff and we will publish if you have some content or whatever. Cool. Awesome. Thank you so much. Thank you.