 So yeah, my name is Vincent, I recently moved from Saigon to Singapore, in Saigon I was doing Docker meetups as well, so when I came here I'm doing the same. So yeah, a quick recap of DockerCon. So first off, how many of you are familiar with Docker? I know, all right, so I say more than 50%, it's about 50%, more than 50%. So I had some emergency slide in case that people were not familiar. This is a new Docker web page UI, I mean, it's quite nice. But considering more than half of the people here are familiar with Docker, I'm not going to go too deep into containers or the architecture. So one of the first announcements of DockerCon was that they have really been working hard on the developer experience. So I'm not sure how many of you are aware of the Docker for Mac and Docker for Windows betas. How many of you have used them? It's like a good bunch of them. How many of you feel like they are happy with Docker machine and the way it is and virtual box or whatever and are not switching? How many, there's a couple, right? Or maybe just not had the time yet to switch. So here I feel like I should convince you to switch. Because so I was basically before I was working on Windows all the time. Now I switched on being converted to Apple and on Windows already, I was using Hyper-V, I didn't like the virtual box and things like that. So when Docker for Windows came out, using Hyper-V I was very learned. I really wanted to use that. So when you are using Docker machine at which, what are you? I'm just using Linux. You're learning Linux? Sorry, sorry, so I forgot to account for those people that are truly running containers on their machine. Okay, so anybody else? Is there anybody still using Docker machine? Right, are you using virtual box or? Running on my lab on ESX, on top of ESX? Sorry, on the top of Ubuntu. All right, so you're running a virtual machine? Yeah, virtual machine. Okay, all right. All right, in your lab basically, that's what you said, okay. All right, so how many of you actually saw the keynote from the Docker presentation? Yay, because I basically ripped the slides and but I cut them down because I just want to focus on some parts. So the Docker for Mac beta and Docker for Windows beta is no longer closed, so anybody can now get it. So maybe a short demo of how that looks, switching to that. So I have here, I think it's hopefully big enough. So I have here an application, which is like a small goal and application, which uses Redis and Postgres. And let me maybe show you the compose file. So basically, this is my compose file identifying services. So I have a load balancer, which is running Nginx. And I have the application, which is a small goaling application. Because I'm using statically compiled binaries and putting those into a almost scratch, it's an Alpine container. I cannot use Docker build. I have to use a make file to build one container, then build the second container. So that's why this compose file is using the image directly. And I'm using Mongo and Redis. So let's stand this up real quick. So maybe you can see that right now I have the Docker for Mac. And it's running version 1.12. So that's a cool thing. It automatically updates and pulls the latest Docker engines for you. So I'm running Docker beta and, okay. So I can do Docker. There is currently no containers except for one container, which we will ignore. And I have built already this container. So it's like a 14 megabyte container for. So if you see the container that actually compiles is 800 megabyte. And the container I ship is just 14 megabyte using Alpine. So, and then I have, yeah, that's kind of the target. I kind of don't like 800 megabytes containers with make. I think this is Alpine goaling, this depth. Let's see. This one is not, okay. This is goal on 1.6. So it's not the Alpine version. And this is my ECS sample binary copied into an Alpine container. I could also use a scratch container, but I had some issues with that when using special things. So it's taking to Alpine for now, okay. So, in my compose file, I have defined several networks, right? I have defined a back-tier network and a front-tier network. I have Redis and Mongo connected to the back-tier network. And the app connected to both the front-tier, alias as apps and the back-tier. So it's kind of like segregated network. And so the demo is actually not all about this. It's just to show you how easy it is to just Docker compose up. And there we go, it created the app for me. So Docker, let me just PS, and I have Redis running. Sorry, I have the whole stack running. And it should have exposed the ports for me. So let me check. I don't see the port exposed though. This is not the right compose file, sorry. I had to jumble a little bit because I'm trying to use a new Docker bundling, and for this, I need to have a different compose file. So let me just take the bundle to... Wait, before I do that, Docker compose, stop, sorry. Not prepared well. So that just removed the whole stack and... Okay, let's try that again. No, did I just mess it totally up? Of course, I just overwrite. Why not? Copy bundle to compose. Let's delete it. I thought it would. I thought it didn't test it properly though. So down, huh? Okay, this is better. It's creating, it's actually creating the network. It's creating the, looking better. But I still don't see... Okay, I messed up my compose files. So let me just pick it from the web. So I have a repository, luckily. Sorry for that. Okay, so let me take this Docker compose file. Okay, and where's my bundle though? Because I had to make a difference between the one that is the bundled one and the one that is, okay. Well, let's do that again, please, this time. So I expect the load balancer, yeah, finally. So my compose file was missing my load balancer. Now the load balancer is exposing port 80 on my local laptop. So that's another thing. If you're coming from Docker machine, managing virtual machines, then with the new, with the new I have here. So it was earlier, it wasn't running. So trust me, now it's running. Okay, yeah, that was testing. So this app is now running on localhost on my Macbook. So I didn't have to figure out where is my virtual machine running. It actually proxied the port from my virtual machine, which is actually running in X-Hive, which is a process like another. And there's a proxy running forwarding any ports exposed on the virtual machine directly to my local host. Now is this, is this right huge? It's a little thing, right? Linux is used to it. But for other developers, it kind of makes the whole thing a little bit less mystical, I would say. So let me take this app where I can upload an image. This goes to S3. And then, so I didn't write this app. It was a sample app of how to write a Golang real-time app. And I just created the Docker files for it. And so you can see that I can do things. Maybe I make it a little bit bigger. Oops. Because one of the things that is cool in this app is the fact that below it's showing me which container is serving. So right now it's the same container, okay? So that's nothing special. If this guy, they're both connected to the same container when he says something, it automatically propagates to the other browser. Nothing special. But the cool thing here with the whole load balancer setup is if I go and use Docker compose scale and I say app equal to five. And it scales up five containers. So I can see I have container D94, BD58, a whole bunch of special containers here. And if I go to, I think, can you hear me actually when I'm talking? Because I'm like talking inside the box all the time. So, okay, so go back here, back to Safari. No, not the Safari, okay, here. So now if I refresh, then you see that they're actually being load balanced across the different containers. And now they're not connected to the same one anymore. But if I do test, hello, it still works across things. So it still propagates thanks to the redness that's running behind. So all of this is for somebody that just started using. I just had to clone the repo, just do Docker compose up dash D, go to my local host, run a scale command and see it's automatically scale and propagate. I think that's pretty cool. So, okay, so that's the first thing the developer experience, right? So yeah, you can just go to docker.com slash get Docker and you can install the beta for Mac which gives you this very native experience. Now, if you look at the keynote of the DockerCon, they actually have a much cooler demo which is running Node.js inside the containers and then spinning up a remote debugger with Visual Studio code. So he actually runs, he connects his Visual Studio editor to the container, the process running inside the container which can go through local host because it's exposed to local host and then he actually can live debug his code. So that's a step up from what I just did. Unfortunately, I was not gonna rewrite my demo. So the next thing is the orchestration bit. So the thing is right now, how many of you are like working with Mesos, let's say like, yeah, Mesos. Kubernetes, Kubernetes too. So how many of you are using Swarm for Docker orchestration? No, so basically there was a whole bunch, like I'm talking about Swarm before the announcements. The original Swarm was basically creating a master agent, I mean agents on every Docker engine and a master and then you needed to set up an LCD cluster, same with Kubernetes, you need to set up a config store to then do overlay networking and things like that. So it was all right, I guess you could do Docker machine up and things like that, but not as easy as it could be. So, and also I guess not really good for Docker's business model. So they came up with something new and they took the config store, they took the LCD parts of the config store and completely implemented it inside the Docker 1.12 engines. So they created a Swarm mode inside the engine, which does automatic cryptography, so you don't have to worry about the rotation of the TLS keys and there's an additional API endpoint, so complete new set of sub comments that you can do from your Docker clients and you can hit the same. So if you are familiar with Kubernetes, so I only saw one person, anybody or two persons, anybody else familiar with Kubernetes? So three, right? So if you're familiar with Kubernetes, the terminology is very, very similar. So first, it does self-organizing, well, Kubernetes doesn't do that necessarily like without a lot of setup, but this part is kind of cool, so let me show you that. So I've just used inside my repo, but I haven't pushed it up to GitHub yet. I hope, okay. Inside my repo, I have a Terraform plan and it creates, oops, sorry, I'll stick there. It creates the VPC, it creates a key pair, it creates three nodes and they're all running Docker 1.12. So I didn't have to do anything in my Terraform plan, well, except for all of these Amazon things, but to get Docker running inside these machines was very simple and well, I lost the connection again, so let me try, okay. And so here I have Docker version 1.12, release candidate three, but with experimental enabled. So it allows me to do some experimental things such as work playing with Docker application bundles, distributed application bundles. This, I think it's Ubuntu, really. Is it important, could it be any? I think it should be. I was mentioning CoreOS, like would it run with CoreOS also the same way? The way CoreOS works is it bundles Docker inside the CoreOS, right? So the- It doesn't have 1.12 yet. Yeah, so even the Alpha channel, they're still on 1.11 or whatever, but still you can force it. Like I did it in the past, like I just curled the Docker thing and put it inside user op-bin and relink it or whatever, or create it. I could force it, but obviously that's not perfect. So I was able to play with it, but it's not a good idea at all. Just when I was doing a demo, I was always using CoreOS, but for this, no, you won't be having CoreOS experiment, won't be having Docker experimental on CoreOS. I also prefer CoreOS, but okay. So this is running the, and I can do Docker swarm init. Before I do that, I have, so this is another application, hopefully loads. This is a visualizer, which is visualizing the swarm. Ideally, I put them next to each other, but I'm going to try and switch real quick. So when I init, it's a bit stupid to do it like this. Anyway, and then here I can join. So I'm using the private IP of this master. So after I did the Terraform, you can see the public IPs are here and the node IP is here, 10.0.1.3.7, and I'm joining here to 10.0.1.3.7. So right now this node joined the swarm as a worker, and let me do the other one as well. So yeah, but it comes up before I can switch to it. But yeah, okay, this is the visualization of the cluster. So I didn't have to set up an HCD cluster. I didn't have to do anything. Well, you don't know because I just show you these three commands and didn't show you anything else. But believe me, I didn't. So this is basically, I'm going to push this all into GitHub, but it's basically, right, it's similar to the demo. So the way I created the visualizer was using Minomark's visualizer, and then I was doing the init, and then I was doing the join. And well, I can also show you that the nodes now are Docker node. Is it node.ls? I think swarm, right? What am I doing? I did run through this before. I should just copy this thing. Yeah. So it shows me that they have a consensus that they are all in a swarm. Okay, so no external data required. And I can, okay, let's go to the next slide. One of the key things is that Docker engine, the new 1.12 Docker engine is taking care of the security. So it's automatically generating TLS certificates and automatically rotates the keys. So nodes are accepted to the cluster if they have valid keys. And if you want to revoke any node, you can take out the node by revoking the key. I haven't played with that though, so. And then there's a new service API. So we can, similar to Kubernetes replica controllers, we can create a service entity, which is then going to, we define the desired state and the cluster is gonna look at the actual state and try to run to or converge towards the desired state. So an example is to run this Docker service create, hello world, I can run it here. So right now the service is running. And if I look at the visualizer, it actually visualized it. So there's one service running. And if I do Docker service, I think it's scale. And then hello world, is it? Let me just confirm. I can inspect, I can scale hello world five times. I mean, to tell him that I want to have five of them. So right now there is only one running. Let me describe. Well, let me go to the visualizer because that's cooler. Oh, shit, it's too late, it's already running. So this is basically what they were using during the demos on the Docker keynote. And yeah, I'm too slow. I should set up my screen split, but, sorry, let me describe. I'm still not used to Mac. Okay, all right. So this is the desired state to have five replicas running. And if I were to kill one of the nodes, like, I haven't actually tested this. So this is live, but this is like from the demo. I should say no. Okay, so this one closed and quickly. Come on, that didn't do much. So it's supposedly one of them should go away and they should reschedule on the other one. I haven't tested this. So we want five. There it goes. I missed the magic word. I don't deserve anything. There was all the Docker people. So okay, so cool, right? Yeah. Is that called self-healing? Yeah, that's called self-healing. That's basically if you are familiar with Kubernetes, but it's kind of the same concept. Like, Kubernetes was introduced open source back in 2014. It became production ready in August 2015. And now they reached just released version 1.3, just like last Friday. And the whole thing of Kubernetes is to introduce concepts such as having replica controllers that match the actual number of services running or number of pods running inside the cluster. And then if one of them, you know, constantly like evaluates what is running versus what is desired state. And if there's a difference like, if you change the desired state, you only want four instead of five, then it will automatically kill one off. If you just change the desired state that you want six, it will create more. And if one of them dies, it again sees that the desired state was to have it running, so it starts again. So it makes the whole self-healing ideas much easier because it's based, it's called a reconciliation loop which comes from robotics, and which is a really cool thing that I think Kubernetes did. I don't know if there's any other framework. Does it come from somewhere else? Mrs. Duwitz. Mrs. Duwitz, okay, let's get on with it. I need to play more with Mrs. Duwitz. Okay, so rolling updates. So the idea of rolling updates is that you just go and change the image that you want to run. And then the dots in your desired state and then it automatically sees that it's not running the right one and it should update it. I don't have a demo for that. I already pushed it with killing a node, so. Yeah, so the rescheduling I was able to show. Then there's this built-in routing mesh. So it's kind of the same as Kubernetes when you, in Kubernetes you run a cube proxy service on every agent. And if you hit any of the agents on the exposed cluster port, then it will automatically route the call within the cluster to find the nodes that implement or the ports that implement what you need. So I don't actually have like a good demo of that here because the application I showed you earlier with the two browser screens, I tried to deploy it but it didn't expose the port. So I wasn't able to hit the port. So I unfortunately cannot show you that part. Okay, so that's all part of the, those are the four concepts that Docker 1.12 gives. So if you are running, if you install Docker for Mac like I showed you earlier, I can actually, I'm actually running this version. So I can do Docker Swarm in it and play with the service concepts and things like that on my local laptop. It works the same as if it's in the cluster. Except Docker for Mac does not run multiple processes, not no multiple nodes at this stage. So I couldn't do the demo that I just showed you on my local laptop because I'm just running one node. So I thought that's not interesting, so I didn't do it. But you can play with the service concept and the tasks concept. So yeah, you declare, I actually went through most of this, right? Scheduling, I mean reconciliation thing. The cool thing about this implementation, the internal distributed state store is taking a part of at CD. So it's using the gossip. Well, it's using rough consensus, I think. And then the workers, they are communicating with the manager using GRPC, which is like much faster than REST API. So it's not HTTP. It's like a binary protocol, right? It's like protocol buffers. It's from- Protobuff, right? So it's faster. And actually, Kubernetes, I mean, at CD3, the new version of at CD, which was released, I think, last week, they also implement GRPC. Okay, so they have a strong, strongly consistent and simple to operate and they're fast. Yeah. Okay. This is very much still a release candidate. It's very much in still under development. It's also, as far as I understand, it doesn't scale that high yet. Like right now, you can only run like, I think 40, 45 nodes or something with this engine, which still needs to be scaled a lot for, I mean, for like the bigger cloud players. This is important. This was, by the way, this comment, which I did not pull out of my head. I picked it up from one of the Kubernetes meetups. So I don't know. I actually should probably fact check, but yeah. Oh, this one I haven't played with yet, but this hell check, so you're able to define hell checks inside a Docker file. Anybody played with that? If you wanna see this, now all of the DockerCon videos have been posted and they're available on YouTube. So you can actually watch and see all the slides there. I actually wanted to do demo of this as well, but I ran out of time. Then there's this idea of a plugin model. I also haven't played with that. So the idea of installing plugins actually don't ask me questions about that because I haven't looked into it much. I just thought, as this is a recap of DockerCon, I should mention it, but I also need to investigate more into it. So sorry for that. Yeah, and permissions. Right, the several parts. So the several parts that actually take care of it actually, it's a rough consensus, like I said, but I haven't looked more into that. Sorry. I think that's about, I'm probably gonna keep it at this. Yeah, one more thing is that the whole orchestration part is completely optional. Obviously, if you are familiar with existing orchestration frameworks, this is completely undermining their whole framework because Docker used to be just a container runtime and now it's also doing orchestration. So it's not bad because, yes, it does make it easier for people. It also puts a target for other frameworks to achieve the same easiness for other people to use it. So I think that's great to make it like that. But if you don't wanna use it, you can still use Docker without it. And yeah, that's basically it. Oh, yeah, there's a whole, there's a third part of the announcements, which is that the similar to the Docker for Mac and Docker for Windows experience that is native or very native to the operating system, they also are working on Docker for Amazon and Docker for Azure, which are currently in beta and you need to sign up for it. So you go to docker.beta.docker.com, you sign up for Docker for Azure and then they will give you the resources for you to play with it. And they will expand it like they have, they reduce the amount of people that have access to it because otherwise everybody starts having problems and then they have a difficulty supporting people. So that's how, I don't have access to it either yet. Yeah, I haven't looked into that. So the last part that I want to quickly touch on is the distributed application bundle, a portable format for multi-container applications, right? I'm reading off the slide. So the idea is to just bundle your whole compose file. So the compose file that I had earlier, that's why I had two different ones because one of the compose files, I have the load balancer in front of it and then the other one, I don't have a load balancer because when I deploy my application, I just want it to deploy without the load balancer and I will let the ELB elastic load balancer from Amazon load balancer across my containers. So that's why I didn't include it, that's why I had to. And so when I have this, I actually have the Docker compose application here with also I can remove the, so this is the, it's visible, right? So I'm running the same, except I'm not running a load balancer. So I have two networks and doing this Mongo and Redis. So let's say I want to deploy this as a single stack, normally you wouldn't. You would put Mongo outside. I think definitely I would put Mongo using Amazon RDS, or sorry, DynamoDB or something else or a CloudFormation separate stack outside of my container runtime. And then Redis. Yes, I can bundle Redis to be scaled as well, but okay, so this is just a demo. So don't think this is like the way to do things. So I can do Docker compose bundle. And to do this, you need to have the experimental features enabled. So this is not yet in the stable branch. I think that's it, right? Gives me a bunch of, maybe that's why I didn't work earlier. I didn't look at the warnings before. Networks, volumes, fine. Okay, so I have now, I have the ECS sample.DSB file, which is a JSON format. Let me, do I have it here? No, shoot. I would like to have JQ. No, I don't have. So yeah, it's, I have it suited to this. I think this might be really small. Command plus, to make it bigger. But yeah. Command shift and command shift and command shift. I think you can make this bigger. Your key is there. Wait, what? WS access. Is it? Yeah. That's okay, I'll revoke it. This is not the onus B access key, by the way. This is my personal access key. Yeah, you can spin up a whole bunch of things with that. Okay, I need to revoke it. I didn't think about that. Okay. So, nobody's on the computer right now. Can you please close your computer? All right. All right, so this is, no, this key only has access to S3. You only have access to one bucket. I did proper. Yes, yes. This key is the, well, you can try it, go ahead. You can upload things to S3 and give me a huge charge on S3, but that's it. So, yeah, so it just has the application defined and you actually don't, they're actually gonna change the name. So, the only thing you have to do right now, if you try this out, you just ECS sample that and then, well, I have the same file here. ECS sample that and then if you do Docker deploy ECS sample error. Frontier not found. Okay. Why? Earlier it worked. Okay, maybe because I don't know. Okay, but I had, anyway, even earlier, I could do Docker deploy ECS sample and it's supposed to, I haven't actually been able to open the port and hit their service. So, believe me, earlier, I don't know why. Let's forget about it. This is, yeah. Yeah, what's the difference between the Docker combo server? So, Docker compose up is just, okay, if you do Docker deploy, it's gonna hit the new service endpoint and create the service definition with the desired state. And Docker compose up is just gonna create the containers, right? It's not gonna do self healing or any of that, okay? So, with Docker deploy, you're creating a service and then you can actually do Docker task, sorry, stack. So, the second new command is Docker stack and then you can do tasks and then ECS sample, but obviously my ECS sample didn't create so nothing is there. But you can then list all of the tasks and so the way it works is you have services which are the desired state and then the services spin off tasks and so it's kind of like Kubernetes pods tasks but tasks, they say it's different from pods because it can be a virtual machine, it can be a container, it can be a unique kernel, it can be anything. So, they wanna just decouple it from the container concepts. That's why it's called tasks. But yeah, so now you have Docker stack to hit the service side, so to define desired state, okay? So, unfortunately, even my deploy didn't work but yeah, I definitely wanna, I'll definitely keep playing with it. So, this was announced back in 20th June so I didn't have a lot of time to play with it. So, okay, so this is still very experimental and that was my demo and that's it basically, right? That's all I have. Okay, thank you for listening. I'm gonna talk. That was all right. Is that part of the question? Yeah, yeah, question. So, how is this, how's the service different from like compost? Like why can't it just add additional features in compost into the demo? Why? No, but you could point Docker compost to a swarm and then you can just speed up the containers there as well. Right? I don't know if the stream captured the question so I'll repeat it. So, your question was that why don't they just add these features into Docker compost? And actually they did, right? When initially they created the swarm, Docker swarm, they created the ability to Docker compost against the swarm and then experimental rescheduling and everything like that. Why do they go different now? I think because they put all of the, because they completely redesigned the desired state concepts, like they took this completely from the other frameworks such as Kubernetes to just implement a replication loop that is running inside the engine. So if you do Docker compost, it's very static. I think if you were doing this against the swarm, I don't know actually, I think they might be hitting roadblocks where the Docker swarm agent was working in a certain way. Maybe not to break backwards compatibility, I don't know. It's a good question. Maybe I can figure that out at some point. Feel free to post it on the meetup group if you found an answer. Okay, any other questions? Any updates on the Unikernals? Updates in the Linux kernel? On Unikernals. On Unikernals. They don't talk about Unikernals, they just say that they did a great job, the Unikernal team, for preparing the Docker for Mac because they're using a lot of the Unikernal technology to run a proxy and a VPN kit. I mean, all these frameworks that they open source recently to create these networks, so to create this X-Hive process running and then to use the, to create. One cool thing I should mention about the Docker for Mac, which none of the existing Docker machine implementations do, is that the Docker for Mac does the OS6FS like the I notify propagations from the local file system to the X-Hive process or the Linux machine. So, your question was about Unikernals and I just answered about Docker for Mac. So, I don't know anything about, I haven't seen anything about the Unikernal team. I think they're working hard on Docker for Mac and other things like that. So, and my opinion, I think that's far out in the future. Unikernals may come, yeah. I think it's a good idea, but yeah. When will it be ready? But the Docker 1.12? Yeah. I think it was, I'm pulling things out of my, but I actually have notes, they actually mentioned, I think it's in mid July, I'm not sure. There's also another thing that they want everybody, they're gonna do a hackathon because they just released like they open source, the hyperkit, a lot of the tools, the internal pieces. Even the swarm kit that is used by engine is also open source. So, they're gonna announce, they wanna do everybody, like every meetup community across the world to do another Docker hackathon. So, in the past they did like for the several iterations of hackathons, right? And if you participate, you actually, you get swag, you can, things, and it's something that the Docker, Singapore meetups groups should also organize and OnSB is always, I think, very eager to host. It's a great place to work. I think we can host all the people and we're not constrained by numbers. Yeah, we'd be happy to host. Right. And we, more food this time, I promise. Right. Awesome. So, to work and to play with these new kits and to, like, there's a whole ecosystem around Docker of tools that I think can be moved and take advantage of these new open source libraries. And I think it would be great. And if you are a good hacker, as it were, you actually can get tickets to go to DockerCon to attend DockerCon and things like that. So, it's, I think, really worthwhile. So, I think the Docker Singapore Meetup should, we will definitely try to organize something with that. Yeah. Universal control plane. Universal control plane, so that's, I played with it, yeah, but I, no opinion. Have you played with it? No, I like opinions on, is it good? Yeah, I heard from people that actually use it, that is really good. My experience, I played with, what was it before, TumTum, right? I played with TumTum before, before they were acquired and I thought it was a bit, I didn't, I had some weird, but I think it was okay, but at the time, we were very, like, minimal, like, tried to do and it's been up a whole bunch of containers on my, on my phone. I see that you use Terraform. Yeah. Terraform is better? I think Docker control plane is much more high level to manage, like, you define your services, you define your workloads, right? Terraform, really, it's very low level, just provision infrastructure. Even for, I mean, not provision, just create the infrastructure. And actually, if you want to do provisioning and things like that, you need to have something additional, like, Ansible or something else or another configuration management. Terraform doesn't do that. Right, good question. Anything else? Anyone? Yeah? Can you recommend any container auto-scaling? Container auto-scaling? All right. I think all those... I think they have. Sorry? In Marathon, they have, but if you're not using Marathon. Marathon? Business? Marathon. Oh, Marathon. Well, I personally like Kubernetes. And they have horizontal auto-scaling, so you can do that with Kubernetes. Marathon can do it. I think there's plan for putting this inside the Docker engine as well now. So, but that's definitely not in it right now, I think. I haven't seen anything. Auto-scaling, no. I haven't seen anything about the auto-scaling right now. But try, like if you're already familiar with Marathon, then actually all of these frameworks can probably run on top of the new engine. But like I said, this is very release candidate. So for now, you stick with what you have. Anything else? Anyone else? No? All right. Yeah, also one more thing, sorry. OnSB is hiring. So if you're interested in to play with containers, if you want to build like an infrastructure orchestration and things like that, please do apply. Very much appreciate it. I think they're a very nice host for this event. So just for information, I had to do it. No, I love it, that's why. OK, so I invite Ruby on real stock.