 Welcome to my talk. So this actually, originally, it was a share talk with Minio and myself. And unfortunately, they couldn't make it, so you just have to deal with myself today. The other person who was supposed to be joining me on stage is Krishna. And they are unfortunately, I think they are in some kind of retreat, all having fun while we're having fun at Mesoscon. Unfortunately, that's what they are missing out. So just to jump back into the presentation, in the beginning, all software was actually rather simple. We were just having, like, a big monolith. If we're looking at, like, the stack, we had some hardware operating system on top, and then our application, all very simple. Then came this new, like, very fancy way of saying we split our application into many smaller parts called microservices. So for me, there exists a lot of definition of microservices, and I have the feeling everyone has its own interpretation of it. What's kind of the important place or important parts for me is that microservices are designed to be flexible and resilient, meaning it's important that I have those well-defined interfaces between them, but the rest of the implementation doesn't matter. They are flexible. And this, in effect, actually gives me some nice results that I can be individually scalable, for example, which we're going to see here, also in this presentation. And those changes actually also have an impact on the application stack now and then. Back then, we had kind of like a LAMP stack. Today, actually, we have multiple stacks. If we're talking about microservices, it's usually like some Linux on top, then a container runtime, then container orchestration, and then a persistence layer where we actually store stuff, replacing our ancient MySQL, for example. And for the first trend, I guess all of us know here and have used Docker or containers before, or is there anyone who hasn't? Great. So containers are actually driving the stack and the two types of containers, which, especially from a deployment perspective, we need to distinguish our stateless containers, usually where we try to keep most of our application logic, and then stateful containers, which are just there to store all the data. The benefits of this division is actually like the scalability and deployment mode, those stateless containers. First of all, they can very easily fail. If a marathon detects a failing container, it will simply restart it. If we're doing the same with the stateful container, marathon, Kubernetes, all the container orchestrators, they can do the same. But it has more pitfalls because actually I have to check that I'm restarting on the same node where the data is, that I still have access to my data, that I don't have right conflicts to the underlying data store, so it simply becomes a different game. So kind of like the rule of thumb, try to get as many stateless containers and then move something into stateful containers where you actually need it. So looking at this stack in comparison to before, we have some infrastructure underneath. This could be AWS, this could be package, but someone who knows how to either provision bare-metal machines or knows how to provision virtual machines, something where it can run an OS on top. On each of those operating systems, I usually then have to container runtime, and then I'm running my services on top. Container runtime, just for those who might not be aware, I suppose most of us are, is actually there's much more beyond Docker, so it's not just Docker. DCS, as you learned last two days, has actually an in-built container runtime. There's Rocket, so there's actually a lot of choice what I'd pick as a container runtime here. The problem, if I arrive at this kind of stack, is that actually I need more. So this is the early stack at Twitter, and what happened actually at Twitter was every Friday those guys would meet, they would check, oh, how many containers have failed, how many of my services up there are still running, how many do I have to restart? So this is what I usually try to refer to as spreadsheet scheduling, because I have a spreadsheet of kind of like the target state, and then operators are going in on regular basis and trying to achieve this. And this is actually why we need something like container orchestration, something like Marathon, something like Kubernetes to basically automate all of this. And there are three main kind of like, yeah, operations or fields where a container orchestrator has to be aware, has to take care of. The first one is container scheduling, how to fit as many containers onto a ship, how to restart those containers if they are failing, and how to keep them running, how to scale them. Then a little, on a lower level, resource management, how to fix the right resources, where to put your container on top. Maybe the small ship isn't the best idea. And then service management, where we care about how can we actually connect multiple running containers, how can we identify where they run in the cluster. So just if you care, it's going to be in the slides in some more detail as I just explained it. And yeah, that actually gives us this stack. So just on top of the container runtimes, we have our container orchestration layer, which is taking care of all this stuff. And now let's actually, the talk is called Cloud Native Way, or maybe just first as a little reminder, we're not really trying to replace Dropbox here in this talk. What we're trying to look at is how would we reimplement it if we would start a company all over. And so looking at this chart from the CNCF, actually they're trying to list up all the different projects in this cloud native landscape. And you can see there are tons, a large number of projects. So I would actually rather look at the different layers which we have. On the lower bottom, we have the infrastructure layer. And as mentioned, this can be someone like Package, who is actually there. It can be something like AWS, GCE, Azure, which you also heard about this morning. So those are basically the infrastructure providers. Then I might have some tools of actually provisioning those servers. So they're Puppet, Chef, all those tools which actually helped me to spin up a platform on top. Next, then, is the actual container run times we saw in the layer. And as you can see, there are a lot more than just a docker, which is somewhere in there in the middle, actually. So we actually have way more choices than simply sticking with docker. Well, I'm very thankful to because they made it popular. But I also believe that choice and open source standards are actually a very good thing here. On top, we then have our orchestration management. So this can be Kubernetes, this can be Marathon, this can be docker swarm. So again, I have a large number of choices here. And then actually, I run my applications on top. And this is even where I have so many more choices, all the traditional applications like Postgres, like MySQL, and then all the actually cloud-native applications which have been written for this kind of stack. And this is also what, for me, kind of defines cloud-native applications which have been written for such kind of environment where I'm conscious about that there's a potentially failing infrastructure underneath which will automatically reprovision potentially. So from this very filled picture, let's go to a little more concrete example. And this is kind of going in the direction how we would implement, re-implement our Dropbox nowadays. We have our infrastructure layer again. We have the orchestration layer on top. And then we actually have our different services. And this is the second aspect of such kind of cloud-native architecture that I'm not trying to build this big blob. We saw on the second slide, I believe. But I'm really trying to break it up into several functionalities which I can then provision on such kind of cluster. And so in this case, we actually have a lot of stuff running in Docker containers. We have Minio running in Docker containers. We have MySQL running in Docker containers. Redis and then Node.js for the front-end part. And you remember me saying that we should try to avoid stateful containers? As you can see here, we actually end up with mostly stateful containers. So often in practice, it's rather hard to actually get to stateless containers. And often we unfortunately still up with a large number of stateful containers. The good thing is, in this picture, like the thing we can scale is actually the Node.js. This is what the customers will see. And this is what the first request will go to. So this is rather easy to scale. So we can still keep, like, latencies hopefully down, maybe not so much for the rights, but at least for the UI which is like the thing I feel as a user. Persistence. So as we were just talking about stateful containers, I actually took a look at Docker Hub and its top 10 images. And it's really interesting that, depending on what time I checked, it was a little varying. But somewhere between 70 and 80% in the top 10 and top 20 were actually stateful containers. And so this already tells us that it's really hard of getting rid of those stateful containers. And if we are choosing a stateful container, it's a tough decision. And this is actually, as we have so many choices, how we can store data nowadays, it becomes even much, much harder. So what we should consider when choosing one of those containers is what kind of data model do we have and how do I actually access the data? Do I mostly store JSON data? Then probably MongoDB is a good choice. Do I have simple key value pairs? Redis might be a good choice. And if I have just have blob data, so, like, a special case of key value actually where I just want to store, like, a big blob, then actually, like, those blob stores or object stalls, as they are called, like Minio or S3 might be a good choice. And so on and so on. SQL, Postgres MySQL, text, I can store in Elasticsearch as kind of a search engine. So just try to come up with, what is the requirement for storing my data and then decide which of the data stores might be a good fit for that? The good news is that in this architecture you don't have to pick just one as we all have those... As we're ending up with this microservice-like architecture, you can actually pick multiple. So one for each use case. But you should keep in mind that, of course, this increases the administration overhead. But it's much simpler as it used to be in the pre-orchestrator world where I actually then had to deploy manually on nodes. I can scale that rather easily, automatically on a cluster. But, of course, there's still, like, this overhead for each project I installed. If we look into Object Store, as we're talking about Minio here, we ended up with Object Stores. So, like, all the cloud providers actually have a solution. And this is why I personally like Minio quite a lot. Because it's actually... It's API-compatible both to GCE and S3. So I can use the same programs. I use the same APIs to write my data onto S3. I can use, on my local cluster, to write the data into an equivalent of S3. And we're going to see that in some more detail on the next slides. So Minio actually consists of the actual servers, which are going to store the data. Minio Client, which can write the data, similar to, like, an S3 client, writing the data. And then the SDK for writing applications against it. And Minio, it actually gives me different backends. So I have to decide when installing or starting up the servers how I want to store the data. The simplest option, and this is what I would use if I'm running it on my local laptop, would be just Minio's server and then some directory. And he would use that directory as a backend to store the data. But the interesting part actually comes in if I use multiple backends, because I can even do that on my single laptop. So once I start specifying multiple disks, what Minio will actually do, it will start distributing it across those different disks, and then also add a ratio coding. So I can be sure that not a single of those disks will be corrupt. Minio would detect that and automatically remove that disk from the serving of data. And I can do actually the same. Those don't have to be on a single machine, on a single host. I can actually also spread that out across multiple servers. And as I'm not with Minio, I can also say what I don't necessarily like about this, that it's actually limited up by 16 servers. And that means if I have really large deployments, what I usually end up doing, I have a proxy on top, which can then use multiple of those 16 servers underneath. So by just having this one proxy on top, I can then actually utilize multiple Minio servers on top for that. As mentioned, we have a ratio coding. So actually, when data is being split, so here we have 16 nodes, and for each block, we're going to split it into eight data blocks, and then we're going to have eight parity blocks, which are all spread out across different nodes. And that means that we can actually survive up to eight node failures in the system, because we can always reconstruct the data from the other nodes, which are still running in the cluster. And it doesn't matter because it's a ratio coding, which of the nodes are failing, as long as it's up to eight, we can always reconstruct our initial data. And this is actually giving me a pretty nice setup. If I don't have too much data, I mean, 16 disks is already quite a large chunk of data, which I can store in there, and as long as this is a good setup for my use case, I can be happy with using Minio for that. So as mentioned, up to eight drives can actually fail on the server, and what kind of their view is on then is that you migrate data to new servers as servers age. So you spin up this cluster with 16 disks in three to four years when that's actually filled up. You're going to have new servers somewhere in the other corner of your data center anyhow, and then you're going to migrate the data over to a new cluster. And this is why they also have this view of spinning up multiple Minio instances. So they don't expect actually to scale more to 16 servers for Minio cluster, because it's so easy to spin up the next cluster next to it. And this is the other kind of notion in this cloud-native world that I can easily scale. I don't necessarily, I'm restricted to a big monolith. If I need more power, I simply going to deploy a second one and maybe have a load balance up front. So when you look at all the different cloud-native deployments, this is a very common pattern to have multiple instances scale out instead of scaling up. There are a number of companies using it. I can only go into detail into the Mesosphere use case, but as you see, it's also supported by package. So it's actually fairly widely used by different companies. Now we come to the problem where actually, which probably all of us know, is what is the challenge then for the data center. So we have seen how would we construct such kind of cloud-native app, but what's now the challenge to my data center? And this is that we often, as we have so many different services, we have Minio, we have MySQL, we have Spark. As we have all those choices, we actually end up subdividing our cluster into many subclusters. And this, of course, as you heard in the keynotes and probably heard multiple times throughout the last days, is leading to very bad utilization rates. And this is exactly why Mesos and DCS spring in to view your entire data center as, like, a single machine and then also be able to address it as a single machine. And now just to kind of a more concrete architecture, how we actually implemented a demo of, like, a Minio Dropbox clone. And so we have chosen, so what parts do we actually need? We need a web server where I, as a user, can see what data do I currently have, where I can upload data and just interact with my Dropbox account. Then we have an API server, because also I need to do that from an API way. So that's the second part. And then we actually need to store things. And we have to store different kinds of things. And so first of all, we need to store some kind of user data that users can log in. We can see what their quota is, how much data can they actually store. And this is, like, very relational data, so their MySQL might be a good choice. Then we might actually want to offer, like, a free text search index that I can search across my Dropbox and find documents which I might be looking for. Here, Elasticsearch is a really good option for storing such kind of text data. We might then have, if we're storing blob data, we actually have to store the blob somewhere. And there are those object stores that are actually a really good choice. But we also need to be able to retrieve them. So we need to have this kind of metadata, which blocks are connected in which way and what do I need to retrieve them. So we need some kind of a metadata index. And there actually, Redis might probably be a good choice to store it because it's simply key value pairs for each blob. We have some metadata which we need to store. And that already leads up to the demo. So I spun up a DCOS cluster upfront, so very simple. Not sure how many people have seen it. What I like about it is basically this focus on just how is my utilization rate, which, of course, is zero right now as we have nothing deployed so far. If we start deploying things, we said we're going to do a stripped-down version of this Dropbox. So we're first going to pick Redis. Great. Redis is being installed. We see it's staging. And next, let's actually pick Minio. Minio, wow, and it's already up and running. And as a last thing, we actually need to expose Minio. So Minio would now just be running in our cluster. As I actually care about the UI, I need, like, my load balancer to expose it. And therefore, we're going to choose Marathon LB. How many of you have used Marathon LB before? So Marathon LB is basically an HA proxy, which is running on, like, the external nodes. The external will reachable nodes in a DCOS cluster. And, hence, they can actually expose services to the outside. In this case, we're just going to use it to expose the Minio to, like, an external resolvable address, which, in this case, is our public agent. And here we already go. And the access key and secret key. This is the same as you would see with Amazon, where you also have a secret key and access key. And this is the same you would use when connecting from your APIs against it. The default ones are very simple. It's Minio3123. Yes. And we don't want to store that. And now we can actually, similar as with an S3, we can start creating buckets. So first test bucket. Now we have the first bucket here. Test2. And we can do the same from, like, any S3 browsers, where they're actually a lot of, and simply connect to this cluster, which is now exposed by, like, also the public IP, because we installed Marathon OB. The last piece missing, which we're going to use in this demo, is the API server. And this I will actually deploy from my other shell. It's the other one. So what we can also do, we can simply deploy stuff from the CLI. So this would be my local development environment. I have a very simple app definition, which we're also going to look at in a second. Marathon.app. Server2.json. And now we have another deployment. What I wanted to show you just for a very brief reason is how does such kind of file look like? So all it does is actually it's going to retrieve a Go binary because we said that our API, our app server, is written in Go. And the nice thing about Go is that in most cases, not all, it's actually totally sufficient to just have this binary compiled for the right architecture. So in this case, it's totally sufficient to just pull down the artifact and then run it. So you see, like, the command is just run the app server, and we don't have to deploy a full Docker container. We don't have to download an entire image. And you can, if you just compare sizes, like a typical Docker image across all its image layers, usually ends up somewhere between 130 to maybe up to 500 megabytes, or even more depending on what's inside. And this is actually just, I believe it's two megabytes or so. So if I don't have to rely on a container image simply because I don't need it, I can only pull the binary or the pieces I need, I can actually save a lot of bandwidth. The other thing which we see, and this is also very crucial if we are coming up with such kind of cloud-native infrastructure, we want the infrastructure to actually be aware of whatever is considered healthy. And therefore we define so-called health checks. In this case, this is an HTTP health check. So what it translates to is actually that from time to time we're simply gonna pull against the slash endpoint, so just simply against the port on which it's running. And then we're gonna see whether we get an HTTP return code in the 200 range. If it's in the 200 range, we assume this is healthy. If it's somewhere like 400 or some other range, we would actually say it's unhealthy. And we can even see that from our UI here. If we look at services, we see here our App Server 2. It's here, it's one task, healthy. We can now go in and scale it. As we said, this is also like an artifact or a crucial definite or a defining part of the cloud-native architectures that we can easily scale. So I can now scale it to two instances and we see one more is deploying. And now we have two running. If one of them would be unhealthy, it would actually be automatically restarted by the container orchestration layer. And this is why those health checks are also so important, because they allow the infrastructure to be self-healing if something is going wrong, if my server is crashing. I can also have more complex health checks, so for example with commands, where I'm checking some database state to just see whether the database is in a healthy state. All right, so this was like the quick way of rolling out our demo. What I would briefly like to talk about, because demos, actually, they're very simple and they are cheating. Rolling out a demo is something very simple. I just click install, you see it up and running, you see it green and healthy, and they're like, yeah, cool. But the hard thing is actually, how do we keep it up and running afterwards? And so the things, if you're coming up with your cloud-native infrastructure or your cloud-native architecture, is you should consider those operations which are happening after we just deployed and it's up and green. And those are, for example, configuration upgrades. If we are scaling, we just scaled that application that was working, but in production, we probably want an outer scaling component that will, if there are more users hitting the API server, which will automatically upgrade the API server and potentially, like, backing other services. Binary upgrades of both the application, I want a new Redis instance. How can I do this without affecting the rest of my cluster? Cluster maintenance, backup, restore, the cluster failing itself, and what's very crucial to all of it, and whenever you're running anything in any cloud environment, you don't even have to be cloud-native, it's just monitoring. There are always going to be failures, there might be network congestion, and you want to be aware of before your users are getting, unfortunately, aware of it. And then after you've gotten aware of it, the next step is you also need to be able to debug it. So you need to think of scalable ways of how you can actually deep dive into your infrastructure and debug such kind of runtime blockages, for which we actually had a workshop here where Johannes was part of a leading, so the kind of day-to-operations where we looked at how can we actually debug such kind of runtime blockages. The slides are online, nodding. Yes, so feel free to check out the Mizzles University session on those day-to-operations. Furthermore, as I said, monitoring, so collecting metrics of both the user apps of the cluster, of the containers running in your clusters. So this is something very vital. Also collecting logging information, which makes it easier to debug as well and might also have some... You might have requirements to actually keep logs, like access logs, depending on what the also government requirements are for your kind of business. Maintenance, so cluster upgrades, cluster resizing, changing network policies, and then troubleshooting, unfortunately, we probably all had to do unvoluntarily debugging, tracing, and just to make it easier to get acquainted to it, we would really recommend doing chaos engineering, so having a chaos monkey introducing failures so you actually get used to and you also get confidence in your infrastructure that it doesn't fail if there are simple failures because you have tested it using your chaos monkey. That's already all of what I wanted to talk about because it's the last session and I suppose seeing how empty the room is, people also want to go home. But still, if you have questions, feel free to ask them. Feel free to also ask them later on why are they very select channels. But right now is also a good time. Judith has questions. Then thank you very much for listening. I said, like, try out if you haven't and Minayo, I really like it, but just be aware of those scaling limitations for now and you can architect around it, but you just have to be aware of it. Thank you very much.