 So yeah, so my talk is going to be about how we are using Kubernetes in our project. This isn't going to be a very technical talk, because a lot of people don't know what Kubernetes is. Kubernetes is, and it's a pretty new thing, though it has been there for quite some years, and Google has been using that in their projects. So right now, what this talk is basically going to cover is why you should use Kubernetes for a project of yours. If you have a very big project and you would like that to handle a lot of load, or if you have different parts of the project which you need to be running together, then I would suggest you should use Kubernetes. So it may seem like I'm advertising for Kubernetes, but yeah, so when I say about OpenEvent, I'm not just talking about the OpenEvent server. The OpenEvent ecosystem consists of a lot of things. So right now, these are the major things which the OpenEvent project consists of. We have the Android App Generator, then we have the Web App Generator, and we have Scrapers. And along with that, we have a companion organizer app which the organizers can use. So let's go one by one to see what each of these things consists of. Yeah? So let's start with the Android App Generator. So the Android App Generator has all these components. So we have a Flask web framework which provides the interface for using the Generator, which Maria showed a preview of sometime back. And then we have to use the Android SDK to build the actual final APK. And then we have a celebrity job queue because when a lot of people want to build multiple Android applications, we don't want all that happening at the same time and crashing the server, right? Because compiling an Android application is going to take up a lot of processing power. Yeah? And once we start getting a lot of load, we may want to run parallel views to process multiple Android application jobs privately in different servers. So we have a celebrity job queue for that. And the communication between the web framework and the job queue is kind of handled by Redis. Redis acts as the data region between both of them. So this is what the Android App Generator consists of. And then next, we'll move on to the next part of it, which is the Web App Generator. Web App Generator, the setup is pretty simple. We have Node.js and Express.js as the web framework, which will be handling everything. Next, we have open event scrapers. So open event scrapers are used to give data from other event websites. Like, for example, you have an event or event project, let's say. And you want to move to the open event project or event year. So instead of manually, you're having to get all the data from event right and typing it into open event. You can use the open event scraper, which can automatically scrape the event right website, page of your event, and then put it in a format which you can quickly import into the open event server project. So that, right now, we support only event right. And it uses Python. But in future, we may have other modules which is for different services. So that is what the foo and bar blocks are because the future development. So these three are the main components. Organizer App, the only component about it, there's no server component for the organizer app, which is because it's going to use the open event server API. Now, this is the main part of the project. This is the server and API in which all the other projects, other app generators depend on. So for the open event server, we have four main components right now, which is the flask web framework again. And then we also have a salary job queue, which handles import jobs, export jobs, and also sending emails for now. Then we have PostgreSQL and Redis. And in the future, or at least for this GSOC 2017, we have a plan of implementing elastic search in Kibana for large scale data searching and analytics. So if we have that, we are going to have those boxes also for the part of this project. So as you can see, the whole open event ecosystem is now very big. Initially, we started with these four things. And then each of these things had a lot of other components. And so, for example, Saptap gave us the deployment process of open event server alone. And as you can see, the whole process was very long. And imagine a similar process for each of these things. And let's say you want to deploy this whole ecosystem somewhere. So the job at hand is going to be very hectic if you want to sit and run each of the commands for each of the deployments manually. And like Saptap said, the deployment process may vary depending on which operating system you're going to be using, which distro. So we need a way of unifying the deployment process across different platforms. Yeah, so for that, let me start with a small example, real world example unrelated to programming. Let's say you have a lot of products in your hand. Let's say you have a car, you have a bike and everything. You want to transfer that car and bike from one country to the other. So let's say you want to ship the products. So a bike is of a different size. The car is of a different size. So let's say you just take your car to the shipping port and then now the shipping port will have a crane to lift things, right? Now that crane tries to lift your car and not be able to lift the car because the crane is not meant for your car. And let's say they make a crane for your car. And then if that crane you're using to lift the bike, it will not be able to do it, correct? It's a very simple example. So what do you do? You put everything inside containers. Yeah, an equal uniform containers. So you put a car in the blue container and you put the bike in the red container. Now any crane in any part of the world will be able to lift this container, correct? And not just cranes, like if you, now you're shipping to another country and then you're putting it on a truck. Any truck can carry this container with respect to what is inside. Yeah, the truck or the crane or the ship need not know what is inside. For the ship, all this is the uniform container. Yeah, it can easily carry. In the same way for programming or applications, computer applications, we have something called as containerization. So what containers are in computer terminology is it's an isolated, let's say an isolated process space where you can run your applications, run your applications or servers or whatever in a very uniform way. That is, so what containers are is they are an isolated process. Containers have an isolated process space and a network interface. That is to anything that is running inside a container, let's say you're running a Flask server inside a container. That Flask server is going to think that it's running inside a whole new computer on its own. Yeah, the Flask web server is not going to know on what host system it's running in. The container can run on Mac and Linux and the results will be the same. So for deploying, all you have to do is just say I want to deploy this container and it'll get deployed. None of the installation process will be there. And the programs inside the container will have full root answers also. And you can't give any more excuses. It works on my system, but I don't know why it's not working on your system. Because when it comes to containers, if that container works on your system, that same container will work in the same way on someone else's system also, as long as it's Linux or Mac, for example. So that is what is containers. Now containers is just a box which is having one component in it. So now when it comes to a big project like OpenEvent, we are going to have multiple containers, right? The post-crispial database is going to be sitting inside one container. Let's say the Flask web server is going to be sitting in one container. Now we need a way to manage all of this. So that's where this guy comes in. Yeah, the cube there now. So what this guy does is, this guy is going to help us manage all these containers. So this guy is kind of like a ship. You can get it from the logo. You're going to put all the containers on this guy, and this guy is going to manage the containers for us. This guy is going to take the containers wherever you want. Let's say you want these containers deployed on AWS or Google Compute Engine. This guy is going to help us do that in a very simple way. So that is what Docker is. So in general, Docker is a software container management platform, so that is the proper term of whatever I said just now. And it's open source, so that's good for us. Come on, we are false. Then you can run any application, any language inside the Docker container. So I can run my OpenEvent server project. Hush can run this Android app generator, and IH can run this web app generator. So it's not going to be an issue. And it has built-in container orchestration. So orchestration is generally managing a lot of things together. You know, you have an orchestra. You have a lot of people singing. So a lot of people singing together in a very organized manner. So that's an orchestra. So same way here, container orchestration, it will be able to organize all your containers in a very manner that you define like how you want them to be organized. Then it's again, same what I told some time back, build once running. You just have to build a container once, and you can run anywhere from the image. Now this is just one ship, right? Yeah, so I have one Docker, one ship like this. Let's consider this a ship. One ship like this for my OpenEvent server project, one ship like this for the Android project, and one ship like this for the web app project. Now we want all of these to work together. So for every ship needs a captain, right? So that is where this guy comes in. This is Kubernetes. Kubernetes is the captain for this ship, okay? Kubernetes uses Docker, and Kubernetes is going to tell Docker how it's supposed to do things and where in a very awesome manner than which I'll be showing you now. So the name Kubernetes, in Greek actually means captain, master, governor, pilot. Yeah, so the name exactly says what it does. So Kubernetes is an open source project by Google, which was built by Google for their own purposes. So most of Google servers right now use Kubernetes for deployment, and you all know how reliable Google servers are. We test our internet connection with Google servers. Yeah, we just open google.com to see if our internet works. So this is the definition directly of the Kubernetes.io website. Kubernetes is an open source system for automating deployments, scaling, and management of containerized applications. So let's see how each thing works. So Kubernetes has horizontal scaling. So there are two types of scaling in general, which is vertical and horizontal. So vertical scaling generally means increasing all of your resources, like increasing your RAM, hard disk and everything. Horizontal scaling is increasing the number of machines. You can either have one machine with a lot of RAM or multiple machines with a nominal amount of RAM. So Kubernetes does horizontal scaling. It does not increase the resources, instead it increases the number of instances. Then this is the part of Kubernetes, which I love the most, which is self-healing. Let's say you have a deployment version one running, and now you're pushing version two deployment. And for some reason, your version two deployment has an error. So what Kubernetes does is it switches back to the version one deployment, which does not have an error. If it notices that the version two deployment has an error. So in Kubernetes, let's say the version one is deployed to the public. The customers are using your website. So when you say Kubernetes, you want to deploy version two. What Kubernetes does is, instead of quickly switching the version one to version two for the public, it first starts the version two, deploys the version two in a sandbox state. And then you can define certain health checks on your deployments. So what Kubernetes does is, it runs the health checks on those deployments and first checks if that deployment works. And only if it works, it switches the deployment to version two for the public. Until then, it keeps it inside. So the users will not have any, users will not be facing any errors due to messed up deployments because if the deployment is messed up, the user is never going to see it. It all happens internally. Automatic roll, that's what is automatic rollouts and rollouts. Like you tell it to put an update and if it doesn't work, it'll roll back on its own. You don't have to tell it. And then there is load balancing. So you have multiple instances of the database server running. Kubernetes will automatically adjust which database instance to use, depending on how much load is on each of the instance. Then storage orchestrations. Nowadays, we have a lot of storage options. We can either use the local drive of that machine or you may want to use S3 from Amazon or the Azure's close storage servers or you may want to set up your own NFS storage server. So Kubernetes support all that. Then batch executions. Now I showed you the whole open event ecosystem, right? For the open event organizers server, we have a set of configuration files in different folders for communities. All I have to do is tell Kubernetes to recursively deploy everything in that folder. It'll just quickly go inside that folder, see everything and then deploy. In a matter of few minutes, I can have the whole open event setup running on a server. Then predictable deployment. This is the same thing which I told you in containers. If it's gonna work on one machine, it's gonna work on that. So if the Kubernetes deployment works on Google Container Engine, it's gonna work on the Amazon Elastic Container Cloud also. Then optimize resource allocation. So this is another cool thing about Kubernetes. You can specify when you want multiple instances of certain things to run. So you can tell Kubernetes that in case the system load on the web server goes beyond 50%, create another instance for the web server. So every time the load for the web server goes above 50%, Kubernetes will automatically create another instance for the web server. And then now there will be enough two instances running. So now we are using additional resources. And once the load goes again back below the threshold, it'll remove that extra instance. So now we'll not be using that extra resource unnecessarily. This is just one example. So Kubernetes does a lot of things like this. Now let's see what a Kubernetes cluster is. Yeah, this is a bit, right now it's getting a little bit technical. So each of these things, Cubelets, each of these Cubelets can be a physical machine. Yeah, let's consider each of these Cubelets to be physical machines. So you're giving control of these three physical machines to Kubernetes, to the master. Master is going to control these three Cubelets. So the master is free to create containers wherever it wants across these three Cubelets. Now inside each Cubelet in Kubernetes, we have something called as a pod. A pod should contain at least one container in minimum and it can contain a number of volumes. So a pod can contain two containers connected to one volume or one container to two volume. A volume is generally a storage space. It could be an NFS server or it could be local space or it could be a GhostFace cluster. That's your wish. So a pod is the minimum computational unit in Kubernetes. You can't go below a pod. You always need a pod. Inside the pod, there'll be a container. So for the OpenEvent project, we have a pod for the web server. We have a pod for the Postgres QL server, like that. And then we have something called as a replication controller. So what replication controller is, you'll tell replication controller how a pod should look like. You'll give a template for the replication controller and you'll tell how many instances of this pod should be created on what conditions. So what replication controller does is, depending on your conditions, it'll create a number of pods. So this is what I told you some time back about the load. In replication controller, you'll specify that for this low threshold, I need so many pods. For this low threshold, I need so many pods. So depending on how much load threshold you're giving, it's gonna create that many number of pods. So let's say all of a sudden, your site is getting a lot of uses. Yeah, so now there's what even, like a site like event A can get a lot of users during that event day. And in the next day, the user count is going to go down. That's how it varies, right? There'll be peaks whenever an event is happening because all will be checking schedules and everything. So during those days, you can't have a big DevOps team sitting and seeing which event is going to peak now and create extra servers, right? Which is really not economical and efficient. So that is where a Kubernetes and replication controllers come in. Replication controller will do that for you. Like as an asset seats a load peak, it'll create pods for you and separate, and plot pods for you and balance the load across the pods. So how does the balancing work? So let's say we have the PostgreSQL pod, yeah? PostgreSQL, let's say there are three instances of the database pod. And your web servers will not know that there are three instances, yeah? Your web server is just going to want to access the database. Your web server does not care how many instances of database you have. All the web server wants is access to your database. So irrespective of the number of pods, you'll have one service attached to the pod. That service is going to do the load balancing for you. You see an IP address written on top of there, right? It's just an example IP address on the port. The web server is just going to access that IP address for the database. And then depending on which pod has the list load, the servers will connect the web server to either one of these things. The web server only has to access the IP address. The web server need not worry about which of these instances to access. Yeah, so that way the servers will give you the one with the least amount of load. So whatever I've told you just now is just a fraction of what Kubernetes is capable of. I'm still learning about Kubernetes and I started using it only a few months back for event year as a part of my last year GSOP project. So this is a complete open event ecosystem. Okay, whatever flowcharts I showed you before and put it all in the same page. So imagine doing all the deployment process manually for this whole setup. Yeah, so for the open event server alone, the process explaining the deployment took 30 minutes. Just explaining it. Imagine doing it on a system for this whole ecosystem. It's going to be very hectic, right? Hectic and probably not practical and possible even because something that works here may not work on the other. And imagine, and you may not, you may want to run it on different in Iran. It's not everybody wants to use Google Cloud, someone wants to use Amazon, Amazon AWS. And someone, if they are very concerned about security or someone is very paranoid, they may want to run it on their own servers. They may have a whole bunch of servers sitting on their room and then they want to run it there. And the steps are going to vary for each of these things. So you can see the number of combinations you have, right? So that is where something like Kubernetes comes into picture. So Kubernetes can easily ensure that this whole setup can be deployed in a matter of minutes. Deploying open event server alone to Kubernetes takes me around 15 minutes via Kubernetes. And I don't have to do anything. I just have to run a command and it's going to do everything on its own. It just loops through all the directories and then it just deploys it. So it's very easy and awesome to use. I kind of love Kubernetes, so that's it. Thank you. Any questions? And if you are interested in this line, you can download it here. So thank you. Any questions? I'm happy to answer. Nothing? Okay then. Thank you. Yeah. Yes, sir. You said that Kubernetes can automatically balance the load. Yeah. Okay, so I think there are many other services that do the same. For example, Azure does the same thing. And there's Distortion, AWS, other things that do almost similar things. So any reason why you picked up Kubernetes? Just because it's open source? That's one reason. One reason is it's open source, yes. And then compared to those things, like what was the other thing? Distortion, Azure, and AWS. Okay, Azure, I don't know. But AWS, I have used it a bit. So in AWS, most of these things, as you can see, you'll need to tell it for this. There's no easy way of telling it in AWS. You'll have to manually specify any project, right? You have to go there, and then you can either use the command line interface, or you can use the web interface and tell them, like for this, you have to increase the load. Like for this part, depending on the load, create multiple things. But in Kubernetes, this is part of your part configuration itself. So when you're deploying, it's just, you just run a command, all of these configurations go along with it. You don't have to, again, go to the Google container, Google Cloud web interface for this, any of this. There's just one step deployed, and you're done. All of this is part of that step. It's just easier to use. But then it's personal preference. Mostly it's personal preference. Some people want to use Amazon Container Service, and I prefer this. So if you feel you're comfortable with that, you are always free to use that. Thank you. Do you have any more questions? I have a question. Can you go to your previous page, because it's a little tricky. What is the yellow part of the process? Huh? There is a duplicate of us. That is for the Android App Generator. I see, Android App Generator is smaller than this. Yeah, Android App Generator's packages are the yellow ones, and scrapers are the purple ones, and the blue ones are the silver ones. I'm just grouping everything together. How much does it cost to run this complex on Google? Right now, we are running only the opening and server in Android App Generator on Google. Billing, I'm not sure. Maria would know. He's the billing owner for that project, sir. Thank you. Can we put all these components into one instance, the physical instance, then manage it by one again? Technically, yes. You can put all of this in one instance if that instance is big enough. Because we have RAM constraints, right? RAM constraints, you need a bigger RAM, then hard disk space is fine, you can manage it, but RAM is a big concern, because especially some things like the Android App Generator is gonna require a lot of RAM value. But then the Kubernetes is still working, right? Kubernetes, we are using right now, our event A site is using three machines. For those low balancing, all the features are still able to work. For those low balancing, all the features in the Kubernetes will be still working in this condition, right? Yeah. Thank you. Thank you. Thank you. All right. Thanks again.