 I am Vijayanan, so I am the Chief Cyborg at CIVILA, so today I am going to talk about capacity planning. So the first question is, so why do you need to plan for capacity, thank you. So any answers on why you need capacity planning, why you need to plan for capacity scaling? So if you do not do any capacity planning, I am sure you are going to end up like that. So what is the capacity? So we build web apps every day, so each web app has a different purpose. So the capacity is not universal for each web app. So if you take a blog, the capacity of that blog is the number of page views or the number of comments it can serve per hour or per day. And if you take a forum, the number of pages, the number of posts, the numbers can post or the number of registrations can handle. That is the capacity for a forum and if you take an e-commerce application, so the number of searches, the number of orders it can handle per hour. That is the capacity for that e-commerce application. So before you go on planning your capacity, you just have to define what are your actions? So each action will contain like a couple of user actions like visiting a page, adding some items to their cart or clicking some buttons that will generate more page views. So for each action, you need to define the performance threshold. It should be completed within like 10 seconds or 20 seconds. Then to simulate the real load, you have to take a combination of these actions. Then plan for that one. So Scribbox is one of our clients. So Scribbox is a mutual fund broker who helps you invest in mutual funds online. The main flow for Scribbox is the signup flow. The flow contains generally like 1, 2, 3 landing pages, the 1, 2, 3 landing page views and a 3-step signup process, the entire signup process has like some 40 fields to fill. So I just have taken the threshold for completing the signup as 20 seconds and the capacity I am planning to support is around like 50 user signing up concurrently. So this is the methodology I am going to follow. First I have taken an initial deployment architecture. Then I simulate the actions of the users using Capybara. Then I calculate the number of concurrent users. This architecture is supporting. Then if I have not reached the goal, again unless the data I have collected, I will find the bottleneck and I redeploy with a new deployment architecture which rectifies the problem in the first architecture. Then again perform the same similar actions and iterate till we reach the goal. So why Capybara? Why not? You can still benchmark your applications or plan for capacity using Apache Bench. So if you take Apache Bench, it can only simulate get request but generally raise applications as more posts request like many of the users are going to submit forms on your interact with the website. So it will generate not only get request, it will have also many adjusts and other type of request like put request. So also the Apache Bench, we are going to assume it only hits a single URL and it is going to hit it repeatedly. So when a user loads a page, that page is going to load assets, all the images and everything. So Capybara actually puts the near real scenario load on the web server, on the whole system. So this is the measurement setup I have taken. So there is the master control. This is our web application which will be sending commands to the RabbitMQ which in terms relates these commands to the Capybara process. This process will be running in a distillation nodes or you can run it on any other high memory server. So once master control sends these commands to Capybara node to suppose similarly sign a process on the web application. Then we have the production setup of the actual live website. This will be set up in another kind of instance. And the Capybara process actually runs the signup process and push back the test results like what is the total time taken for this signup process again back to our master server. And in the production server it is going to send us, publish its CPU usage, memory usage and the distas along with the breakdown of the CPU usage and memory usage for each process. So generally we use Capybara for acceptance testing. It actually runs on the local RAG server which is booted by the Capybara. So this we have to configure to run on a remote host instead of a local one. So we have to give our measurement host where our production environment is set up. Then we have to tell it not to start a boot of the RAG server. And after including the Capybara DSL to run that test, so if you are writing an acceptance test you can almost take the specs from that acceptance test with minor modifications. You can just put it here and you simulate your user actions. Then after simulating it and publishing your results back to the master, you have to reset your session and cache. So this will actually simulate a new user. So first we start with one Capybara process. We give a command to one of the nodes to start simulating sign up. So once it has done the 10 sign ups we issue another command for another process to run simultaneously. So after 10 sign ups we have two Capybara processes running the sign up process. So like that we continue increasing the load at regular intervals until we see that the service is resets are maxed out or if you have very long sign up times. The initial deployment architecture I have chosen is the single machine architecture where we have all the Apache, the delayed jobs and the MySQL process and everything is in the same box, the same machine and we collect data about what is the CPU usage of each process. So if you take a look at that the average sign up completion time for each sign up is around like 90 seconds and we have eight concurrent Capybara nodes running. So after we have added the ninth node if you see the total sign up completion time goes on increasing. So we are taking the eight until eight node is optimal after the eight node the sign up time increases. So for our calculations for users to have the optimum what you call the best experience the maximum concurrent Capybara process can support is around eight. So the entire sign up process has around like six page views. So now user comes to scriptbox.com they just go to three pages and just start filling the sign up form. Between these performing two actions on the website there will be something called thinking time. They will be thinking, they will be reading some content or they will be thinking what to fill up in this form field. So that is what I am calling as a think time. So on an average the think time for between two actions that will generate the next page view is around like 10 seconds. So between six page views we have around like five think times which will have more like 50 seconds. So the total time taken for a normal human visitor to complete the sign up process would be around 69 seconds. So if you just convert this hour how many concurrent users it can support it's around like 29 users. That is a concurrent users signing up at the same time. So our goal is to support 50 users but it's still you know we have not reached our goal. So now if you take a look at this graph the green one is the passenger CPU usage. The passenger is taking the highest load. So we are going to scale the passenger service in the next architecture. So I have taken a new architecture we have a load balancer as NGNX which will be passing all the request to the two passenger servers which in turn you know talk with the MySQL server. So we again repeat the same experiment on it and we just you know graph the whole data. So now it is for the same 90 second completion time it is supporting up to 14th node. So in this architecture we can support up to 14th node for that minimum average time of 90 seconds. After 14th node the sign up completion time you know increases continuously. So this is the memory usage if you just take a look at that the passenger servers are maxing out on the memory of like 8th node after this stays constant but still up to 14th node they are still able to you know serve the optimal time. We just convert the same you know the same type of processing to the concurrent users the concurrent human users so we get you know 5th one up 5th one users approximately. So we have reached our goal. If you take a look at the graphs the passenger server is still you know maxing out there on the memory. So we might have to increase the passenger what you call the resource of the passenger systems to support more passenger processes. And if you take a look at the MySQL memory usage it grows continuously as we know increase the load. So it still has not reached the 100% memory usage but I think you know after once you scale passengers then the MySQL surely will be you know hitting the its bottlenecks. So that is recap so what you have done so we have taken a single machine architecture then we simulated the you know sign up actions we have not reached our goal. So we determine the bottleneck as a passenger we scale that passenger server then we again we deployed in a new architecture okay and again we you know measured the capacity of that deployment architecture. And then we reached our goal of 50 years. These are the references there is this local acceptance they have the you know the methodology on how you can calculate the concurrent users and these are some of them I have used this for the data visualization and the RabbitMQ server for you know messaging between this cabibran nodes between these two between the different process you know running on different servers okay and for the system metrics you know collection I have run the you know the command called PID stat which actually breaks down the system research users for each process. So you can give a time interval of 10 seconds or 1 minute I have collected it for like 10 seconds which I found it is a little bit too much sampling you should I think maybe you can increase that time to 60 seconds. So for every minute you can just collect the you know resources for each process then actually calculate which process actually taking more resources and scale that is it. Any questions? Thank you.