 Hi, good afternoon. My name is Ramesh. So, I'm from Indian National University. Today I would like to share in my story about how we did skating. On the social media side, I'm one of the climbers. So, a little bit about the process. I'm one of the climbers. I have a climber. I'm not in the world of climbers. I'm not in the world of climbers. There you go. So, I'm ready to get fast back here. Welcome to this place. Keep it closer and help me. That's all you want me to speak. Is it all over there? Yes. So, I have a great past right here. So, my life is one thing and skating up infrastructure or various clients. And I like numerous other sources of information. And it is like these are the solutions unique to the process. And these are the products that we are going to get in the future. So, this is the one of the great scale-up of clients. It has a lot of good client expectations. What they want to do. What they want to do. A lot of seamless nights. It didn't happen overnight. It happened when they are approximately from 6 months in the years time. And at times a lot of frustration when we are scaling up the systems with different challenges with different solutions. I can see fate also when you are scaling things. So, who are you? Are you a system administrator? Are there any system administrators? Please design. Okay. Great. Who are the developers here? Okay. And any business people here? Absolutely. So, the main point is when you are in a different system there are a lot of challenges. Important. Let's see what the project was. So, my client is the US based digital media company and they have a lot of digital properties. Right from any platform and you say network and radio stations and various devices. So, for them the project was they had a club, a social networking site and they were going to scale it up. They had already chosen AWS as a platform for this environment. And this site was for media spaces. The state of the project was it was not one but figure size. What was in English? What was in Spanish? What was it? Different markets. Yeah. The application that they chose was Jumla. So, to get into the market and to capture that particular segment they thought that Jumla would be a good platform so that they can go to the market and buy that subject. Okay. And that's it. As you guys can see it's a good thing to go with in the project. And at the end of the project you can see the system. This one. This one. This one. This one. Oh. I am I am I am I am I am I am I am I am I I am I am I am I am I am got and I am got One of the most, this was the screen of the program, we could go on. You know, I have to do this guy. These servers were there, the web server was something called any extra-large server on the server. And it's really hard to get a server server. One of the greatest thing to do was, they were trying to screen this vertically. You probably think, what the marketer end for you on this game is. Now, server stack. That was the system. Let me go over this. Now, what is this? What is this? I don't know. Where did that addition server wave come from? What is this? So, Joomla said, page-by-page application. Thank you, sir. He said, everybody has a part. Everybody is on a different page. It's me. Yeah. Database was the most nice thing to do. What would they have for this game? My question is, it is a game, not a game. And the thing was, it was a game. I could have won. What was the feature, sir? What was the feature, sir? So, for now, this is the entire version. One set was the third one. Second version was the third one. The second version was the third one. The third one was the third one. So, when we look over the immediate challenges, and we found the first challenge, the number of upgrade users. Approximately, as soon as the system on the graphic used to shape the possibility of a complete user, it used to manage that. What do you know? So, that was the expectation of the management. The first thing that you need to do is to scale it from what you can do to create a new version. In a way, if you have a user on the side, it should still perform. That was the first challenge. Challenge number two, sir. Do you know? As soon as they know, it's not very difficult to scale it very far. There are various things you can do. For example, for every page request, they are corresponding to 50 database calls. Next week challenge, suppose if there are 1000 being used in your servers, then correspondingly, maybe there will be 5000 database plans. Why is it important? Because that means scaling data is very difficult. We can spend hours and hours on Chungha forums and ISC channels, talking to various community members, even the project maintainers of Chungha. How do we scale? And for everybody, they can mention that scaling for databases, what you need to do is, take it to your office. So, you need to do the easy-going scale, or whatever, get a bigger database scale. How do you scale it for the right answer? When we took over, we were just given the access to database. How? Nothing else. No knowledge answer, no documentation, nothing. You go, you find out, you find out where things are, how they have been set up. We don't know where the document is, how many servers are there, how they have been set up. The thing that I am showing you, is after we have found out, we need to figure out how to set it up. So, there was no knowledge answer, nothing. Challenge 4, what was the gift for people? The big team has came from an S&M account. So, we were using S&M, as if something S&M. Everybody was taking their own sister, on a share, five sisters, and then they used to send their new corner to their relief manager, and everybody was using copy paste, and got the score, and then he used to operate that course. That was the state of the project, then he saw that things are there. So, I am telling you all these different aspects, it is not only your servers, it is not only your architecture, it is also your workflow, processes, how fast work, big other things in the network, in the entire network, and challenge fighters, in the counties, sometimes for no reason, there is no role of the server, there is no huge spiking that way, on the real estate, the site is not working, you are just getting a policy, that was another challenge. So, the first thing that we decided was to design a scalable architecture, for a client. Now, how do we do that? There are two ways to scale, one is to do it vertically, when I say vertical scaling, what I mean is, today suppose you have a dual core processor, which states two bits of land, as an easy to scale, or it is physical, it doesn't work. And you want to scale, one of the ways is, get a bigger house, maybe get a quad-core server, or maybe a quad-core server, with eight bits of land, 16 bits of land, second is, multiple instances, or systems, and attach them together, and use them, basically cluster them up. So, that is for that, one of the scale. So, rather than having one server, for example, one server, you have, say, five servers. And you distribute the entire traffic on five different servers. So, rather than having one server, one big server, you have five servers. They are close to it. Why? Because today, if three servers go down, for whatever reason, then that's equal. But if I have five servers, even if one goes down, the rest of them, which you want, they may be contained. Hence, I will request. So, we are conducting lots of different instances, in terms of large-scale, small-scale, scale, so when we say, which instance we want to use, quad-core server? How do you determine which instance we want to use, whether should I go with M1, M3, M4, X1, R2, how many of you have government data base? So, most of you would know why M3 is interested in C1, X1, accessibility. They are configuration based on the number of CPU cores and RAM. So, it's based on resource of having. So, let me say compare your server instance to it. So, you want to go out for a different thing. And you go out very less often. And we mean, for once in a while, you think that first, do you want to find a person? It makes sense to have a task. Whenever you try to just hire a person and then take it out on anything. This mechanism saves a lot of cost. Because the bigger instances cost more. Smaller instances have less instances. So, whenever you want, you can add more instances and take them out where you don't need them. The philosophy of actually using the cloud or data base is there you go. And you should only use what you need. If you are producing this extra, which you really don't need, then you are basically unnecessarily wasting your money on the interaction. And use automation. So, this was a design goal that you should be using automation so that we can scale up things really fast. So, shared is the tool that we chose. Why? We will come to that in future. So, as a part of the solution for every day work, we train that as a key on how to properly use it. How to go to school, college, merging, resolving conflicts. For many people these would be some different things. Some people don't know how to do that. Are we advocating differently? They might think that one should know all these things. But for a few percent, no way. There is a different background. For them this could be water less. So, we train the development team on how to use the keys and the workflow. And we gave them an entire workflow that this is how we have to do the entire development process. Another thing was collect and analyze statistics. Why is it important? Because if we don't know what the current metacritic is, how much CPU usage are we actually saving? What are the resources we save in our systems? How are we going to know whether my application is database-intensive, CPU-intensive, RAM-intensive, or disk-intensive? So, wherever the bottlenecks are, if you go to where the bottlenecks is, how are you going to solve that problem? It is somebody coming to you saying if you are a big successor, I have a fever. It's just a medicine to get rid of the fever. But if the root cause is not solved, the fever will come back. Similarly, in the infrastructure also, if there is a problem, if you don't know the problem, if you don't realize where the bottleneck is, how do you solve it? So, for that, without talking really, better thing would be to select statistics. And we did that itself. So, for application performance management, we chose Neonetic, very good Post-it platform for application performance management. It gives us good visibility on what our response is, what is the server response at, what is the end user response time, what is the database response time. So, at a very micro level, Neonetic gives us all those statistics to determine whether there are bottlenecks in the system. These head art developers realize where to improve, what to improve, which varies to, which varies to optimize, and so on and so forth. We used AWS CloudWatch for getting the server statistics. What is the CPU stage, RAM stage, memory stage, et cetera. Center login, very important part of the data infrastructure. If you have various servers, if you don't know what is getting logged, what is deleting your servers, it becomes very difficult, especially if you have 10, 20 of servers or however of servers. Logo to any server where SSH is being put in, and doing a tail on the system, would be not a very feasible or optimal solution. And Google takes for getting the pattern, how are user, how are users interacting with the site, which pages they are deleting most, what time of the day or within 24 hours, they were actually active on the site. So, with all this information in hand, we said we want to change the package. First thing we did was, we chose NJLX over Apache. There are various good points of choosing NJLX rather than Apache. If you do a Google search, Apache versus NJLX, what are the plus points, you'll get it out of it. I just mentioned one of them. One is it is event-driven. Second, it will not keep the connection open or together. Apache will keep on having the connection open with the user, for example. Then from what PHP, we shifted to HP FPM. So, HP FPM is a fast HCI process manager for HP-based applications. In our evaluation activity, we saw that at the lower stack, you have Apache and more PHP. The better performance, if you want to get a better performance, you have to use Apache and more HCI. On top of it, you have NJLX and NJLX and more HCI. And the best performance for HP applications currently is NJLX plus FPM. This is the performance. The database is MySQL 2.0. Your version from MySQL 5.1 to MySQL 5.5 gives you a huge performance. I don't have a grand strategy, but I will try to publish it on my website, so that you can go through it. So, in the Chef. What is Chef? How many of you are already aware of Chef? Show hands. Great. So, I will not take much time on the Chef part, because you already know that. Open those platforms, infrastructure as code, it can scale well, and it basically accelerates the time to market. It gives us a consistent solution to take power, always. Chef is a systems integration platform. What we will deal with is systems integration. You have various pieces like from web servers, data base areas to load by answers, from the property of the servers, and all those things. You take all these various pieces, and you make your work better. That is the system integration. Configuration management platform. Configuration management is a better mechanism, which has nothing but OS, and may be one of the successful ones. And then, from there, come to a state where you can deliver services, that you intend to do. Very simply, Chef can, Chef allows us to do whatever we can do, we can augment. That is what Chef can do for us. So if you can create a user body like this, Chef can do that for you. If you can make more things, applications, and data, and servers and services, Chef can do that for you. So one other thing is, you need to tell Chef what to do, not how to do it. That is, if they get one minute. So if you say, install a package, you don't have to tell it how to implement it. Chef is intelligent enough to know how to install that package. Alternate, alternative DevOps or platform. So one of the other factors for calling this concept is called as DevOps. Development operations. Am I together? DevOps. Historically speaking, there were system admins, there were developers, and they were working in their own cycles. They were used to work on their own stuff, their features, and their client requirements, and all those things. And system admins were, they were working in their own cycle. And they had their own agenda. That was our only contact. That had already been, it has been a part of furniture. System admins was a consistent state for the system. It was going to keep on changing the system. But for that is their objective, their goal. For developers, it is always kind of saying new features, every week, every day, you have to write new code, new features, new releases. That is at a developer's line. So new concept partners DevOps has worked on all this, so that a percent should know more development as well as operations. And how do we do it? We write a code, which basically get larger and higher in project size. Consistently, again and again. So how chef works? Do you want me to explain how chef works? Please show off your hands if you want me to do this. So what happens is, there is an administrator's workstation. As you can see on my laptop. So this is my admin workstation. There is a chef server. It is somewhere. There is a chef server. There is an admin workstation. Then you have your systems, which you want to configure. Now your servers, they could be a virtual machine. It could be a cloud instance. Or it could be a physical server. It really doesn't matter. It could be you for chef or any other platform. It really doesn't matter. So what happens is, if you can provision it, whether it is a physical server, virtual server or a cloud instance. It doesn't make any difference. So we write code on the admin workstation. We upload it to our chef server. And we tell it what to do. And then we deploy it. So there is a chef client that keeps on running on all the instances. Or other servers where you want to take out. So for example, I want to take out these servers currently. So I will create new servers. I will encourage chef client. I will add them to my chef server. And tell them what to do. I will show you the state or some snippets of the code. Which will give you some examples. So after a lot of redesigning, requirements and all these things, this is what we had. The current state. This is just one zone. Assume that it is a US data center. On the US data center, it is AWS, the region is US. Inside it is VPC. Any idea for people what VPC is all about? How many of you know VPC? Do you want to explain what VPC is? So VPC is virtual private cloud. The benefit of that is something like a virtual data center for yourself. You have your own private network, private submit. Everything is there in a jet environment. So inside VPC, there are availability zones. Amazon has various data centers. North Virginia has US East 1, 1B, 1A and so on. So as you can see, one of the availability zones. East 1A. And you have various availability zones also. And these are the servers in various subnets. And they have a NAT gateway to which they can interact with the internet. So what we did was we used ELB as a tool and the data we used to create the web servers inside. This was really relevant from a security standpoint of use because nobody could access the web servers or database servers or any servers directly at all. All the servers had private IP. So the only way to get to these servers was through an internet gateway which we had hardened and secured so that nobody unauthorized access can be collected. And this entire thing was replicated from Gears Zone to South American Zone also for our time. So at the end of this, we had approximately 23 servers in total. Out of that, many web servers, many some staging environments and so on. So people would say, if you split your jubla, what is the, how much time does it take to deploy one web server? Any idea? Anyway, so if you want to deploy jubla, what's the rate? Five minutes. Five minutes you are anchoring. Anchoring. Set up the server already. Yes, set up the server already and apply it. Okay, if we have an emcee of jubla, I think we can have it as well. Sure. Yeah, so 23 in any side. If you just take in the normal, you would have to download jubla, jubla, install it and all that. Yeah. Basically about whether you do your job or not. If you have to figure things out, you can take to us. Okay. So now, let me give you some of the steps involved in setting up this server that we are talking about. So you have to provide an easy-to-instance, okay? Upgrade and update the OS. Install LDP. Do people know what LDP is? Why do you need that? LDP is data type protocol. It is required so that your systems are running on time. If it varies differently, then you will get your system up and you will get screwed up. And not on the case. Install gate. Configure your gate keys so that you can pull the source code from GitHub. Create a document loop for the application for every website. Now, we have three websites. So I have to create different document loop folders. Should I say document loop or should I say just folders where you want to deploy applications? Modify the configuration file. So I have to take this entire code that the developer has written. Modify the configuration so that we point to the right database servers with the correct credentials. Then, we have to open five different files. Write from the configuration file and other files. Specific for searching and occupying position and language settings, etc. So those are there. Install and configure S3 rules. We were keeping a lot of our static files and huge data on the S3 servers. Condemnation. Install and configure PHP. When I say configure, you know, remember it's very, very small thing like increase the file upload size in the PHP.ini and other things that is required for your application. Configure NGNX for virtual servers. So you have to serve three different websites. You have to create a virtual understand, tell it where the document loop is and how to serve those things. Configure NGNX. So all our servers have to be configured, install NGNX, configure it so that it will send the data to the NGNX server for application performance management for us to get the status gains. Then we used to send our mails to the NGNX every day. So we had to use mails, configure mails. You know, it is a different story of how we did it, we tried sending mails, Amazon blocks a lot of certain number of mails in a day. So then we have to use the entire gateway servers, relay it to that. We applied for that these are legitimate mails and we are not feeling the right so we have to go do as what we are doing. So if anybody is interested, I will explain that also. Set of seven ground entries for every website. That one we did. Configure a central working server. So we used a service called as Login for logging all our logs centrally. So that was Login and it started when we were modeling. Any idea what modeling is? So Monit is a service which can restart or return your services. So your Apache or database or any other service, it is ready. What I can see if it goes down at the middle of the day, you have to make up somebody who will take you up and ask you to start again. Monit is a service which keeps on continuously monitoring whether the service is alive or not. Whether it is a working number. Yes. There are various automatic solutions for Monit. Supervisory. I think God or something. So we chose Monit and we went ahead with it. So now, can you tell me how much time it would take for us to implement this? So you were doing this before you started? So if you do it manually, how long would it take? This is a test. Just one server itself would be greater than that. More than that? Yes. If we had to do it manually, it would take 4 hours for us to get one server after that. Back to it, everything goes smoothly. Now, people who have done server-connected, they know it hardly happens that things go smoothly or the way they are. You forget the semicolon. You forget one step in between. No matter if you are given the entire documentation also, we take time for it and then you have to troubleshoot where things are going or why it is not serving, etc. With Chef, you could do it in mathematics. It takes approximately 5 minutes to bring a new instance up and then for the deployment, another 5 minutes. So why not use auto-scaling? Some of you might have that part. Why not use auto-scaling? Auto-scaling is a very good feature, but one of the difficulties is our developers are consistently pushing new code every hour, every day, throughout the day. So if I have to use auto-scaling, what it does is basically it takes an image of that application and teaches it. And then when I go to deploy, it will just create a new instance or do a mock-up and deploy it. So if there is an old code on that, your entire user base will be served with an old application, not with a new one. That is the problem. Otherwise, you have to keep on creating new MI images for auto-scaling. And if your developers are pushing new code every few hours, imagine what it will be. So some few statuettes of the code. So if I have to install post-tricks, it's in the main server. All I need to do is tell the chef server to write the code, package, post-tricks, action, install. That's it. I am not telling you how to install post-tricks. I am just telling you how to install post-tricks. Now if it is... I am not telling you how to keep the domain in the center for a very long time. Yes, so I can do that. So this code will just install post-tricks. That's it. Okay. How do we do it? On an RGB system, if we use on a 1.2 system, we can use MPT here. And for other systems, we will choose... because we are on the system. You won't have to tell what to use. Chef is also available in OSX. Yes. So Chef is available for Windows, Linux, A&A X, MacOS X, Solaris, now IBM, other system B and all those things also. Virtually for anything. Yes, honestly. We need all the things for particular packages. So either you can use your own app on increase or you can let it install it from the normal application. Say, if you have a local repository in your office and you are installing softwares, you won't want to go and get those softwares every now and then from the Internet. You can point it to the local app. Yes. So you can tell where to go from the Internet. Everything. So if you are assuming that you have configured the server app to go to the Internet and go to the normal repository and get it. So if it is AppGal, so if it is an Ubuntu system, it is ubuntu.org.org.org.org at the relevant package and store it. These are the templates which you are asking. How do we configure the package? Configure the package. So basically it is say take a template and install it on SACTC post-example that is the post-example configuration file. This one. This is the location. This is the source which is a ERV template. Something like what you are doing with PHP. Yes. So that path may be different on different systems. Which one? Post-example. Yes. So we can specify. This is what we are writing. So we can say that rather than on SACTC post-example you have to go to some other place. You have to write in something. Post-example setup. Post-example is not available on news projects. If I have news group, it may be somewhere else. But at the same time I want my chef recipes. My chef recipes to be dependant of his platform change when I am actually doing one. Yes. So we can specify that based on platform now. So if there is some platform specific code that you want, you can say, if I assume that my OSS Mac OSS, then this template should go to some other location. If it is over to, it should go to some other location. And so forth. This is what a template looks like. You can assume it like an ASP code. Even by ASP there is a different type. Even for PSP, it looks like almost the same thing. So you have the variables which you have specified. Like my host name. This is basically a node.name of the variable. Which will dynamically replace it once you get it. And here are other relay host and what is the relay port etc. I have to use it. So this is a DSL, Ruby DSL which specifies that I want to enable these postfits at one time at boot. And also it should start right now. That is what that could make. So some of the issues that we have are things that we have driven. Because RDS versus DB are easy to do. We were using MySQL from a technical perspective. What we want to be able to define once in RDS for once when we use EC2 instance. For encroaching MySQL. And RDS have been there if you use MySQL or EC2 instance it gives you much better performance than RDS. That has been our experience. Maybe everybody would have experienced this. We shifted from MySQL to MariaDB after we got acquired and MariaDB came to existence and they released a lot of new 2 ways in the portless. And horizontal scaling with Gelrata script. So we are in short of time but if you want to know more you can do MySQL plus frame in multi-master mode and scale it up as we do with web servers. So your database you write to any of the databases it will be replicated to other databases without any space. So if you want up to the data I will tell you what are the pros and cons of Gelrata script. Caching we had a very interesting issue with caching part and that white screen that I was talking about was because of one of the reasons I think it was called up cache. If PHP have up cache something so that was basically screen up. After lot of results and everything we used to try to debug it, print and everything even in scale it used to go and show the white screen. And that was caching. Once we take that out of the picture it was resolved. So we stopped using that we chose FunSystem cache for caching perspective and when we tried Nemcache and to our accomplishment Nemcache was not working as expected with Joomla. Everybody suggest use Nemcache with your Joomla application to get a good performance improvement. But when we tried to use Nemcache it was basically degrading that performance improvement. And that was because of all the Nemcache white screens. And for that we have even tried to that developers of Nemcache the project maintainers even gave it to us. So maybe for such a case instead we need to we tried a lot doing work for us. What is the reason Nemcache was not created? Nemcache has get request and put request. So all the put request which is basically write to Nemcache in memory was degrading. And because of that it was taking a lot of time. Now why we really don't know? We don't have an idea. We tried a lot to check with various other developers in the community to find out. We tried varnish really good really amazing performance improvement. Only thing we have not really to production right now is we got the session things. So if you want to log into the pages or the articles it will work fine. But if you log in to the site and if you log out maybe it will still show you guys log in. If you have logged out even still that was the discrepancy which we need to figure out that varnish session based things. Other than that varnish is really amazing for caching purpose. Security we had this environment on VPC different subjects different security groups everything locked down and on top of it we were doing activities of firewalls to further block the system and only keep it off any patients. So no way can anybody get into the system that will do our systems and hack. And to manage this entire thing that we are talking about it doesn't take only one person. Any money. Benefits as we assume on high availability lot of service we basically scale the system 10 times more some specific question and that's more users than it was actually when we took off when we took over the project. Learnings it takes a lot of time and effort to do what this is. It can't be more than it can be more than one. It takes time and error thinking out of the box. Communication between the teams are really important because if your developers are not supporting you if you are not supporting your developers you will not achieve what you have intended to achieve. Work is a team really important. Questions. So we are out of time you can make questions offline. And thank you.