 Hi everyone, I'm going to speak about Bikflikov, Babov, Bikflikov, Puppet, and Hyra, and how all the things match up together, and how their humanity means to me and that sort of thing. I work for NGTV Convergence, I manage their infrastructure, and how NGTV Convergence is a fully owned subsidiary of NGTV we take. There's supposed to be some slides here about the numbers of what we're doing, like the unique movies, entertainment views, and all that stuff. I don't think you're here to conduct, the traffic is going, each month, the mobile is sort of catching up to the desktop users, and how we are organizing. We basically are a news media organization, so the content, the journalists, they take precedence over everything, so they come, they have control every pixel and whatever they want, it's their side, it's their say, so it's their job on the line, most of all. So after that, there are the sales guys, and the marketing guys, and then come the devs, and then come the operation guys, because this is what the hierarchy is, it might be a bit different than a typical IT shop or a software development shop. But we have very much to do, like we manage all of the stuff which TV doesn't go into, we manage internet, we take care of mobile, web, value added services, mobile apps, even into internet video, that's all taken care of by us. So, I'm an assistant admin, so, no devops, I'm not a devops, whatever, I'm an assistant admin, anybody else who is competent enough to call themselves assistant admin here? What are the managers, programmers, so anybody who is into like a devops professional like me? So, I have no problems calling myself as an assistant admin, I am a co-assessment, and these guys might be my great-great-grandfathers, so I still have their genes in me, I'm still a co-assessment. But these assessments are very passionate for some things, we are passionate about our uptimes, we are passionate about our servers, we like, let's just show them what we are doing. But things have changed, like cloud has changed a little bit, cloud has got enough, like it's some devops sort of environment nowadays, so we would separate that developer to do this. We'll sit behind Ramsey and we'll just migrate a site to a fair of a location and attempts, but that guy would be armed, he won't be like going there barehanded, he will be armed with the latest metrics that he has, he will be armed with the whole infrastructure at his disposal, he will be armed with, he could be like, apply a new version, if there's a regression, he would be able to apply a earlier version and do all that stuff. So dev has all the tools and they are very confident and they are like, they can handle the bad guys. And we have some of them, they might call us, they might say that they have, it's still a fight, it is, it will be fun. Since I am just sort of, mainly you get, okay, but what the fight has been and will be, will be like this, something like this, we are, we still have, there's somebody who is responsible for something. Now it's a collective thing, we call it DevOps, dev people are also very much responsible, they also take ownership of the code that they have deployed to the servers and if a deployment goes through and within five minutes there are a lot of errors coming in. So devs are the ones who know okay, so whatever we push right now has created something, so they take responsibility and all that stuff. But still, there is a sort of wall of infusion and all that. Devs, people push stuff over the wall and we say, what the hell is this? And we push it back, but this will go on, this happens sometimes. We have to work through it and we have our own egos and all that stuff here. Somebody has to take care of something, but all of it then comes out to, this has been a lot of loot all the time, I don't know what the mind is like, I have to get food, man, it won't work. So how many of developers have root access to your production servers? Any developers here who have root access to their production servers? Do you want root access to your production servers? No, I don't. Yeah, one guy. Good. Now it's all nubby-dubby. So DevOps is a much larger picture, it's not only developers and operations guys meeting together, it's also marketing guys having the latest, digital tools available to them, have the latest knowledge about what the site is performing like, the number of movies they are getting, the number of view doggies they are getting, all that stuff is being like transmitted to them. So they are also involved, it's not like it has to be developers and operations also, it is the whole big company as well. So if all comes down to show me the money, so if all boils down to that, because in the end that is what we all are involved. Work together, get the things sorted out and get the things rolling up. The Phoenix project, how many of you read this book? And why reading this book? What are you able to associate people and characters in the book with people in your organization? And have you pushed the book to your boss like, man you should please read this book and after reading the book, pass it on to your boss. Puppet, anybody like this? Have you heard about puppets? Using puppets? Yeah, using puppets. Any other conflict manager, conflict department or management tool, chef, ansible, using it in production? So puppet is a conflict management tool or it really gets something and it really makes the whole jink bag thing work together. So I won't go into detail about puppets that much. Just Hyra. Hyra is something which couples very lightly and very easily with puppets. It can also sort of do the same thing with chef. It is not like it is hard coded into puppets, it's a independent program on its own and it suffers very well. So what exactly it is? It is a hierarchical based key value lookup tool. So what do you mean by hierarchy? A hierarchy would be your infrastructure, your servers who have been constructed into a specific role. Then your projects which have to be deployed on those servers. Your data centers A, data centers B have different set of networks. So that is how you are layering your infrastructure. So this helps, this tool helps you in defining those hierarchy for your arsenal. Why we use it? Because it really makes puppet better. So the data which usually we use to be putting in puppet manifest or puppet code that we call it puppet manifest can now be put into Hyra. And puppet will just have to run, it fetches the data from Hyra and does its stuff. So you have the multiple levels of overrides, you have a level of a particular server, you want it to be a snowflake. In the sense that it is unique in its own sense, you want something to be different on that server. You can do it with Hyra, so I will show you some examples of how we are doing it. So puppet has a puppet forge where the community would ship modules. Modules are those blocks which manage a particular stuff like a module for Apache, a module for IngenX, a module for TomCamp. Maybe there is a module for WordPress where you can call either IngenX or Apache or whatever you do. Those are modules to do specific stuff. Even you can also publish your modules easily because you are not sending your specific data with your modules. So those are totally different, different sets of your code is and puppet and your data is in Hyra. I am going to show you an example of how Hyra looks like. So we are using AWS, so if you know about AWS, these two are based on availability zone which would map to your regions. So are you familiar with AWS terminology? In stance type would be your different hardware models, actual hardware, hardware with 2GB RAM, hardware with 4GB RAM. So those would be different levels. So this is the Hyra of YML file. We start with the back ends. Hyra can have different back ends. So it is a simple YML file back and it can also be JSON back. You can have a couple of back ends together and it will look into it as from the top to top. So first we will look at it as a node, let's say directly, in which client-cert is a fact which is passed by a puppet. So client-cert will be unique back from puppet of a particular instance that puppet is managing. You can match to that. If it doesn't match to that, so there is cluster where we just put all our clusters together. Group of servers which are doing the same job are the neighbor cluster. So the name of the cluster, then if you look into availability zone or data center, this cluster's value for this data center is something. Or this cluster, if we are like scaling up or scaling down, we like scaling our servers vertically or horizontally. You have server servers where we start it up with a small server, a bunch of servers. And then we scale it up and we are using a bigger server now. For bigger servers you might need to have some processes or threads running much more than they would want to be running in a smaller server. So here we will be doing that definition. Then actual cluster and cluster. Common stuff, particular cluster. Then we come to our projects. Like projects are like we are going to deploy like a project for our project site, a project for our prime site, a project for our mobile site. Those are being left together as project. So those projects have their own deploy environment. So this project for a scaling has to have less than this. This project for production environment has less than this. So then there is a common project which are common for the particular project. So then a global file which is there. And then it is defining the data disk here, fetch this, all this file. So it is a flag data character. So this is what my directory looks like. There is a directory in which there is a file called hijack.1.data. So that.data.edata is my client server. Then there is a cluster. I have like two clusters or three clusters here. There is a hijack.gaml. There is a hijack which also have two different files for the instance type that I have. And then there is prime. All prime is like this. We have different values for some data centers. And then there is profit data environment which actually needs to be that. So that's why all this data is coming to one point. So it will not be sufficient. Then there is a project. So we have a staging.gaml file which continues this staging setup for all the projects. We have the testing.gaml file. Then we have the production.gaml file. Then there is the command.gaml file. Then there is the program. Questions like one particular snowflake server that we want something to be false. So this is the key. And that's the value. So the key is from our proper module. So that base module in which there is a monitor class in which there is a monitor variable. So that's a key. Which is in the assignment of false value for that particular search. So it's matching. So it will only match if the public central server of HACHAB1.com. So this is for a C3 large. So here I am defining a stack. Stack is the hash which has FF1 codes. So basically we have PHP. So we are defining different things for our servers. So stack is a unique pool. It has its own sort of like you would recognize these variables. Like what were you. So for a C3 large I am having a lot of like max children would be higher for this. But this server can take that much of load and can take so many threads out. So these are what I am defining here. So this is a reject.yaml file for the first person. So first if we have a instance of C3 it will fetch the stack values from there. But the rest of them will fetch the stack values from there. So that the stack they will get is the stack and there will be another pool of defined as duc. Which has a lot of the same values. So on the top in the next we have a worker processor. So in the next module worker process and passing processor count. So the number of processors in the instance in the next we have the 3-2-3-4-4 threads. In the next we have the part file. So that is where my server name is. I am calling server. Those are my server name and I am calling it like that. So in the next module we have a lot of reject.com. Passage type has to be an efficient socket. So we can have an efficient socket or a unique socket or a PCP port or a balancer also. To have a balancer and to be expected. So we have a unique reject value. My server name is same. So that is it. Value which will be passed on a configuration file which is being used by the boss. It needs to go and get something from the slave. And then again the stack has to be. Everything. But for us it is having a different... For that data center I have a different value for a slave. Any instance which thrown up in that data center will get that slave. Rest of them. Or I can have for each data center I have kind of a different DMF file which we have assigned. That. The normal file which has. So all these values would be much more in the domain of an operations guy. So he would be much more concerned about how I am going to be deploying my infrastructure. If this particular application needs a bigger server. If on a bigger server. How much resources does this application have. And all that stuff. So it is much more... Now it is all being done for an operations guy. Like concerned about who. Then a DMF. Next is like... So this is our stack app. The next app that would have like done. So if you don't have it. Go download it and install it and do your... So you are forced to... Do all this stuff. So... So when we took care of it. It was an app. So there was an app team working on it. They've been working very diligently. They've been working through a lot of them. And there was supposed to be a jack dot plumb place over side. Just telling what it was. Simple static sign. This was a contribution. In a production Ml file. So when you have to put this up. It was the product. Jack server now. Jack. Server is true. There's a line missing. And a stack. So I will also think. What stack is this. I was defining it as internet. Static sign. It's an internal stack. We have an internet static stack. Which we want to deploy a patch on that server. We have internet. We have the fpm stack. Which we have internet, fpm, fpm. Install on server. And then configure using. And then again. fpm. fpm. Then fetch the data from there. So that was. Specific to the project. Production environment. There was something wrong. Jacks. Whatever the source was. There was something slash. Like it was to the slash. Anything. As we didn't project or slash. By default it was configured. The index file. The index file. Simple. Static. The next static. That's the style. The people who are allowed to deploy this. So there's a group. Dev. This is what I have done. And that's what we've done. It's going to be accepted by. iOS. And then there's a lot of marketing. There's a lot of stuff happening. There's going to be a lot of. In the channel they're going to talk about this. So the people are going to be away soon. There's going to be a lot of online marketing done. At the channel level. So that's it. So this is what we do. I'm home. From 19. And having my data. And they'll decide no. We don't want to call it. Check. Any. We're going to call it. Production. Change. Check. And the DNS changes. Whatever. And on the same servers. And it's. So. Everything not. Simple. So. The app has something of a feature where you can. You are able to share photos. You're sharing it through. The social media. Or you can share it with the app itself. Because when you share it with the app, the app sends you a URL. Which you can forward to your friends. So somebody noted that the URL is very. Or produce. Something as 3.0. AWS. Or something like that. And isn't. We should have a proper. Check. The app. Sharing the files on our service. We are just getting the files on the. And we're pushing it to the. And we're past. We're maintaining the. Which you are finding a. The user. In our database. And we're. And we give that link. It has to be. Now. It goes to the website. Say. What side of the. Can't you do. It was supposed to be a simple static site. Like. Simple HTML. Some JS files. And that's it. Nothing. So. Those guys are working on it. Now they. Now they change the stack. From an internet static site. Now this is what developers are. They have this facility. They change the stack. To an HPF team. They. Change the. Representing. Was. Something else. They. They made a branch. They started in the. Hp stuff. There. They have. Up to. When they started. We need a PC. We need to. Later on. Later on. They made Jason. Because. They're getting the data. And they. Nothing. They can actually. App guys are using a. Using a database. So. We need to have the. Hp that is. Stuff installed. So. So. These are the. Which. Whatever the requirements are. They can just fill this up. And this is what. Start. Apps gone live. Everything is happening. Now. Somebody opens a. Site. On. A tap. The game goes back to the devs. Hey. The site's not looking good on a. And the site. Is being used by. The. App. I mean. The users are going to be on tablets. These are going to be a mobile. Fix that. On the same. Code. What. These guys do. They'll do. They'll make a new. Directly mobile. And they. Make. One more. At a more server. It's not an ally. It's a different server. Configuration. And. But. Document. Mobile. Server. That's some sort of rules. Which we like. What. The next. Next file is going to be. What. The. What. Find it. They can use that or. For example, So. This. Some supper. Lost. Like. This was. The class. Jack dot p. This is a public. Last. This is a perfect. Last entity. Slide. All the keys will. entity. Boulain. Had. The black environment we have match the production or testing. This was getting at top level. when this class was going to be added. As seen using it, if something which is being saved in a Hira configuration, that was just because there's been a problem. Document rules, fetch modes, as seen there are users, project name, track, server name. So HiraHash is a function in puppet. It goes through all of your Hira ML files and will merge all the hash and give a one big hash which will have all the value that you want. So all that is now being passed. All these values are coming from Hira in this class. And then all these values are then passed on. This is a big defined project name. So defining something which has to be in the class can be called multiple times. If you just do this class, projects are like a single server can have multiple projects deployed on it. Project would be a project project name which is defined. So it can have multiple different initiation of those defines. So all this data is given to this and this is the main bunch which is doing all that stuff. Making, initiating the project, it gets to know the stack. On the stack it knows it has to deploy the next or a budget on PHP, or CTI, or FPM or all this stuff. The environment it can pass on. The server, there's a server name hash also. It can pass on the number of like that. So the server name, there are aliases and it has to make rules in the configuration. The document rule, if this is a default document rule, it is. You can also have a different document rule like in the jac.com. The document rule is different. So if it is over that, that. So this is what jac last. This is an example of what I had to do. And there was this requirement by everybody that we need to change this. So then the devs were managing it on their own. The same server which was like configured to have a static in the next site. But now we have PHP installed and it's configured the way they want it to be. And they were doing all that stuff. So this is where they had this power to do it. So this is just an example. So developer doesn't need to run a lot of stuff here. So developer doesn't just have to understand the values that they have to pass on. The values that available stacks that we have. There's an array PHP mod where the developer has just put in the PHP modules that they want to install. PHP live is there that they want to install. So developers have to go through that. So ours is like the development team would be like 20 to 25 people and there's a lot of people. Instagram also started. It's not an informal site like the other one. It's per second. It's 10,000 to 10,000. The servers are then scaling up with that. So the developers also are very much involved. They'll have access to servers. They'll get it out if something happens. So they'll get another best alerts on their own. They're on the top 24 hours. So both are configuration management tools. Both have their pros and cons. So Chef was phoned from puppet and it has its own life now. And it's very independent, very robust software and a community and a company and everything. And both puppet and Chef are growing. Seriously, at the time when we were looking into it, both were like together. So why I went to it? I saw in my peers on ISE, the guys, I was following. What were they using? At the time, some were into Chef and some were into puppet. And they were ahead. So the flame was there. So I was like, some people who went through, I've been reading their blogs and know how their thing came. And they've written why they've chosen puppet or why they've chosen Chef. And those are tools I've been with puppet. I didn't do it like I installed puppet and I'm going to see if it's going to be like, it's going to do my thing. Check out Chef. No, I just went into my talk about it and then I just did puppet. And puppet is going to be downloaded, start reading about it. Because we had to do it. I mean, I had to use something. It was going out of hand. SSH, for two days, not the way to go. It takes like 10 to 20 hours. But after that, you're going to lose your sanity. It's not going to escape. So that's how I chose it.