 Okay. Okay, we'll get started. Steven Ellis talking about resource allocation using C Groups. Hi, a few of you will probably remember me from previous Linux cons. I recently started as the Red Hat solution architect in New Zealand. At the moment we're currently based out of Ingram Micro's office there. Red Hat's got quite a good presence at the moment in the New Zealand market. I'm here to talk to you about C Groups. Anyone in the room had a play with C Groups yet? Anyone know what C Groups are? Okay, alright. So what are C Groups? Why are they so useful and important in your infrastructure and your operational day to day use of Linux systems? So what's it all about? Well at the end of the day is your hardware starting to get faster and faster. We're starting to put more and more workloads on their equipment and in the traditional Unix space there's been a series of methods to manage those resources effectively in order to meet various SLAs and prevent resource contention. That's been slightly lacking in the Linux space so now C Groups has come along to fill that gap and it basically provides an internal feature to control that resource management and then the user space tools and libraries in order for you to set the rules around how those resources are going to be allocated either on a per process basis per user basis and there's a whole variety of resources that it can manage at present and this is an ongoing project so expect additional resources to be pulled into the project going forward. So here's a kind of typical use case in this example. We're talking about a virtualised environment so we do the resource management at the hypervisor and we say that virtual machine A is only allowed a maximum of 40% of network but 50% of the CPU and so forth and break things out but this could easily be applied at an application level so that you've got a say a mix of database service running on the same server and you want to control the resources allocated on a per database instance and there's a lot more to this. For any of you ever had the instance where you can't actually SSH into a box because it's that busy. Brilliant use case all of a sudden we can now prevent that by saying that no matter what's going on the SSH process has a higher resource allocation or has a higher priority over other system resources so the control group is just the framework for allocating and managing those resources within that we have a series of controllers first of which the memory controller so we can say that this resource is only allowed a certain amount of memory from the operating system then we've got a scheduler to attribute a portion of the CPU resources which is going to like a waiting system so you say this one's got 500 shares this has got a thousand shares and it will wait the load accordingly. Come on go forward the IO controller so here we want to control the amount of disk IO so that say your PostgreSQL server doesn't get swamped when you want to run a backup. We want to manage the network which I have to say I haven't played with myself it's actually interesting learning curve in order to do the network controller there's not that many great examples out there at the moment so let's jump forward so here we go let's break things out a little bit more so you've got your memory controller CPU set is a bit like task set so say these processors are allowed allocated are allowed to utilize particular CPUs on a multi-core system CPU account gives you a breakdown accounting level of the number of cycles used by a systems allocated to a particular group and then the device controller actually allows you to disallow access to particular devices so say that you want to spin up some aside from using tools like SE Linux to do control around what resources a process or family of processes of access to you can actually also do some smarts using C groups there's some other controllers in there things like the network controller this freezer and this keeps moving around and it has a AV system will turn off in 15 minutes I'm being told touch the screen cancel warning right and it has a hierarchical model which means that you allocate a when you start process up you can allocate it to a C group like it's a demon sequel and then every or demons HTTP that every child from it is basically picks up all the same resource pool and it's the family of resources the tree of resources that has the restriction applied so if you say that the resource pool that was running your Apache server has a upper threshold of a certain amount of RAM yeah then you know that the Apache instance and its children will not exceed that amount of RAM so let's look at some of those subsystems in detail with memory you can limit the memory and also pull statistics back from the C groups hierarchy about the amount of memory currently being used CPU usage what was CPU accounting here we can pull back the number of cycles being used so you may be charging a department or a client for the number of cycles that they're utilizing and bill accordingly CPU shares so here's a waiting model so we'll be in order to do this would say wait sequel with a thousand shares and HTTPD with 500 shares and therefore it gets when there's resource contention it will wait in the way of the sequel server but it won't deny HTTPD CPU resources we go jump so if you want to start using C groups it's on modern most modern Linux distributions have it now you can just go your install I've get install they'll usually come with an example CG config which is what you need in order to set up the virtual file system that C groups uses to manage its configuration on say Ubuntu that would be under slash dev slash C groups but under well six and for Doric tends to be slash C group then just simply start the demon and away you go so it was you can then basically do an LS and look into slash C group slash CPU set see what the default default rules are they also installs a bunch of command line tools so that you can use CG exec to start a process and allocated to a particular C group move CG classify effectively takes an existing PID and moves it into a given C group you can create C group rules either that directly onto the file system or you can use the CG create CG delete command line tools to manage those but you can actually just script this into the CG config script so that when you start up the the background demon it already has a series of rules defined so here we've defined a basic rule for Apache where that its memory allocation will never exceed a gigabyte in the case of Enterprise 6 and Fedora 14 the HTTP demon has a Sysconfig file as soon as you add that line in and start the service it's immediately C groups managed now I haven't looked on this on things like Debian Ubuntu so you'll have to go and dig that one up having a chance to have a play but in the case of RedHouse distributions it's very easy to manage if you're doing this with say virtual machines you may decide that you want to limit your virtual machines to never exceed three and a half gig one and a half one way of doing that is on a most Linux environments you're going to manage those via liver so you restrict liver then everything liver creates live within that three and a half gigabyte pool and you can say well I only want liver to access CPUs one through three and they'll be locked into their word of warning if you only have two CPUs CPU zero and it's CPU one don't say one through three because it won't start up actually checks to see how many CPUs you've got and again liver has a Sysconfig entries on a rel six or Fedora 14 system so you just create the appropriate entry need to see Sysconfig liver and you live that's fully managed then by C groups come on jump forward CG Red is another demon that allows you to then also attribute rules to users or groups of users so here you may decide that users on your system are assigned the staff group but then you can also go further and say that if that user starts up the FTP demon then they also pick up an additional C groups classification and it may be that the FTP you want to disallow access to a whole group of devices just as additional security policy or give it a different set of permissions or a different level of resource allocation and you can manage all that by that demon if you want to go and have a play these slides will be available afterwards the rel six has some great documentation as part of Red Hat's normal documentation which is just available by their website Fedora's got an overview on the Fedora project wiki Zonka's been doing a series on C groups on server watch he's done two articles today I think it's going to be an ongoing series of articles so that's one to watch for I highly recommend having a look at that and if you're using it on Debbie no Ubuntu have a look at the bottom link so any questions I suppose the obvious one is this it works you put your deliberate logic bombs in there and they just get constrained to their little X percent so you put the deliberate logic bombs you know the the one the 15 character or bomb or those sort of fork bomb type attacks those sort of ones they get constrained yet they should get restraint constrained I'll be interesting one to try out in terms of testing that it's behaving on your system one that tried out is actually using some of the lipvert stuff and making sure that given virtual machine actually doesn't get over a bit as an allocated amount of CPU use so you know cat there view random or pipe it through G zip in a virtual machine over there do something else over there and monitor the CPU allocation and it's in kernel it's been in kernel for a long time but it was called something else and it's been renamed C groups I'll tell you in a moment I've got up here process containers yeah okay I've just got a question how does it compare to the hard separation that you get from say VMware ESX or or KVM something like that is it strong enough that you can use it maybe with containers or something like that namespaces to give you something like that or is it how close is that I wouldn't rely on it's as it's only part of the solution if you look compare it with like a container based model or using virtualization it's the way of controlling resources it is in a way of controlling you seeing other resources so it's only a piece of the puzzle so say you're running multiple databases for different clients but on the same server you need to meet various SLAs it's one way of doing that but if they need access to the server where you they'll see that you're running other database servers on that box doesn't provide any security per se except say at the case of the device C groups that you want to black block off access to selected devices on the system and what we don't