 Chuck Norris wedi bod yn wneud hynny hefyd yn ddiolch. Rwyf ni'n gwych yn cael ei wneud am fwy teimlo i weld gweithiau ynchân Ymwêr. Rwyf wedi bod yn gweithio. Rwy'n gwych yn yr hyn maen nhw'n lleol yn cael ei fod gennych. Diolch yn ymddangos i'r dyfyrdd am y cyfaint. Rwyf wedi'i gweithio i'r union yn cael eu cyfaint yn cael eu phiwyr. Rwyf wedi bod yn cydweithio i fod i'r fwy. Felly, mae'n gweithio ar Ymwêr. We're going to look at some ways of attacking VMware networks and I'll also show you the tools that I've developed to help us do this. So afterwords hopefully you'll all be leaked VMware hackers able to own an entire network just by popping the one box. My name's John Fitzpatrick, I work as a pen tester for MWR info security in the UK. For the last six months or so I've spent all the spare hours battling against VMware's APIs. In the process I've found some pretty interesting stuff. It's my intention today to share this with you. We'll demonstrate some of the potential attack vectors. Hopefully you'll learn something interesting. The slides have changed slightly from what's on the DVD but not massively. The latest slides will be available to pull down from our website and I'll give you the URLs at the end. As was mentioned I'll be in the QA room afterwards if anyone's got any questions. So why VMware? Well virtualisation's taken off. Virtualisation's taken off and it's rare that I'll go on a pen test now and not see some form of virtualisation. Normally it's VMware. In fact this research was all kicked off when I was on a pen test. I gained access to VMware server console but the desktop was locked and it meant if I wanted to get anywhere with that I'd have had to boot the guy off that was on the box. Which probably wasn't a good idea. In this case it wasn't such a problem. We could gain access to the box in other ways but I thought well if that comes up again in the future I want to be able to get on this box and I want to have a play. So I guess probably a lot of you have played about with VMware before. Some of you maybe have got virtualised networks back in your office or wherever you work. The structure of the presentation. We're going to start by looking at VMware, the different flavours and some of the key concepts. We'll move on to VMware server. We'll look at some ways of attacking that. I've got about six demos in total to show you so hopefully they'll all run fine. Some of these demos are for ESX and we'll look at ESX, some of the different attack vectors for that. We'll move on to a tool which was developed by a colleague of mine called Drardis. Which is a good tool for information sharing, sort of structuring your information whilst you're on a pen test. Then of course we'll shift on to some recommendations afterwards and see how you can help to take yourself if you're running VMware. We'll kick off with an introduction to VMware. There's loads of different flavours of VMware but here's a few listed here. You start off with the basic player which is there simply to play virtual machines. Workstations, perhaps a step up from that. You've got far more functionality there. You can create snapshots, that kind of thing. VMware server or kind of what used to be called GSX. ESX are the two main ones. They're the ones I'm interested in anyway because they're the ones that expose a service on the network. Purely because of that they provide something that we can attack. We're going to look at server and ESX today. The key concept with virtualisation, you have one server which can run multiple guest operating systems. Essentially one piece of hard work can be many many boxes on your network. In fact with VMware's virtual infrastructure this kind of comes out more in a many many relationship. Perhaps you'll run multiple ESX boxes again with multiple operating systems on it. VMware server looks something like this. You've got your hardware. On top of that you'll install your operating system. Whether this be Windows, Linux, whatever. The operating system is the light blue layer at the bottom. On top of that you'll install VMware server. On top of that you can create your virtual machines which all have their own operating system. Whatever applications you're running on them. ESX is slightly different. ESX essentially is the operating system. By running ESX you strip out one layer of the complexity there. Your virtual machines therefore can talk to ESX which can talk directly to the hardware. If you want to give two gig of RAM to a virtual machine it's got no operating system to contend with for that RAM. It can talk directly to ESX which will give it the RAM it wants. Now virtual machines because they're virtual they're made up of several different files. Here's a few of them. The VMX files, the main configuration file. This is what's actually executed by VMware and it contains things like the virtual machine name, settings, MAC address, hard drive settings, that kind of thing. VMDK is the disk file. It's essentially the virtual machine's hard drive and it stores all your data that's stored on your virtual machine. VMSN, because of the way virtual machines work you can create snapshots of them quite easily. If you want to go screwing about with something you just create a snapshot of your machine. If it ends up messing up your machine you can just revert back with no problems at all. That information is stored in the VMSN file. Then you've got the VMM file which is your virtual machine's memory. Now you're not going to see this unless your virtual machine is running or maybe it's crashed. You'll probably see it for some of your snapshots there. Now because virtual machines have a disk file we can look into reading data from that. The first demo I'll show you will read some data from a disk file and show how important it is that you ensure that this is secured and protected on your network. The tool we're going to use for this is a tool provided by VMware. It's called VMware Mount. Can you read the text on that on the back? Is that okay? Yep, yep, cool. Here first of all we're going to take a look at the position table on the virtual machine. It's a Windows XP machine. You can see that. Pretty much from the name of the disk file we're specifying. If we run that we can see that there's just one partition. It's NTFS which is kind of what we'd expect if it's a Windows machine. Next step is just to mount that. We do that again with the same command. We specify the partition we want to mount and where we want to mount it to. In this case we're mounting it to slash mount slash VMware. Just say yes to them. There you go, we've mounted the drive. If we just change to a new window there. There you go, we can see the contents of our Windows hard drive. From that we can have a quick play. In this case we'll jump into the system32 config file directory. Grab the syskey and then pull out the password hashes. From there it's nice and easy just to run a tool like John Ripper. We can use that just to crack all the passwords. Which is done quite nicely there. In this case we went for the password file. For the passwords for the box we could have gone for anything, any sensitive files. One thing that's worth noting is because you've got the ability to create snapshots. As soon as you build your virtual machine all of your files on that are there. Any changes you make are then made on top of that. If you say that you can revert to your original virtual machine your original files are still there. Anything you delete on your disk file it's still there even if it's not present on the virtual machine. We've had a look at some of the key concepts of VMware. Next we'll shift on and have a look at VMware server. VMware server looks something like this. In fact it looks exactly like that, that's a screenshot. VMware servers manage through the console which is what you can see there. On the left we've got the inventory which contains a list of all the virtual machines. On the right we've got the window there with the desktop on there. That's all VNC. If you're administering your VM from the console be aware that anyone else that's logged on to that server can also watch what you're doing. Which is probably part of the reason that VMware recommend that you don't manage your virtual machines from the console. Also I guess there's some performance issues there if you do. VMware server will authenticate against the local accounts in the box. Be careful if you've got a guest account enabled because someone could just jump on your box and browse through it. Whether you can run a virtual machine is kind of down to your file system permissions. So make sure you lock that down as well. Now if we want to attack VMware server the first thing we need to do is spot it on the network. Now this can be spotted on port 902. That's where VMware AuthD runs which is the authentication daemon. If you netcat onto that you can see in the banner that it's VMware. The rest of the ports I mean they could be anything. It depends where else is running on the box. Now communication with VMware server looks something like that. This is just us logging into the box. By default it does use SSL but if like me you don't quite have Brush and ISV and can't decrypt SSL in real time then it's quite useful just to flick that off and you can do that from the console. Now VMware provide several tools for managing their systems. For VMware server they provide a tool called VMware command which comes with VMware server. It's a nice tool, it's got some useful functionality. It's kind of a pain to use against a large number of systems or if you want to perform lots of tasks because you've got to pass the configuration file to it every time. To get around this problem what I've created is a Ruby module which essentially allows you to access this functionality. It works as basically a wrapper around VMware command. So if you're on a pentest and need to perform lots of actions or if you're just administering your servers and need to perform lots of actions that's a useful tool to have. Something else provided for VMware server is the VIX API. I don't know if any of you have used it. It's quite a powerful API. You can do stuff like transfer files back and forth from the host operating system to the guest operating systems. You can even run programs in the guest operating system if you like. Now it's got Perlin C bindings. In order to use it you've got a fair amount of code you need to write. But a lot of the operations that are there they're not available from the console directly which is probably the reason that a lot of people are spoken to you don't even realise that that functionality exists. One thing that's interesting to note is if you're copying files back and forth using the VIX API it doesn't use the network so there's no traffic for someone to detect. It talks directly with the virtual machine. Now I've written some Ruby bindings for this because when I'm testing I don't want to sit and write 130 lines of C code. So here you can see how easy it is to script up with my Ruby code. The first line simply includes the library I've written. The second line there will execute a command within the guest operating system. In this case it just adds a user, a VM user. I don't know if many of you have played with Ruby and C but they talk quite nicely and essentially you can write all your Ruby code in C if you want. Which is kind of useful. Especially if you want to play with something low level like memory or something like that. We'll move on to a few demos. These use some of the tools that I've developed some of the techniques that I've played about with in order to gain access to a VMware server and the virtual machines. I should probably explain the setup I've got. I've got a network with my laptop connected and it's from there that we're going to perform the attacks. I've got VMware server running and on that is a Windows 2003 box. Now we can assume that that 2003 box is completely firewalled off from the rest of the network. In fact maybe it's even sitting on a different network so gaining access we could use that as a proxy onto this new network. The first thing we want to do if we want to get onto the box is to obtain some credentials. The VMware command tool I mentioned earlier because that allows you to remotely administer and perform actions against servers one thing you've got to do is log in. By simply wrapping around that script we can perform a brute force because that's got to tell us if our credentials were good or bad. Here we've just brute-forced the user account JJ on the box 172.16.93.1 with our dictionary and you can see at the bottom we've succeeded with the password Defcon16. Now we've got some credentials we can take a bit more of the look at the box and see what else is there. Now VMware command can pull out a fair amount of info about a box and using the Ruby module I've written which essentially wraps around VMware commands we can pull out a whole load of information. At the top there you can see we've got a whole list of the virtual machines on the box. We can grab things like the power state and what users are logged on which is kind of useful if we can see who's logged on the box we can see more accounts that we can attack. At the bottom there we're reading the config file of this particular virtual machine it would be nice and easy to look around and read the config file of every machine. Now this works by pulling one parameter at a time from the config file and we've pulled out some useful stuff like in the MAC address. Now why is that useful? That's useful because if you're attacking a guest operating system it might well be on the network you're on and if you can find the MAC address you can tie that up to an IP address and then you can attack it from its IP address. Now I also mentioned that I've written some modules for the VIX API some Ruby modules which we can run nice and easily. I'll show you some of the potential of that now. Now what I've done is I've created a Metasploit payload which is a .exe file which we'll place onto the virtual machine and because we have the ability to copy files back and forth that's nice and easy to do. After that we'll execute it and we've got Metasploit sitting there listening waiting for this connection to come back. So we'll kick that off now. So first of all it's got to transfer the file across in this case it's called defconel.exe and you can see that's succeeded and it's executed the program. So now back in our Metasploit window we've got a session back. Now if we interact with that session we can do all sorts of things with the box. First of all let's take a look just to make sure everything's worked and now using Metasploit we've pretty much got full access to that box we've grabbed all the password hashes from the box as well we could have done pretty much anything we liked we can transfer files back and forth from it we can run whatever programs we want. So there's a fair amount of power if you're able to use the VIX API when you're testing. Well that was VMware Server we'll take a look at ESX. ESX architecture is slightly different and there's some different attack paths. Now ESX can be administered in many ways one of them is through a web interface which is what you can see there and again on the left we've got the inventory on the right we've got all the settings on the virtual machines and we can open up a console from there as well. Alternatively we can manage it using the virtual infrastructure client now this is a .NET application written by VMware and it talks SOAP to a web service which is running on the virtual machine and if you can I don't know if you can see but on the right there's also a virtual network ESX has the ability to create virtual networks as well as virtual machines so that's another thing that we can play about with an attack on an ESX box. Again if we want to attack ESX we're going to need to find it first because ESX doesn't sit on top of another operating system all of the ports we can see in the port scanner are relevant because they're ports on the ESX box. The most relevant ports they kind of depend on what you use it for but I'd say 4.4.3 is pretty important because most of the useful services run on that. One thing interesting about ESX is your ESX boxes are always assigned VMware MAC addresses no matter what manufacturer your network interfaces are it's always going to have a VMware MAC address. One thing I haven't mentioned is ESXI which is kind of similar to ESX it's a free version it was made free maybe a week or so ago it's pretty much I wouldn't say the same as but it's very similar to ESX it has fewer services running but again you'll be able to detect it on the network through looking at its MAC address and what services are running and map or fingerprint them quite nicely. So the services we have running slash SDK on the web server that's the soap interface which the VMware virtual infrastructure client talks to and some of the tools I'll show you in a bit slash UI is where the web interface is so you can again have a play with that slash mob is really useful if you're doing any development with the virtual infrastructure API it shows all of the objects on the ESX server and you can connect to and talk to them, query them pull out information, perform tasks so if you're doing any development I'm pretty sure you'll play about with that. VMware auth D still runs on port 902 but there's no VMware server D so you can't play about with it as much as you can on VMware server and there's also a console operating system which is running Red Hat and that's what you'll connect to if you connect through SSH interestingly root access is disabled by default for SSH but if you want to perform a lot of the admin on ESX chances are you're going to want to log in so I imagine most people would end up enabling root login for SSH right the virtual infrastructure API I mean it's got all the functionality you're ever going to need to play about with ESX now I've written some tools that talk to this API which I think are pretty useful when you're pen testing if you're going to use the tools VMware provide the majority of it's you're using a GUI which is a real pain if you want to perform repetitive tasks particularly if they're against the number of different servers so it's nice to be able to talk to that directly from a script or whatever things like checking security settings across all of your ESX boxes is something you'd probably want to script up at some point so yeah I've written some Ruby code which would talk to this web service and I think it would be fair to say it's not the nicest web service I've ever played with but it's pretty powerful so I'll show you a quick demo of what I've done now it wasn't very practical to ship my ESX server over here so I've had to record it but hopefully it will be clear I'll show you the output and talk through it afterwards as well the first thing we're going to do is connect to the box we can grab things like the version without authenticating but once we've authenticated we'll get back a session and we can perform loads of tasks, list the virtual machines find out a list of all the users even identify non-standard users or users with shell access we can then grab all the network settings we could play about with the network if we want firewall rule sets one thing that you can do with this that I don't think you can do with the tool provided by VMware is grab a list of all the patches that you might want to apply to that host so if you want to scan all your ESX boxes and find out which ones need patches this is quite useful I'll show you the output from that and we'll talk through that a little bit more because I don't imagine it was massively clear there I won't go through everything because a lot of it is self-explanatory so we've got a list of the virtual machines we can list all the sessions on the box in this case there's only one so it looks like we're the only person connecting because the users are users of the red hat costs they're pretty much your standard Linux users but we can find the non-standard ones in this case MWRITES but these are the users that are perhaps most interesting because they have shell access we can then grab all the network settings stuff like DNS servers and we can see here that the box has two physical network interfaces now here we can see that there's a couple of virtual switches set up we can see several different networks so it kind of indicates that they've got some quite good network segregation going on here we've even got separate VLANs on some of the switches now because your networks are completely virtual and your machines are completely virtual your ESX server knows which box is where so we can make use of this to ensure security the MAC addresses of your virtual machines aren't going to be changing regularly your IP addresses again aren't going to be changing regularly and your virtual switches know exactly where your virtual machines are and what they should be doing so you can disable forged transmissions MAC address changes actually if you enable these properties then any traffic that's forged will be dropped by the switch which adds another layer of security to your network additionally you probably don't want promiscuous mode enabled on your switches there's no need for your virtual machines to see traffic destined for the other virtual machines so we can flick that off now you can see in this instance that these settings aren't particularly secure for most of the interfaces that's just the default with the ESX we can grab the rule set for the firewall take a look, see what rules are allowed in and out by default there's a default deny inbound and outbound which is quite nice now if any of you use VMware server quite possibly use it as a secondary operating system maybe you run Linux as your main operating system or you want access to windows in which case it's quite nice and useful to be able to copy and paste back and forth between your VM and your main desktop when you run in a server probably you don't want the ability for people to copy and paste between the host and the guest operating systems and you can disable this functionality the last thing you want is to go copy in your root password and someone on one of your guest machines to paste it onto their desktop so you can set some settings that allow you to do that by default these aren't set in ESX so it's something you need to go out and do yourself and I mentioned the ability to check the patch levels here we can see a fair amount of info on the patches whether it requires your box to be rebooted afterwards what patches supersede it whether it needs you to turn your VMs off afterwards that kind of thing so there's a lot of information you can extract here and this kind of script is probably quite useful if you want to just check the overall security level of all your ESX boxes and whether you've got potential vulnerabilities there right now we're going to move on to a tool called DRADIS quite possibly a lot of you are sitting there thinking well there's quite a lot of scripts there that's a lot to play with it would be nice if they were put in a more tidy form now DRADIS allows us to do that we can pull them all together in DRADIS and make them nice and useful to us now DRADIS is a tool written by a colleague of mine it's free to download, it's on Sourceforge it's intended for information sharing and structuring of information during a pen test but it allows various different modules to be plugged into it and in this case I've plugged in my VMware modules so we can use all of their functionality now it works on a client server, architecture you run the server and anyone can connect to it add information, read the information so multiple pen testers can use it if they're on a pen test it's Ruby based and it's pretty extensible like I said I can add all my VMware modules to it nice and easily we can use it to put together a methodology for testing VMware networks if we like now it looks something like this on the left we've got a list of different objects or concepts on the right we can see the properties of each of these these concepts so on there I know it's probably a little bit small but we can see the settings for the firewall and for those of you that like the command line at the bottom there's a command line so we can run programs and operations from that as well there's also a notes tab so we can add notes specific to a host or to a test that we want to share with other testers as well now how does dryness work? like I said we've got concepts in this case we have the concepts of a VMware server of an ESX server even of a virtual switch or a network interface and we have various actions that we might want to perform such as creating a user rebooting a box checking the patch level that kind of thing and we build all of them into what's described as a universe so our universe describes all these different concepts and actions in this case we've got a VMware universe but it could be anything you could have a universe to describe your network in which case you might have web servers mail servers and you might have different actions such as checking the version or anything like that and you just plug it in as a module so let's take a quick look at a demo again because it wasn't possible to ship my ESX box over here we've had to video it for you now the first thing we want to do is boot the server up it's running web brick it's a Ruby server and then fire up dry disk like I said because it's client server architecture there could be multiple clients running and talking to the same server it looks like this we don't really need the command line at the bottom so we'll just move that down for now to give more space our concepts are on the left so in this case we'll create the concept of an ESX server now the properties of an ESX server in this case is just the version and IP address so we'll give it the IP address of our ESX server there's no reason you couldn't extend one of the modules to detect ESX servers on your network as well now we have properties for these concepts as well in this case we've got a version property so performing that action we can grab the version of the ESX server in order to log on to an ESX box we need some credentials and surprisingly there's a username and password so we'll enter them in so we've got a question if we didn't have credentials what if we didn't have them well let me just pause the demo for a sec if we don't have credentials then we're more limited in the actions we can perform there's a small amount of information we can grab from an ESX box most of the information you're going to need credentials assuming there's no vulnerability that lets you grab information from it first so for the purpose of this we're working with credentials but credentials can come at several different user levels so it might be worth running checks with different user levels to see whether your permissions are set up properly that kind of thing in this case yes it would because port 902 is open and running so the question was the tool I showed you for brute forcing authentication work and the answer is yes it would against this box because it's got port 902 open alternatively we could have brute forced the web service as well because it has a login function and we could attack it through that also we could brute force SSH so there's a variety of different options for trying to obtain credentials and then there's also the possibility that they share credentials across boxes so if we look at the case where we mounted the hard drive maybe some of the credentials we found then will be valid here as well ok so just jumping back here we've logged in and we've got back a session ID next we're going to take a quick look at the firewall see if there's any dodgy rules in there that might let us jump onto the box through other means so we'll populate the firewall ruleset settings and I mean we've seen these before from the script but it's a pretty default firewall ruleset and we can take a look at some of the authentication settings here we'll grab a list of sessions that are active on the server and we'll grab some of the non-default users as well because their accounts are normally quite interesting to attack and here we can see there's several sessions to the box so probably we're not the only person using it at this time and the next thing we're going to take a look at is the patch in and we'll find out what patches need applying on this box now in order to check the patch level you need a repository which contains the patches in this case our repository is on a web server and the web server structure it contains all the patches and the downloads it takes a moment or two to run because it's got to scan the web server for that information it will return at some point okay it's come back and we've got some details of the patch patches which we need applying which we can see there and again we've got information such as whether we need to reboot the box and that kind of thing and some recommendations you've seen some potential attack vectors if you're running VMware on your network you know what should you do well the first thing to note is nothing I've shown you today gives you access to the box without some weakness being exploited but perhaps there's vulnerabilities in VMware that we could exploit as well to get on there so it's important that we ensure the box is secure in the first place so how do we do that well VMware has written some really good documentation on securing well in particular ESX it's not a bad read so it's definitely worth having to look through if you've got ESX or server deployed you need to ensure that your updates are there I know it's it's one of those things that's always said but there's been a few advisories released I Defence and Core have put out some stuff recently and also because the cause is running Red Hat all of the Red Hat vulnerabilities also need patching as well so it's important to ensure that you keep on top of this and also Defence in depth steps you've taken to protect yourself VMware is always going to be a nice target for attackers and because it's probably running several of your servers it's important that this if there's one place on your network that you you go out of your way to protect then it really should be your VMware boxes and also don't forget things like VMware tools which are running on your guest machines they do leave the potential for attack like I said VMware is always going to be a juicy target for an attacker one of the key things that VMware bang on about and quite rightly so is to keep your management network separate from your corporate LAN the attacks we've seen today we're all focused against the management networks if they're not there then the attacks quite simply they just can't be performed and as I mentioned earlier disabled MAC address changes disable promiscuous mode ensure your boxes are secure as they possibly can be and your networks are secure as they can be it's unlikely that you're going to want to change MAC addresses regularly on your on your guest machines and also as I mentioned virtual machines if they're servers it's probably something you don't need if you've got a CD drive on your ASX server or your VMware server disable the guest machines from reading that last thing you want to do is put something sensitive in your CD drive which everyone connected to one of the guest machines can also see VMware's documentation states that only port 443 and port AC are needed for general operation of your server so if your deployment doesn't need access to any of the other ports then close them off in the firewall one thing worth noting is that guest operating systems do write their logs to the ASX server so it's important you make sure that they're rotated and tidied up pretty regularly the last thing you want is for your file system to fill up through a malicious user filling up your log files well I've still got plenty to play with I've found some pretty interesting stuff which I'll chat to VMware about sometime soon I publish it all when it's done but there's still a lot of technologies that VMware have produced that I've not played with I've not played with Vmotion for example it's something I'm definitely going to look at in the future well to conclude I mean the tools I've shown you they're all going to be made available on to the DEFCON CD because they weren't ready in time but they're available to download or will be very shortly from my website or www.mwrnfacesecurity.com Drardis is all I showed you you can download from sourceforge that's drardis.sourceforge.net so yeah go away have a play see what's useful to you if there's something that you think could be implemented drop me a mailer have a look into it there's no room to take questions but we've got a few moments if people want to fire anything at me we have a question the question was by default is there an account lockout on server on ESX not that I've come across I guess it depends on the settings you've got on your box one thing that I should have pointed out which I haven't is the auth daemon that runs on VMware server by default will run through inetd or xinetd if it runs through inetd then after about 250 will request to log in inetd will stop the service listening so you can cause a denial of service through doing that so make sure you run VMware auth daemon through inetd or if you're going to attack it will ensure it's running on xinetd before you do but back to the original question account lockout is down to whether you've said it I've not seen account lockout on either ESX or VMware server definitely not enabled by default now sorry by default server doesn't know that info is on the cd yeah right I'll move through to QA room if anyone else has got any more questions