 this is using Amazon's web services from Ruby. And, you know, get the question, does it scale? Well, when I was thinking about that and, you know, why you would use Amazon's web services at all is to really help solve scaling problems. You know, being able to, with their services, they handle basically the problem of being able to scale your applications. But with Ruby, you need to know the ability to how that works. So when I was thinking about that, I thought, you know, there's this Isle of Lucy episode that basically, to me, is a perfect example of needing to scale. And so I'm just going to play a couple of minute clip of the episode there. It's the Chocolate Factory episode. What are you doing up here? I thought you were downstairs boxing chocolates. Oh, they kicked me out of there fast. Why? I kept pinching them to see what kind they were. The apartment I'd been in. I didn't do so well either. All right, girls, now this is your last chance. If one piece of candy gets past you and into the packing room unwrapped, you're fired. All right, so clearly that's a question of does it scale. And in that case, it did not. But, you know, oftentimes we don't have the opportunity to just eat all the chocolate, right? So we've got to be able to figure out how to handle this. And so Amazon Web Services can help us do that. Yeah, thanks. That was it. But it would have been useful to know people around. But thank you. So the Amazon Web Services that I wanted to talk about in particular are the SQS Simple Q Service, the EC2 Elastic Compute Cloud, the S3 Simple Storage Service, and the Simple DB. The two that I'm going to really kind of be showing the Ruby code for are the SQS and the EC2, mostly because the API is similar on accessing any of those services with the particular Ruby code that I'm using. So I'll be showing the SQS and EC2. The RubyGems that I'll be using are the write AWS RubyGem from write scale. There's a lot of different Ruby libraries out there for communicating with Amazon Web Services, S3 libraries and SQS libraries and EC2 libraries. What I like about the write scale gems in particular that they, in the single gem, support all of the different services I just mentioned with the SQS, EC2, S3 and Simple DB. So in addition, it's a well maintained library. They update it often. Amazon updates their web services. This seems like quarterly. So the write scale guys seem to be on top of that and always release new updates for whatever new features are coming out. As well as they've got a robust HTTP library where Amazon occasionally will throw you error conditions and things like that that you can retry. And so their HTTP library will automatically retry that for you so that you don't have to have as much error handling in your code to handle those types of instances. The other library I want to mention is, it's called Kato. It's an EC2 pool manager and it's basically a port of the Lifeguard Java library that I wrote. I didn't write the Java library. I wrote the Kato port of that. And I'll be using that today. So using the write scale gems, we're going to just show the write AWS SQS. So you just require your gems and then the write scale gems can use because Amazon's web services communicate via XML. You have the ability to, with the write scale gems, specify that it uses the LibXML library. And that speeds up performance quite a bit rather than just using REXML which it'll default to if you don't have LibXML installed. So here we're just basically setting the parameters that are required to connect to SQS. So Amazon gives you an access ID and an access key. And really those are the only two parameters that you need to specify if you're going to be communicating directly with Amazon because it'll default to communicating with them. But optionally you can specify the server, the port and the protocol. And that's really handy for doing local development where you may not want to talk directly to Amazon or if you're doing a presentation like this where I don't really want to be communicating with Amazon right now because the internet connection may not be working. So SQS queuing. So to create a queue or to get hold of an existing queue you basically just call SQS.queue in whatever the name of the queue is. When you have a reference to that queue you basically just call push to put a message in the queue and again this is write scale specific other libraries may do that. In the most recent update to SQS they use the maximum message size from 256k to 8k. And so you don't really want to be using this for pushing large messages into the queue and pulling them out because it's not really designed for that. So typically what you'll do is you'll put in some sort of identifier that you can look up in the database or maybe a filename or something like that that you can look up on S3 and get your additional information that you could have put in the message. To get the number of messages in your queue you just call queue.size and it will give you basically an approximate number of messages that are in their queue. The reason why it's approximate is because of the way that Amazon has their cluster set up that when you add messages to the queue and you call back out of it it may not go onto the same servers in their cluster and so it's basically just an approximate size of what their best guess is to what the number of messages you have in there. To get a message out of the queue you just call queue.receive and if there are no messages in the queue it will just return nil. If there's a message in the queue obviously it will return it sometimes. Even if you put a message in the queue because again how the clustering on Amazon's services work you may not actually get that message right back out because where you put it may not be exactly where it's pulling from when you call it receive. So it's just something to keep in mind that it's not always guaranteed that you'll be receiving messages all the time even though you know you put one in there. So to access what the message is you just call body on it and that returns the body. To delete the message out of the queue you call delete. The message is by default when you call receive on them. There's a visibility timeout that you can also specify where the message will not be visible to other receive calls until that timeout expires. So for instance if you've got a process that calls receive on the message but then never deletes it after that timeout expires that message will now be visible again so that if you did call receive that message could come out again. So that's the queuing. Feel free to ask any questions at any time. So using the EC2 right skill gems is pretty much like using the SQS gem. As you specify your access ID, your access key and optionally the server port and protocol. Basically all the right skill gems to connect to the Amazon services connect exactly like this so whether you're doing S3, EC2, SQS or simple DB. So it makes it you know the API is kind of really nice that way. So communicating with the EC2 instances if you're not familiar with EC2 they're basically virtual servers where Amazon stores a copy of a machine instance that you can basically boot up as many of those copies as you want and boot up time is on average around two minutes though I've seen it go quite a bit faster recently. There are different size images that you can boot up as far as the kinds of hardware, virtual hardware if you will that gives you more memory or more processors and they charge accordingly. So to describe what kind of instances that you have booted up you just call it EC2 describe instances and it basically returns you a hash that is then other hashes of information regarding the uptime on the machine, what the IP addresses are, those sorts of things. To run an instance you basically call run instances whatever the name of the Amazon machine image is, what the minimum number of instances you want and the maximum number of instances and you may not actually get the number of instances that you request so it's a good thing to check to see I requested start up five instances it may have only started up three. To terminate an instance as part of that hash that you get with a describe instance or the run instance of also return the same type of information you basically get an identifier that you can use to then later terminate those instances and it basically just works to do that. So Kato the pool manager basically what I wanted to be able to do in scaling an application is basically dynamically determine should I start an instance up or should I close one down if they're sitting there idle and so I did a lot of searching and stuff like that and tried some code on my own but ultimately I found a Java library called lifeguard with some pretty good documentation on you know it basically solving that exact same problem and so I basically ported that over to Ruby and that's what Kato the pool manager does is it basically looks at your queues determines whether or not it should start up or stop instances and then it relies on the right scale gems for basically running those things that you just saw with the queuing and receiving messages and those sorts of things. So I'll talk about the config just the next one. So you can manage multiple queues, multiple pools of EC2 instances. So let's say you have two different machine images. You have a machine image for your website or you have a machine image for your database or some other kind of processing servers application servers or things like that. You can use Kato to manage those multiple pools independently. So let's say your application servers are just getting hammered. Well it can manage starting those up while even your web servers you know just running normal or vice versa. So you can specify the minimum number of instances you want running the maximum number of instances. You can specify ramp up and ramp down intervals so that it can monitor the load and if he sees that you're getting just a huge amount of load in your queue it can basically ramp that up a lot faster and if it sees that you're just incrementally the load is increasing it can do that a little bit slower so that it's not starting up so many instances. Same thing with shutting down so if suddenly it just your queue gets emptied you know all your processes do all the work. Basically it can ramp down slowly or it can ramp down pretty quickly. There's also you can specify with Amazon they charge you for the full hour of each service that you have of each instance that you have running. So if you're only using that instance for you know a couple minutes to do some processing it doesn't really make sense to shut that instance down as soon as it's done processing because Amazon is just going to charge you for the full hour anyway. So you can specify with the minimum time that your instance is running so probably I would think that you could set it at you know 55 minutes or something like that unless you had a reason to set it shorter. Yeah. So all the cabinet that you're spying about Amazon seems disturbing. Yeah. Well there's definitely things that you have to consider in deploying to Amazon. It's not you know this utopian deployment environment where you know everything just is going to be rosy for you. Basically you know it's just not perfect and so you have to account for these sorts of things if you're going to be using Amazon's web services. Yeah. I'm not sure like real world kind of statistics on you know how often that happens. In my personal experience not with EC2 but with SQS I've had a lot of different issues with working around those sorts of things where you know I'll put a message in the queue. I know it's in the queue but I call receive message and I don't get anything back but ten minutes later hey there's my message you know so it's like I said it's not perfect but you know it's not totally horrible either. Cool. So I wanted to actually put together a sample application that actually demonstrates this and not just you know talk about what the code can do but I actually wanted to show code actually running and working and so I thought you know back to that I love Lucy episode I thought hmm you know that's a really kind of cool way of illustrating you know the ability to scale but the problem is you know they couldn't just dynamically add and remove people as they needed but with Ruby we can and I wanted to put a little bit of a Ruby twist on that so rather than making chocolate we can make chunky bacon. So I've got a little workflow here little flow chart of the process of making bacon. So we start out with a slicing we call package on a slice of bacon and basically what happens is it gets added to the database so that we have a record of that and also gets added to the chunky unpackaged queue. Now we have Kato the ET2 pool manager sitting over there off the side monitoring that chunky unpackaged queue and we also have the cartoon foxes there those represent each one of those represents our ET2 instances that are running. So what Kato does is it monitors the chunky unpackaged queue to see do we have any bacon in there and how much and it looks into the chunky unpackaged status queue where the ET2 instances basically report am I busy or am I idle you know am I just am I really doing something or if I'm just sitting here and that way the pool manager can know well you know hey they're really busy and there's a lot of stuff in the queue start up more or you know they're just sitting there idle not doing anything shut them down. Once the instances started up it starts pulling those messages from the queue and it starts packaging them up. Once a slice of bacon has been packaged up it gets put into the chunky packaged queue where we have another process that's sitting there that's basically just doing nothing but pulling the slices out of the packaged queue and just updating the database that hey this slice has been packaged. So we'll just take a look at that code which again uses the right scale gems for that to kind of give an example of you know how this process is working. I basically just have a SQL model for this and so it goes into calls create which creates the entry into the database I get a reference to the chunky unpackaged queue I push the ID of the slice that was created in the database onto the queue and then I just return the slice. So the foxes they sit there and while they have you know wall packing they basically attempt to pull a message out of the queue they send that status update that I was talking about that hey it's now busy doing something and then basically all this does doesn't really do any work but it basically just puts that message body back onto the packaged queue and then deletes that original message so basically when we added the message to the queue we just put the ID of the slice in there and so basically the body contains that ID and we just push that back onto the packaged queue we send the status update that hey you know I'm not busy doing anything anymore and then just waits a couple seconds. The fox get packaged bacon basically does this kind of same sort of thing where it gets a reference to the packaged queue goes in it pulls a message out of that queue also have it do a check to make sure it actually exists in the database in case it didn't for some reason and then it just updates the package that time with what it is and then it just deletes the original message out of the queue. So yeah how often does the delete message I'm not sure but I wouldn't be surprised if it did and so you should probably account for that in your code. I've had instances of messages actually being delivered twice and so that's very possible that you attempt because it gets delivered twice with the same message ID and so your first time it gets out of there it gets deleted but somewhere in the Amazon's cluster somehow a duplicate of that exact same message is floating around and so if you attempt to operate on that it basically fails so yeah again you're you know the question earlier about the question here was how often or is it possible if the mess when you call message delete if if that would generate it could generate an error and yeah it totally could good all right so we've got our chunky bacon application here and it looks similar to the chocolate that we had in the Isle of Oosie episode and so we've got our cartoon foxes to help us out here and package up this bacon and so each one of these foxes represents an EC2 instance that's running now because I mentioned before it's not you know wouldn't have been wise to actually try to rely on really connecting to Amazon to do this I've basically emulated EC2 and SQS locally here on my machine so that when I do API calls to the right scale jams it's basically going to my emulated processes rather than going really out there to Amazon so let's get some letter roll you know letter roll and so we'll let her roll here so I'm just going to loop and slice some bacon here and so let's see come on here we go so the bacon is going down the conveyor belt right and so it's being packaged up as we can see it's being transformed there and we can let this go for a while but I know that you know it'll probably be fine so let's you know speed her up a bit so let's go ahead and do that so basically just double the speed on that from basically was doing every three seconds it would go into a slice and so now every one and a half seconds it's creating a slice so that's we can see that it's getting down the belt you know pretty good but the way that a half thing is configured it considered the load to be enough to go in and start up on another fox which it did and so it's still going down pretty good hey get another fox all right so basically it's monitoring that queue that we're putting the slices into the Kato process that I have it's monitoring the queue and it's it's saying the instances are reporting whether their their idol are busy and the queues taking that into account or the EC2 pool managers taking that account and starting and stopping foxes as needed because this is I'm pretty much trying to do this in real time and your Amazon instances aren't going to start up in seconds you know it's going to take a couple minutes so it's not exactly perfect here but it works okay so let's just you know really kick this up there we go you can see that it's not actually getting to all the slices as each slice goes down the conveyor belt I'm basically doing an Ajax call to say hey give me the status oh hey there's Darth Vader in the sombrero right on right on so as each slice is coming down the the conveyor bell it's doing an Ajax call every 1.x seconds or whatever to say hey am I packaged yet am I packaged yet am I packaged yet and we don't have enough instances I basically set the the minimum instances at two as as we saw when we started up the app the two were there and it'll basically just cap it at six so are there any questions at all I've got some time yep sure I have a service that I that I have that basically processes continuous integrations in an automated fashion and so when somebody makes a commit to the repository basically records in the database that hey you know I have got this new commit in there and needs to run continuous integrations on there so I can use Amazon cc2 service and sqs service basically go and fire up processes if I need to to go and run those builds and so in my in my particular case I don't really care what the message order is or what order I receive the messages in because it's just not critical to the way the application works and so for those types of things that works fine I'm sorry to repeat the question what was the phrase it used application but so is this the benefit of the queuing service being patting on that you can run to start as many instances as you want and all I've got is a queue okay yeah so it's the benefit of using Amazon sqs service as maybe a post just using your own queue on your own servers or something like that in the in the benefit that I found to that because I was using my own queue was that I can rather than managing my own queues and opening up my own ports and adding my own security and those sorts of things I can just have Amazon basically manage that for me and the service is so cheap that you know cost wise it doesn't really cost me a whole lot to do that and I you know address the reliability kind of issue where in my particular case I don't really doesn't really matter all that much if the message doesn't come right away or doesn't come in order and so basically using Amazon sqs services for that would be great there's a lot of new queue things queue services kind of popping up that you can run on your own service and if they manage authentication and they manage that sort of thing then by all means that that would definitely be a good benefit but I just didn't want to kind of have to reinvent the will to do all that So the question was rather than using Amazon sqs service because you may not be guaranteed that the messages are going to come out in the order that you put them in as could you use their simple DB service to do that and so presumably you'd have you know some kind of key or identifier that you could get the messages out in the proper or the you know the data out in the proper order and the answer to that is is that from what I've read and I haven't actually used Amazon simple DB service is that it could potentially have the same problem where you put something in the database and it'll get there eventually but it's not guaranteed that the next time you have a call to that database that it's going to be there so you could run into the same possibilities that you're running into sqs in that regard yeah The question is basically how do you decide whether or not you're going to use Amazon sqs web services for either your whole application or parts of your application for me that was pretty simple in the service that I built is that rather than provisioning a whole bunch of servers to basically be able to handle a seasonal kind of load I could have done that and I could have done that and I could have done that and I could have done that but rather than doing that with I have parts of my application that aren't running on Amazon's web services and the parts that I need to basically be able to dynamically scale without having to go through a provisioning kind of process on servers that's where Amazon really had value there for me and so I would say that you know the ideal kind of use case or candidate is that sort of thing where you have an application where you need to basically be able to scale up pretty quickly and if you need the ability to be able to scale up dynamically even more so you know a lot of cases you could just manually fire up your instances not have to worry about checking queues and that sort of thing to determine whether or not you needed more but you know you can have a case where you know you're just going to get a high load for a day or two or something to start up some additional services to handle the load and then manually turn them off when you need to so I would say that those kinds of use cases where you basically need to be able to quickly bring up new instances and or shut them down quickly would be the use case for that. On the web services and some just standard? No, I do that myself where I have part of my application running on not Amazon's web servers just as server at a colo because I need it running full time and it's not something that I need right now to be able to scale at all because it's basically just doing simple database queries and things like that that I'm not really hitting any kind of scalability issues in that regard but on the other hand with being able to run continuous integration builds if a whole bunch of developers check out a whole bunch of code at once or in a very short amount of time the static server that I have sitting there won't necessarily be able to process those continuous integrations in a timely matter and so it would be a simple thing for me to just fire up some EC2 servers for an hour and basically chunk through all those continuous integration builds and then turn them off as it needs to. So yeah part of my application runs on just static non-EC2 servers and the other part that I need to be able to scale can run on EC2. EC2 just recently announced a day or two ago that they're adding static IP addresses to your instance so basically your account gets a reference to a static IP address that you can then assign to your instance because one of the big things about EC2 before this was that it just gets some random IP address somewhere and so people were wanting to run their entire application everything on EC2 well that's and you know when I go to www.whatever.com how does it get to your EC2 instance because the instance gets allocated a dynamic IP address and so there are a lot of different workarounds for that and dined DNS and those sorts of things. Well Amazon just recently announced that you can have a static IP address now so that when if you have an instance go down which again I'm not entirely sure how often that really happens I would think Amazon says that basically your instance can go down if they have some kind of hardware failure or you manually shut it off and those are pretty much the only two kinds of situations that would cause your instance to go down but should it go down you could basically start up another instance and even start up another instance and maybe a different colo they're now a cooperation center and basically start up another instance point your IP address to it and basically it'll start using that so any other questions? Yeah Where do you keep your data? Where do you... Well in here I just kept it in a minuscule database Do the nodes communicate or do they go the instances don't communicate at all to the application server or the database which is part of the reason why I chose to use SQS for basically doing in the service that I'm actually running for continuous integrations So what I did there is I mentioned earlier on the slides where you can only put so much data in as a message so you're pretty much going to be using like an identifier or a file name for S3 so in my particular instance what I'll do is I will go and upload the code to S3 get the file name for that put the file name in SQS along with a couple additional pieces of information like what database to use for continuous integrations and that sort of thing so the message is still really small so I'm going to go and run the build when it's done running the build it will go and it will basically use part of that identifier from the first message put that into basically a done queue and will upload in a YAML file the results of the continuous integration build onto S3 then I just have a process sitting on my application server on my non-EC2 servers that just go and pull that queue go grab the file out of and update the database there yeah you can basically S3 as a data transport I could get my code to the instances would still be S3 but I could use like simple DB if I wanted to get the status of the build just store that in there I probably could do that but unlike the SQS the simple DB I'm not entirely sure seems like there's a lot of different options that are starting to pop up out there that could basically do the same thing maybe even better or easier couch DB we're going to have a talk on that you could probably use that you know there's the stroke DB and those sorts of things where rather than the simple DB you could use those sorts of things because I think eventually it will have authentication and those sorts of things so that I have to worry about setting that up like I said with the SQS if I wanted to spend the time to get authentication working so that my instances regardless of where they're at can communicate with my queue then I'd probably go ahead and probably not use SQS anymore because it does have issues you have to account for when you're using it cool so resources you can actually get this application on github and it has all the code up there the entire thing like I said I've emulated EC2 for this and so what that does is just basically a MIRB app that responds to the API calls that the right scale jams use and basically uses God to load a config file for what my foe EC2 instances and does the processing that way and then I've also got another MIRB application that emulates the SQS queue for this particular use case because I needed things kind of real time it's just hammering the heck out of the queue doing multiple per second queue requests in kind of a real life situation rather than this kind of contrived example you wouldn't be hitting your queue so fast more than likely and if you did it's going to be kind of slow because it's having to do an HTTP request over the network to Amazon process that and get that back to you and so that's not always a really performance critical kind of thing like I wanted to do here so here I'm just monkey patching the right scale SQS jam to basically just talk to a local database instead of talking over HTTP that does emulate the parts that the app could have used if I didn't need it to be so performance critical well thank you