 It's my privilege to be here. I've been associated with the FATHP session for a while now, presented at PHP in the cloud and couple of other events often as the. So, my name is Janaki Ram. I don't have a slide. This is a very interactive session. I'm not going to. Okay. This is okay. I'm going to stand here. Actually, no, no, this is good. So my name is Janaki Ram. Before I started off on my own, I was with Amazon Web Services for about a year. And prior to that, I was with Microsoft Corporation for about 11 years. I've been involved in cloud related stuff for about four years now, worked on Windows Azure, worked on Amazon Web Services. And now I run an initiative called get cloud ready where I primarily help individuals and ISVs to develop applications and refactor applications for the cloud. So before I get started and walk you through some scenarios and use cases, let me take a quick poll. How many of you are familiar with Amazon Web Services? How many of you are attending a cloud related session or hearing about cloud or have never done anything on AWS? Okay. Have you have you deployed a server on AWS? How many of you? Okay. Great. Okay. So this assumes that you're familiar with Amazon Web Services. I'm not going to do an intro to AWS, but given the fact that there are more number of people who are new to AWS, I might just give you some background. And then I'll actually make you appreciate the value of cloud formation. So to quickly walk you through AWS and the current landscape, what is actually happening in the space? So cloud has certainly changed the way resources are being provisioned, resources are being dealt with. And cloud is all about automation. What typically takes to set up a stack on a physical piece of hardware will be reduced to a few minutes when it comes to cloud, right? So cloud is extremely powerful because of the automation capabilities. And there is a new concept that's being evolved, which is called DevOps. How many of you have heard this term called DevOps? Again, three, four of you. So the most interesting part of cloud is the cloud turns developers to be administrators and administrators to be developers. When it comes to cloud, you can't say you are a developer and you don't care about firewalls, load balancers, server sizing, capacity planning. As a developer, you still need to understand some of these techniques and the best thing is you'll be able to write code to provision most of this stuff. So that's the power of cloud. And that actually creates a new job function in the environment, which is called DevOps, where administrators become partial developers and developers become partial administrators, right? Imagine writing code to set up a load balancer to set up a firewall, creating four different servers in a web form, adding them to the load balancer, then creating a master slave database cluster and pointing them to your app servers and then completely declaring this whole stuff in code and then executing the code to set up the complete virtual data center. So if you carefully analyze this, you have developers trying to administer the stuff through the code. So those individuals who are capable of writing infrastructure code and treating data center as a programmable environment are the new DevOps professionals. They are the developer operations professionals. They do what administrators do, but they are much more cutting edge because they actually script it and code it, right? So this has fundamentally changed the landscape. This has turned into a new revolution in IT operations and deployment world because today it's all about repeatability, right? You can't just assume things will work. You may want to create a web form, a complete high availability architecture, a topology, and you should be able to replicate it multiple times. So with that concept, there are new tools like Opscode Chef and Puppet. Actually, there is a parallel session happening on Chef. So all these techniques evolved primarily to manage and help you manage the configuration at runtime. So what I will do is I'll quickly walk you through a scenario and then I will show you what we are going to do with cloud formation. So this is basically the AWS console and because some of you are new, I will spend exactly five minutes walking you through the nuts and bolts of Amazon Web Services and make you appreciate the value of cloud and then we'll move to the next phase of cloud formation and see what it can do for us. Okay, let me ask you, if I am running a website on my local host on VAMP or MAMP, how long does it take for you to go live with a live domain and take it beyond your local host to the public world? How long does it take? Sorry? Three days, okay? Any more? Any more? That's a simple application. It's a LAMP application. Depends on the web hosting provider, right? The turnaround time could be anywhere from 24 hours to 48 hours to even 72 hours. That's a maximum, right? Okay, so I basically have a local host application that I am running and what I want to do is to go live with the same application on the cloud. So let me walk you through what it takes to basically launch this. So Amazon Web Services is primarily the cloud computing environment that comes with fundamental building blocks of doing anything meaningful for your application. So essentially today when you are going live, you need to deal with a bunch of building blocks. Definitely you need compute, you need servers that have tremendous power. You need massive storage for durability and for scalability. Then you need a little bit of networking in terms of firewall load balancers and so on. Then you need databases. So all these four are the essential building blocks for developing and deploying applications, right? So traditionally you go ahead and procure them. You set up physical machines. You install the stack on cloud. These are available. These building blocks are available to you in the form of pay as you go model, right? So EC2 or Amazon Elastic Compute Cloud is primarily a flexible elastic compute environment that's available to you, right? So you can launch a server and you can scale up or you can scale out your application. So what does it take to launch a simple server? So let me launch a plain vanilla Linux box and then set up Lamstack and quickly go live with my application. So I want to first show you the manual way and then we are going to automate the same process through cloud formation. So I'm not sure if this is visible. Is this visible? Okay, great. So here you have multiple varieties of operating system choices. So Amazon has a distribution customized for the cloud called the Amazon Linux AMI. So I'm going to select that one. And in the next step, I'm going to choose the configuration. So if you're able to read this, there are multiple configurations. For example, the entry level configuration is called a micro tier, which comes with the burst compute capacity up to two ECUs, which is two CPUs roughly, and you can go all the way up to 20 ECUs with eight core siege and seven GB or even 70 GB of RAM. So you have a variety of configuration choices that are available. Let me stick to the entry level configuration and I can launch multiple instances at the same time, but I'm going to keep them as one. And then you can safely skip this. And here let me just call this the lamp server. So I'll create this and then I need an SSH key to log into my server because sending the username and password in clear text is not an option, particularly when you're on the public facing internet. So there is a private key that's available to me and I'm going to choose that. And then I'm going to configure a firewall for this server. So I'm going to choose HTTP and let the traffic come from any IP address. I'm going to add that then I'm going to open up SSH port. So basically I'm configuring a firewall for my server or I could choose one of the existing firewalls just pre-configured. So I'm going to choose that and then I'm going to finally review this whole thing and click on launch. So this process has essentially launched a plain vanilla CentOS-based Linux box. So in a few seconds, this server would be available to us. Now meanwhile, what I'm going to do is to bring up a very simple script that I've already saved. So this is still pending. It should take about 30 seconds for this box to come up and to become available. I don't think I'm walking you through AWS 101, but only when you see the manual operations you'll be able to appreciate the automation part of it. So now I basically tap this public IP address and let's open another browser instance. Obviously this is not going to show up anything because we are not running a web server, we are not doing anything meaningful in that. So now what I'm going to do is SSH into my instance and I'm going to log in as DC2 user. So this will get me into my box. So this is the beauty. Now I quickly become a root user and this is my box. It took us less than 40 seconds to bring up our first server. Then I'm going to run a mundane script to get us the lamp stack completely set up. So I'm going to pull it from the M repository that Amazon internally maintains. So it's extremely fast. In few seconds you would actually see that the entire lamp stack is going to be up and running. Then I run a bunch of other commands to configure the lamp. So this is going to make sure that my Apache, MySQL, PHP is set up the right way. Then I'm going to basically pull the latest build of my source code either from an SVN or from S3. So I have already uploaded my latest build of the website there. I also get a database dump into my box. So now I have what I actually need. You're not able to see this but there is a website.zip and then there is db.sequent. So now I go ahead and unzip my website into... okay there's a typo here. So I did that. Then I'm going to quickly import my database into MySQL. Now I come back to the browser and do a refresh and there we go. So that's our lamp website that has gone live. And this whole process has taken less than two minutes. Like from the time you launch the server we have gone live. So in this process we have gone through a series of steps. We have actually launched a server, we configured a firewall, we attached a private key to it and then we accessed and ran a lot of scripts. So let's say you want to do this across multiple regions, multiple data centers in a repeated form and you don't want to really do it manually because when you are attempting anything manual the chances of making an error is far more higher. So how about achieving the same thing through some kind of a declarative form where you're going to basically define your cooking recipe and then just publish it and get the same stack up and running. So this ability of automating and declaring your stack in in plain English is what is called as CloudFormation. So CloudFormation is a runtime environment on AWS that lets you define the stack that lets you define your entire configuration and your server orchestration in plain English and when you actually publish that the same effect that we achieve now can be completely automated. So that's exactly what is called as CloudFormation. So let me show you one of the very, very simple scripts here. In fact, let me start creating a simple JSON file on my desktop. So what does it take to launch a server automatically as in instead of really going to the manual stuff and the manual process of clicking through it, how about declaring the whole thing. By the way when it comes to Amazon Web Services there are multiple ways of achieving the same thing. Basically there is AWS control plane and then there is an API that is exposed by Amazon Web Services which is very, very popular and multiple tools leverage this API. There are command-line tools that you can actually use or you could also write program in any language of your choice all the way from Java to C sharp to VB or even you can script it using Ruby or Bash and so on. But all those require some kind of scripting, programming and dealing with the API. This is the first time that Amazon has come out with something which is very similar to a Chef recipe but this is more towards orchestrating the server than configuration. CloudFormation is not a replacement to Chef or Puppet. It's more of a complementary tool or a technology that works really well with the Chefs and the Puppets of the world. So Chef and Puppet are essentially meant for configuration of your software after the server comes up whereas CloudFormation is more about orchestrating your cloud resources and basically provisioning your cloud resources. And once CloudFormation sets up your primary mission it can bootstrap Chef and download the cookbooks and the recipes and can actually execute whatever you think might. But this is a level above the question of the Puppets of the world. How many of you are familiar with the Chef cookbooks and the recipes? So let's get started. Let me walk you through what it takes to basically declare the launch of a server or the configuration of a server and then use CloudFormation to achieve the same scope that we have. So let me start with this. So I'm going to create the very first so CloudFormation is primarily a JSON file. It is extremely simple to get started. So it just consists of a variety of name value pairs completely serialized in the JSON format. So it starts with a very first line where you define the template format version and this template format version is based on a specific date. So currently we are using a version that is 2010 0909 after that you can give a simple description which is going to define the description of this template. So basically a template is more like the decorative format and from the template you can create multiple stacks. A stack is a live implementation of a template. So now we have the very basic skeleton here and this can have multiple resources that can be defined. So these resources are going to be for example we are going to define an EC2 instance. So it starts with this and then we can actually start explaining multiple resources. For example EC2 instance is based on type so let's define the type. Don't bother much as I type you're going to get a lot of clarity but once you understand what is how a stack or a template is created you'll be able to relate to a lot of things. So for now I'm basically creating the simplest cloud formation template to launch one server. So I'm going to create an EC2 instance type. So that's the type that we're going to launch, the cloud resource. But the instance in itself requires multiple properties. For example you need to define what is the AMI type, what is the key that you are using, what is the firewall configuration and what is the instance type, are you launching a micro instance or a large instance and so on. So you need to explain a lot many details when it comes to cloud resource. So here we start another simple snippet where we're going to define the properties. And within the properties I'm going to define the key name of my private key. Then I'm going to define the instance type. Let's call this the T1 micro which is an internal name of the micro instance that AWS gives you. Then of course we need the image ID which is the Amazon machine image. So basically when you're launching any instance on AWS it has an associated AMI ID. So this is the AMI ID. So let me grab the same AMI ID and put that here. Oops, let me get it from a better place. This is not the best way to copy. I'm going to grab the same oops. This is the AMI ID that we are trying to launch. It's kind of unfortunate that we had to go through this step to get the AMI ID. Then we are going to define the security group which is the firewall related to this server. So I'm going to add another name value pair. This time it's actually so basically this is the basic format or the cloud formation template that we have just created. Now I wanted to spend some time analyzing what we have typed here. So these two are more like the mandatory requirements and after that you start explaining and articulating your cloud resources. So it starts with resources and within that we put easy to instance which is mapped to a resource of the name space. For example if you're adding a load balancer that would be awsvc2 elastic load balancer or if you're adding an elastic block store the type will change here and once you define this we are going to have multiple attributes that will explain the resource better. For example what do you require to launch an instance? You require the private key, you require the instance type, you require the image ID then you require the firewall that's associated with it. So this is the simplest very first cloud formation template that we have just created. Now we will save this and then come back to aws management console and go to the cloud formation tab. By the way if this process is not very intuitive as in obviously when you're automating you never want to deal with the GUI or the browser there are command line tools for you to launch the same template from the command line but to keep this simple through the conventional model of cloud formation I will use this wizard. So now let me let me call this Linux and I'm going to upload a template file. So this is the template that we just created and I can also give advanced options related to this. For example I can raise a notification which will tell me whether the stack has been successfully created or not because some of the stacks are extremely complex and long running. If you are setting up a load balancer and if you are setting up a cluster that might run into a few minutes and you don't want to stare at this so you can associate this with a notification mechanism within AWS called SNS and you can actually get a mail or a or an sms the moment this process is completed. Now during the process there may be a failure in which case you can roll back the entire stack for example in a transactional mode you might want to roll back and reverse the entire operation all the way from creating a firewall to creating a new EBS volume load balancer the moment something goes wrong in your stack the whole stack gets reversed. So that's the option here roll back on failure and you can also define a timeout in seconds. So this is going to basically sorry this is in minutes and you can basically tell how long the stack can execute and when does it gracefully exit. So that's the advanced options here and then this is going to parse our simple template that we have created and it also uploads this to a location that is centrally available to AWS. So the simple stuff that we created here is now available and it's being uploaded to one of the cloud storage locations. So now I'm going to click on continue. So the moment I click on continue it's going to parse everything and starts the creation of the stack. So while this is happening let's come back to the EC2 console and do a refresh and if that has kicked off the process you would actually notice that there is a new server that's going to be created for us. So the question is based on JSON can we have any other format? Thanks. No, this is actually a JSON standard. You can't really use XML or you can't use any other format because the runtime that's going to parse the cloud formation template is designed to understand only JSON. You can set up command lines tools for the same thing. So let's see if we are back in action here. So we will review the options here and then when we continue this is going to kick off the whole orchestration process and if you look at the events here it's going to tell us what is going on behind the scenes of the cloud formation. So currently the Linux EC2 instance is being created and while this is happening we can go back to the EC2 console and do a refresh. So here you'll see that there's a new server that's being provisioned. So this is coming from the same JSON that we just created. So now this should be done. So here you'll also see the status create in progress. The moment this turns green it's an indication that the stack has been implemented successfully. This will also turn red in which case it's going to get rolled back and resets the whole configuration. It's going to undo everything that it has done as a part of the stack. So now this is complete. So this turns green. So this is one of the simplest AWS cloud formation template that you could create. But next what I want to show you is slightly more complex template to automate the installation of WordPress. So this has multiple things. So at a very high level any cloud formation template will have parameters, mappings, resources and output. So these are the various sections that you can put in a specific cloud formation template. So parameters are the dynamic parameters. For example, you don't want to hard port any of the instance types but you want to prompt the user. So when you create a parameter and when you are executing the stack, it's going to ask you for the input values and you can also define a set of valid choices by creating the appropriate values here. Similarly, we'll prompt the user for the DB root password and you'll actually see there is something called no echo is equal to true. So this is an indication that what you're prompting the user is a password and it shouldn't be shown on the screen. So when we're creating the WordPress stack we are primarily asking for the instance type and then we are asking for the DB root password. So that's about the parameters. Then there is another section called mappings where you're actually going to pick up from an existing map. Okay, this is a lookup kind of a thing where when you are executing this script across multiple regions of Amazon web services this is going to map to multiple machine images. So you can take the same script and run it across eight different regions of Amazon web services and when you are executing a specific region this is the mapping that it's going to use. Similarly the architectural details of every instance type also comes from the mapping. So mappings is primarily an array of values that you can look up and it's almost like a drop down when it passes during the interface interpretation. So once you create the mappings then you go ahead and create resources. So here we're also creating a user. Are you familiar with the concept called IAM Identity and Authentication Management in AWS where you can create an enterprise level account and you can have multiple sub accounts. So what we are doing here is we are creating an identity and access management user only to access the cloud formation metadata. So we create that user and then we use his credentials of the access key and the secret key to log on to the web server and automatically install the LAMP stack that's required and then it can also pull the latest version of WordPress from the location. So it's going to get this and automatically expand into the W3 HTML directory and then it's going to create a very simple script which is to create the WordPress database and it's going to get the details that we entered during the setup. So now you'll actually see there is something called ref dbroot password. So this is the dbroot password that would have prompted the user to enter during the execution and that's going to be replaced here. So this is a specific notation that's being used within cloud formation where you're dynamically replacing this value with whatever the user has entered and then this is primarily the WP config file that's going to be created and finally it's also going to make sure that the the HGTPD, MySQL, mail services are being started automatically and it basically runs all the script to basically run the MySQL commands to create the database and to execute the database script. So finally this is the five wall configuration that's being again described declaratively within the template. So we are opening port 80 for this and this is a CIDR notation where the traffic can come from any IP address and finally the output section of this template will join the web server public DNS name and finally gives you the final URL of the WordPress website. So let's go ahead and execute this. So I'm going to create a new stack and let me give the name of the stack as WP and I'm going to upload this the WP.json in the next step it's going to parse all the details that we have asked. So here as you see there is a DB root password and instance type. So this is primarily coming from the parameters section that we have defined here. So here the instance type is a parameter default is M1 small that's what you would see here the DB root password is also being given here with no echo. So let me change these parameters let me change this to a T1 micro and because we are creating an IAM user I also acknowledge that I'm going to give access to this guy and click on continue. So this is going to provision a plain vanilla lamp server and then download WordPress build from the WordPress.com site and automatically configure everything that we require. So if you keep an eye on the events tab this will take a while because it's going to do a lot of stuff all the way from creating a plain vanilla server to creating multiple firewall configurations and also downloading the latest build of WordPress. So it does whatever we are supposed to do to set up WordPress manually. So in about two and a half three minutes we'll actually notice that the entire WordPress stack is up and running. So while this is happening any questions? So a good question I think I partially answered this before you joined us. So basically the fundamental difference between cloud formation and complex share is complex share for more like complication management. That is much more useful after you bring up a server like for example I mean a combination of with for me which is right this isn't particular which which right. So the number one reason is if you are an Amazon customer and you have a massive investment you can you can integrate Chef and Puppet with cloud formation. For example just like I am bootstrapping WordPress I can bootstrap Chef and I can point it to the cookbooks it can download and perform exactly the same. But to bootstrap Chef I need an external environment even before I get up to that point and that is cloud formation. Of course but this is an official Amazon flavor of the same and this is meant for customers who are going to be married to AWS. It doesn't give you the platform agnostic advantage but then if you are already massively invested in AWS this makes it further easy and there are going to be many features coming into it as in see inherent capabilities of adding an elastic block store or creating an elastic IP is going to be very hard with third party tools because they are not aware of certain certain integrities of dealing with AWS specific resources but cloud formation is completely aware of those. For example have you played with AWS you are familiar with okay so you know EBS right so in the same declaration I can actually block let's say one TB of elastic block store and I can add it to my instance and mount it create a mount point and dump my database log files on to that while declaring the stack. So that's a kind of integration it has but I totally agree it's not very unique as in you can achieve the same effect same impact by using Chef and even Ruby scripts but this is more like an official Amazon flavor of partial DevOps. Okay so hopefully we are done with this formation now so yeah creation is complete and now if we come back to the management console and look at this so we have a new server that's up and running and let's grab this public DNS wow it hasn't really deployed WordPress yet no it actually dumps it in the root because if we look at the script it is in the root right oh okay okay let's also try this you're right cool so okay my guess was we have dumped this into yeah so when you obviously expand WordPress.zip it dumps it into WordPress right so that was essentially the script for the WordPress deployment I know this is like scratching the surface many more use cases in fact I have one more cloud formation template that I am working on where you basically get the same e-commerce portal that I have run earlier automate the whole stuff but somehow in the last minute I couldn't really get that done but you can pretty much automate everything that requires manual intervention and you can it's almost like baking versus cooking so according to Amazon Chef is more like cooking because when I get the portions recipe are like very accurate it's almost like baking if you are dealing with Chef it's almost like cooking because the proportions may vary so you get more accuracy when you are using cloud formation I don't work for Amazon but that's what I used to say when I was working for Amazon okay so I don't have much time but questions yeah anything as long as you yeah so the question is can I use CentOS actually depends on the AMI ID you can pick up an AMI that belongs to Ubuntu and you could pretty much repeat the same stuff anything anything that the OS can understand can go into that yeah does this integrate with route 53 absolutely in the resources section you can actually create a new zone and you can start registering your domains there so is the question installing the certificate related to SSL right so the question is can you also associate the certificate to a specific domain during the cloud formation template I'm not too sure but my understanding is it has to be done after the instance completely comes up and then you have to actually execute the route 53 command line tools to do that absolutely so what you can do is as a part of the installation for example here instead of downloading the WordPress binaries you would actually download the route 53 command line tools set them up update the environment variables and execute that specific command which will map your certificate you can invoke that yeah any more questions so your question is this is the last one the question is how do you build an Amazon stale environment in your own premises there are multiple things you can actually set up a private cloud either based on euclipters or cloud stack or open stack and emulate pretty much a lot of things that Amazon gives you but let me tell you Amazon is far ahead in the game getting all those capabilities within your premises is very difficult to achieve but you can emulate most of the core capabilities through either euclipters or cloud stack or open stack so thanks for attending this session I am around for any discussion thank you