 Awesome. So thanks for joining us. Nice to see the room full. It's a sign that you care about OpenStack and whether it's cost efficient or not. My name is Bruno, I'm the general manager of cloud computing at Catalyst IT. I feel very privileged working for Catalyst IT because I had the experience of going through planning, designing, building, operating and running an OpenStack cloud, the public cloud, the first public cloud in New Zealand. And as a result of that, I think I have developed a good understanding of how the numbers stack up for OpenStack, how the cost model works, the total cost of ownership and how that compares to other cloud providers. We're asked about how our cloud compares to other cloud providers very often. And we also help customers to do their own OpenStack implementation, their own OpenStack private clouds and in that context they also ask us, you know, how much will my computing senses in my private cloud cost compare to using this or that. So that's from where that experience that I'm sharing with you comes from. And this is my co-presenter who made it. Yes. Thank you. I'm Bruno Morel. I'm the software development director of the internet basically. And I'm sorry, I'm so jet lagged that I slipped through. That's okay. As you can see, I'm all sweaty. I'm not used to that weather. We work in Montreal so you can imagine that the weather right there is not exactly on par. So we are a public cloud provider. So we provide public cloud-based infrastructure. Everything is based on OpenStack. So that's why we think that we have a communication and some things to share about the comparison that we can make with Amazon, especially in pricing. I have a theory that you're sweating because you think OpenStack can be up for. Yeah, I'm so scared. You're worried. I don't know what's inside the presentation. I didn't make it. So I never know. Right. So would you like to start with that? Yeah. So of course it's very complicated to compare, as we said, Apple and Orange. So basically what happens if you know a bit about Amazon Web Services is that there's so many variables in the way that they price their offering. So as much as we can, we try to make it fair for them. Because the point is not to cheat or anything. The point is to make it so that if ever you were to have the same exact services, one of OpenStack Public Cloud, one on Amazon Web Services and maybe one on your own private cloud, that you would be able to compare and see what's the difference. Okay. That's the point. So we add to simplify a bit. Just for the sake of it. There's no way that you could compare exactly the model that Amazon is using simply because their model is not exactly fully transparent if you're thinking about how commit of the different way that they calculate the VCPU, for example. And there's also we won't go into storage differences. We won't go into the way that they calculate some behind the scene of their services. So the idea was basically to find the baseline we could compare and from there we can expose what is exactly the difference. And of course, depending on your own workload, you will have to try to custom that to your own needs, basically. I'll give you an example where that applies. So Bruno is comparing AWS prices with the Internet Public Cloud, whereas what I'm doing here is I have produced a total cost of ownership model for an OpenStack Private Cloud implementation that is being shared openly under a Creative Commons license as part of this presentation. So you guys can go in and see how that was done. But for example, if you have a computing system on Amazon with 3.75 gigabytes of RAM and on the Internet Cloud Bruno has got something with 4 gigabytes of RAM, then there's a slight difference there in terms of the comparison. What we've done in those situations is to actually put that advantage on the Amazon side and we've chosen on our side a flavor that actually delivers more CPU or more RAM than what Amazon is delivering just to make that as fair as possible. Yeah. So let's talk about hardware. So the first thing that we took into account, of course, is the compute itself. So we started with the CPU, trying to find as much equivalent. So on our side, we have two flavors of Cloud Plus bare metal. What we did is that we took what we call the A1, which is an overcommit ratio of 2 to 1, for all the M1 of Amazon. But then again, you will have to go into the detail if you think it's fair, we do think it's fair for Amazon. For everything that was more than M1, we used what we call the B1 on our Cloud, which is basically an overcommit ratio of 1 to 1. So you get one VCPU for one core for one VCPU. So from there, we looked into the storage because, of course, you can have no storage. You can have SSD. You can have a spin dig. So we tried to match as much possible, not only the type of storage, but the capacity of the storage, and the RAM. Because, of course, that's also an important part, depending on your workload. The number of gigabytes of RAM that you get is really important. And once we move all of that, we looked at the different price to match as much possible the price per hour that Amazon was providing you. So one thing that I would like to highlight is how these prices vary per location. So if you consume public cloud services, you probably have seen that public cloud prices tend to be cheaper in the United States and more expensive in Europe. And in our case, we're based in New Zealand, Australia region, and that's where the prices are even more expensive. So what we've done is we are using, for the Internet comparison, Bruno has got a region, Canada, the United States. So to be precise, it's US East, so New York, New Jersey, West, so the equivalent of San Jose for us. And in Europe, Amsterdam. So we took the Virginia for Amazon. We took the California one, I don't remember the city. And Europe, it was Amsterdam too for Amazon. On the private cloud side, there is something important that I need to share with you. I have included in that total cost of ownership reference prices for the actual hardware. So when I talk about a compute node, for example, you will find in there a bill of materials with all the, you know, the chassis, the server board, the CPU, the RAM, and some reference prices there. But what is important for you to know is that I've based those prices on what I can buy in New Zealand, which is typically more expensive than, you know, those of you coming from the United States would pay for hardware. So once again, that is an advantage for Amazon in the United States because I'll pay more for hardware in New Zealand than Amazon would in the United States. So I kept that as it is. The only thing I've done in the private cloud model is to convert the end cost, which is in New Zealand dollars, to US dollars, just doing a current conversion. And I'm actually going to go back and compare that with the Amazon price in the United States on this course, which is where I believe you get the cheapest prices for Amazon AWS right now. So, yeah. So on Amazon, there's a lot of services that you can get. So there's EMR, there's DynamoDB and so on and so on. There's a lot of Galaxy of services. OpenStack has some of them. Some public cloud providers do provide some of them. Some don't. So what we did is that we removed that from the equation, as sad as we were, just for the sake of being to compare something that's fair for everybody. Of course, we will compare the price of our own public cloud. So there's nobody of rack space that's here. Anybody know? Okay. Yeah, hello. So just to be fair, we would like rack space to do the same exercise. As Bruno said, we are going to share the spreadsheet. So you just have to put the number there and be able to see what's the difference. So your marketing should be able to do some stuff around with that. And so for the services and all the bells and whistles that Amazon gave you, we simply make it so that we made the hypothesis that you had to manage that yourself, basically. So we added engineering hours, basically, of a DevOps, two hours. What we made is we took an average of what we think is a fair salary, even though you could even go into that and put your own salary and be able to see what the numbers look like. So it's $90 an hour, because there's a difference between even inside the US it's different, in Canada it's different, in New Zealand I'm guessing it's different. And we considered it overall during a span of a year to just maintain, for example, a DB. A database instance. A database instance. You would have to input two hours per month. Then again, you can change that and CO2 E3 Act. If you need more than two hours, that means you have other problems, but then... So that was the assumption on the Internet public cloud, whereas for the private cloud, I'm assuming you're running Trove, I'm assuming you're running the Neutron load balancer using something like Octavia. So on the private cloud side, I haven't added any time for a database, a managed database service or a managed load balancer. Now there's also something I would like to acknowledge here and that is first the fact that Amazon has been doing this for a very, very long time. And I'm saying this because many years ago, I was doing a lot of cloud computing on top of Amazon AWS before OpenStack existed. So the fact that they have a breadth of services that is not yet available in OpenStack, we see that richness happening with the BigTent and more projects coming up, but there are things you cannot find yet that exist in there. One example, and please correct me if I'm wrong, if you're deploying Elasticsearch as a service in OpenStack, as far as I'm aware, there's no service yet that allows you to do an Elasticsearch as a service on top of OpenStack, whereas on Amazon you can go and say, give me Elasticsearch, right? So one, we would like to acknowledge that there is a breadth of services in there that doesn't exist yet in OpenStack and it's a matter of time we are a young community. The second thing is just acknowledging the fact that a lot of what we've done in OpenStack as well has been inspired by what Amazon AWS did. So when I look at things like Nova or even Heat, the fact that Heat started with cloud formation templates and then the actual heat templates after that, you can see that we derive a lot of inspiration from what's happening with other cloud providers. So just acknowledge that fact. I think it's important. So let's get started. Enough of the introduction? Yeah. So that was basically the model. Just to explain where the difference will be. What we wanted, again, was to be fair with Amazon. I don't know if anybody of you know the Amazon Monthly Calculator. Looks like this. You're not expected to read those things. Don't worry. What's important is what is underlined is basically the template. That's what we took. We took the template. We made an equivalent of all the bail and whistle of Amazon. And then we looked at the number. I think that it's the fairest. It represents a real business use case. That's the way that Amazon interpret them. So I would maybe challenge some of them in the way that they are spread. But then that's the way you can compare something that's fair. That's the way they at least try to simplify and demonstrate what their price point is. So that's what we are going to use. These templates, when you go to the AWS Calculator, on the right side you have common customer samples. And you have things like a large web application. And if you click on large web application, that will pre-populate the calculator with some web application servers, some storage, some for you on your behalf. So we took that reference architecture that goes into the calculator and we opened stack prices to it to have a like-for-like comparison. So there's one thing I would like to tell you about the total cost of ownership model on the private cloud side. If you're deploying a private cloud, I have made an assumption that you're not trying to go to market and make money with it and have a profit margin there that would allow you to afford a big marketing team, a big sales team. What I'm assuming there is that you want to recover your costs. You're doing a private cloud and you want to pay for that private cloud and make sure that you recover your costs. So I won't show you the actual spreadsheet because I'll probably, everyone will go to sleep if we do that. You're welcome to have a look at it later. But the one thing that I want to show is that to make this comparison very interesting, I wanted to show you what the bottom line, what is the lowest cost you can get from open stack would look like and try to get as close as possible to it. So we're talking about a large private cloud. We're talking about a private cloud that will have something like 200 compute nodes, about 100 storage nodes, a few racks, a spine switch layer, some top rack switches. And it will have staff operating this cloud 24 by 7. So I have assumed six people operating this cloud 24 by 7. And what you can see here is that per month that private cloud would cost something like $428,000. It's a large private cloud. You can scale that back in that model if you want. But the important thing is if you're using all the resources provided by this cloud and there is a lot in there, there are 86,000 vCPUs, 226 terabytes of RAM, a petabyte of object storage, almost a petabyte of block storage. If you are using all of this and selling it for what I say that cost is, you would get back the money, the investment you've made on your private cloud to the point where you're recovering your costs. So ultimately it's a cost recovery model. It's not, I'm going to market it, I'm going to sell that private cloud. This is what his model is. So you get both tastes of it. And we wish that we could sell all of our capacity 100% of the time from a thing that's not always the case. There are variables in that model that would allow you to tweak them and say I won't have my capacity use 100% of the time. I'll use 80%, 70%. So feel free to go in there and tweak that. Yeah. So let's look a bit about the number to give you a bit of an understanding of what we are really talking about. So if we take an M3 medium East Coast instance in Amazon, that would be one vCPU. Again, you can go into, look into the detail of what really that vCPU is. We consider that it is a full core. You get 3.75 gigabyte of RAM. That's the Amazon thing. And you get 4 gigabyte of SSD. So the way that we matched it on our cloud would be what we call a B11. So again, it's one core per vCPU. That's one vCPU. You get the full core. Enjoy it. 4 gigabyte of RAM and 4 gigabyte of SSD. So it's pretty much a match. Whereas on the private cloud side, the CPU overcommit ratio on the TCO model is 4. I believe most cloud providers, public cloud providers are doing some CPU overcommit, not memory overcommit, but some CPU overcommit, because the reality for most of them is that their servers are underutilized and they are trying to bump it a little bit on the CPU side. So there is an overcommit of 4 in the private cloud model. And by the way, we kept the prices here all per hour. Notice that there is an additional zero here compared to that. Maybe we should have done prices per month, so it would be easier to read it. So if we look at a second example, so that would be kind of a big machine. So it's a C3, sorry, 8x large. That's something that you want to use with the huge workload. That's something that your mother should rent. So it's a 23 vCPU. That's the word number, but that's the number. You get 60 gigabyte of RAM and you get 2, 3, 20 gigabyte twice of SSD. On our side, unfortunately, we don't provide such a big virtual machine. So what we decided to do is that you use the bare metal. So look just at the price. That's a big difference just in the way that we price our offering with bare metal. So it's fair in the sense that with Amazon you probably get a bit more, but it's still expensive. So what you get with our E5 2621 TV3 is that you get 12 cores, you get 64 gigabyte of RAM and you get 480, sorry, gigabyte of SSD. And it's bare metal, it's yours. So all in all, even if the comparison is not exactly a match, we think that it is fair. You can still go inside the spreadsheet and play with the corresponding matches. So you can change that and see how it affects the different prices. Whereas on the private cloud side, I have included exactly the same, a flavor that actually has got more VCPUs about the same amount of RAM, 4 gigabytes more of RAM. One important thing is I don't necessarily believe that it's a good idea to include storage prices in the compute side of things. If you're buying compute, you're buying CPU and RAM and if you're buying storage, you're buying storage. So I leave all the block storage costs if you need a root volume for your computing sense. That's a block storage cost that is coming from SEF down here. And talking about object storage, we are comparing with Amazon S3 using three replicas of your data, not a reduced replica. On the private cloud side, I'm comparing that with a Swift implementation that also has got three replicas, no erasure coding, just plain three replicas out of the box. Anything you would like to mention about the object storage for Internet? So yeah, we basically use a solidifier if you want to know storage. So this is exposed through Swift. So basically the performance should be comparable. If it's not, please call me. I guess the main difference is object storage is backed by SSDs, whereas on the private cloud side, that's backed by SATA drives and I imagine that on the Amazon side they're also using SATA drives, not SSDs. So I would expect your object storage to be faster to be honest. Which would be. On the block storage side, that's EBS volumes using SSDs. So on the open stack side, I'm comparing that with SEF running with SSDs. And if you want to look in the spreadsheet you have the number of replicas, how many node failures are acceptable, all those variables are there. And as in Cinder, but with SSD backing them. So let's explain what's the simple marketing website. So sorry you're seeing all the number, but we'll just focus on the part that was on the left. So that's basically what Amazon considers a simple marketing website. It is an eye traffic, static web page, static content kind of a website. So you get a load balancer, that's a little thingy up where. You get two web applications, basically servers, so it's basically PHP. And you get two terabytes of object storage and you also have one terabyte of bandwidth out. So if you look at the number we won't go into it too much, but you're already starting to see that we do have a slight advantage. So that's still a 50% different story for the public cloud. Surprisingly, as it is a simple configuration it doesn't make a lot of sense to have your own private cloud for this. But still if you want to know it's a bit less expensive than what you would get using Amazon. But there is one important factor we want to show you. If you break down that total cost from Amazon you will actually notice that storage and compute are just a small portion of it and that in this scenario there is a lot of data traffic. So the bandwidth, the outgoing traffic accounts for a good chunk of your costs in this scenario. And what is important is on the private cloud model you are deploying this private cloud in your data center. So I cannot make assumptions on your behalf on what agreements you have with your ISPs in terms of how much bandwidth will cost. So to be fair what I've done is I took the data transit costs from Amazon and applied that to the model. Right? So if we look at another example that's a more meaty example that's a large web application. So that's something that's doing less traffic but it is more complex inside the application itself. So there is a little balancer there is two web front what Amazon called web front so basically engineering instances probably. There is a two web application so it will be it's Ruby an example the example is based on Ruby there is one MySQLDB relational and one NoSQLDB and a bit of object storage. So there is a bit of everything and then again you can start to see that we are making a bit of a difference but now the private cloud starts to really make sense. So the third one is a mobile application. So for those that do mobile application I hope you know most of the processing should be done on the mobile application itself. So there is not much that the application does so it's basically the API layer so that would be your Ruby, PHP, Python layer tool balancer. There is a lot of object storage of course that you can upload or whatever they interact and create if you want to store it. There is a two relational database MySQL in the Amazon template there's two non-relational NoSQL database and there's a huge load of traffic quantified per month. Now you're really starting to see the difference if you look at the number that's almost 40-50% difference for the public cloud, the open stack public cloud. Of course if you go the private cloud way even though you have to manage yourself, it really does make a difference. And this is how that specific template, the breakdown of course and you can see that in this case you have more compute, more storage than bandwidth is not necessarily the significant portion and therefore your private cloud costs start showing the difference there. And there's something also that you need to notice still the bandwidth is a huge thing that people neglect when they go and look at the price you will pay a huge amount for the bandwidth that Amazon serves you out it's really important to take that into account most of the open stack public cloud providers do have some free tier Amazon does have two, which is really low honestly for us at least some players does have one and that makes a difference at the end of your months and probably year. There's also something else that you need to think about especially those of you that use a lot of the bell and whistles so the services, the RMR the MR sorry DynamoDB and so on and so forth is the more you use it and the more you pass the free tier threshold and the more you are going to pay for the API request so the get you get a huge amount of get for a very small sum but the put to the list it adapts very quickly so again when you do the equivalent and the estimation do not neglect that. So I guess the takeaway here is don't ignore the costs that will come in relation to API requests and on the private cloud side what I've done is if you're interacting with object storage or put to object storage I assume that that control plane where you're running your Swift API are actually an overhead of having that storage so in the price per gigabyte of the object storage there is the API request baked in and it's up to you to decide if that control plane is big enough for your needs and scaling that out. And on the public cloud side you basically get it for free unless you really are a crazy API based consumer we're going to get you the request for free no public cloud provider that will make you pay for the open cycle APIs. Unless again you have so much calls then it starts to show in the graphs but for now you should be good. And by the way I'll mention something in relation to the Catalyst public cloud we are not charging for object storage API request because it's not possible in open stack but that's actually a deliberate decision right now because one of the complaints we've heard from customers using Amazon AWS is that the pricing model is so complicated that they cannot figure out how much they will pay for something so we try to simplify that model where they can tell us how much storage they're going to use and we made a few assumptions on their behalf and on average those assumptions are correct some customers will use less API requests some customers will use more and on average we keep a track of that and they are correct. So another template is a different one is disaster recovery so basically it goes to say that you should have disaster recovery if you don't that's something that you should look at it implies of course that you have multiple regions so the template is very simple in the Amazon monthly calculator it's basically 2 dBs for each region some traffic in between the region and object storage ok nothing much there you can see that as it's more a compute based workload the difference is not much between the public cloud still around 20 10, 20% but then the private cloud kills us for sure and last but not least big data so the Amazon template is basically one job task node and 12 worker nodes each of those nodes are big giant compute basically that's really the end of the line for Amazon and that's our best bare metal on our side so there's a bit of object storage because you need to start those results somewhere and of course you need to get them out unless you just want to leave them there so that's where honestly it makes a huge difference and people do neglect the difference that both the bandwidth and the compute can make but at that level the difference is going to make a huge end result so you can see it's a 50% per year which probably you should really think about if you go and use Amazon for that and private cloud I don't want to talk about it it's depressing for me look this is the bottom line no one is going to market with prices like this but it's what you can achieve internally so I guess the conclusion here is can open stack beat Amazon AWS in price and the answer is when you're talking about compute, block storage object storage, networking that common layer that we have yes it's definitely possible and if you've taken the time to actually work the mess look at the model you will find out that doing your own private cloud will bring substantial financial benefits to your business but the important thing is it only makes sense at a certain scale right go back to that TCO analysis that I was doing for a large private cloud that costs you maybe 400,000 per month and that includes everything power, cooling, rec space, the servers if you only have one virtual machine running in the private cloud now you have one virtual machine that costs $450,000 and that's not very good or if you can't afford it do it, it's pretty cool so there is a place for it right, if cloud computing is an important capability for your organization, if it's core business then it does make sense for you to build an engineering team and pursue deploying your own private cloud or if it's not necessarily core business but it's important for you IT strategy maybe partnering with a company that will help you to deploy your private cloud and maybe manage that remotely for you so that you don't have to build that big engineering team yourself will make sense and also what we see a lot too is a mix of both so for some workload you're going to use a public open stack public cloud for the others you're going to make your own private cloud again it's all about provider, we do have economy of scale, especially in bandwidth and some of the hardware and so on and so forth that you cannot have if you do your own private cloud so negotiating with super macro or intel is something different when you buy you know, you buy them by the boatload or you just want 200 of them so it's always I think important for you to think about what should be my private cloud workload and what should be my public cloud workload especially if you use open stack it's going to be the same APIs so maybe the IP the endpoint will change but that's all, it should be just a configuration you just need to talk to your developers so yeah, it's possible, look at it if you have questions or comments we'll have five minutes now at the end of the session but you've got our contacts there feel free to get in touch and discuss how the model was done we're deliberately sharing the spreadsheet, the model so you guys can have a look at it and see how we've done the comparison at your own time and potentially contribute to that because I think it's important for us as a community to explain to people this is what open stack looks like on paper when we add up the numbers and finally, if you want to get access to this presentation get access to all the spreadsheets and the calculation play with it, please that's the one you want so that's where you're going to find all the information, the model that he uses for the TCO of the private cloud the simple calculation that we use for the public cloud, the different templates that everything is there everything is available to you you can change it, you can tweak it you can make it your own you can add complexity to it we wish you would do it because we really do think that's something that we should talk about and make it visible it's okay, sometimes we're going to compete with the Amazon or between each other and we're going to be good and that's healthy I think but we should have this conversation thank you if you guys have questions please use the microphone please we have five minutes for questions yeah, go ahead how many employees do you foresee for an in-house open stack installation because yesterday I went to a presentation from Cisco saying we need at least eight full-time employees to go for production and the open stack engineers are twice as expensive as we are depends where maybe, right we are a public cloud provider in New Zealand and I can tell you when we started with our public cloud we didn't have eight employees full-time operating the cloud and yet our customers were getting some pretty good service levels from us even though we didn't have eight people nowadays we may have more than eight people you grow right the important thing is on that total cost of ownership there are some variables in there one of the variables is how much an open stack engineer will cost per hour the second thing is when you're adding up everything what I've done is to propose three scenarios one is you have a small private cloud eight by five and in this case I'm saying you have two people if you consider this is not enough for you and you need eight people for an eight by five small cloud with three compute nodes you're welcome to change the numbers and see what happens there so there are three scenarios the small private cloud, eight by five a medium private cloud and then the one I was comparing with the large private cloud 24 by 7 where you have 300 compute nodes 40 block storage nodes 50 object storage nodes and the price point of course the cost will vary depending on the size of your private cloud so in a lot of the cases your costs were like two, three, four, five times private open stack cloud so why can't you bring your costs to 50% more than open stack sort of more consistently so what's stopping you from bringing down your costs closer to the cost of a private open stack cloud so you're asking that to me as a public cloud provider or you're asking that to me as a private well if you want to compete with private cloud in this talk you're making the argument that at a certain scale and which is not actually a really large scale just a few hundred nodes you are better off going for private cloud so why can't you drive your costs down since you do have some economies of scale I presume again can I address it initially I think the important point to observe here is if you've done the mess a lot of people say that public cloud computing is a race to the bottom and maybe it is but if you've done the mess you see that the bottom is very far away from where it is right now and at the moment it's good business to be doing right so the margins in the public cloud business are reasonable and some companies have a very big marketing sales team and overheads that you wouldn't have in your private cloud in order to sell their public clouds they will do events with 10 million people in Vegas they will you know have a lot of solution architects that come to your place to show you how you can use more and more their services and that has got a cost and that cost overhead is applied and in India their margins are not as big as you know this so remember that in a private cloud environment you're going for cost recovery you're not going for the market to sell something so that's where basically the difference is any other question I'm not a native English speaker but I think there is a word for that and the translation from Portuguese would be something called scheming the market scheming something like that yeah I think that's a marketing technique isn't it enough about that the microphone if you please I'm sorry because it's recorded give me your question and we'll repeat ok cool one question thank you very much