 Hello everyone, I'm John Furrier, host of theCUBE, and welcome to this special presentation called Savings in the Cloud. Only pay for what you need with AWS Cost Optimization. We have Rick Oaks here who's the Senior Manager of Cloud Optimization at AWS. 90 year veteran over a decade in the trenches making it happen. Now it's prime time for optimization, certainly as companies grow to next gen and the economy the way it is. People want to squeeze every efficiency out of it and also make sure they're putting their money in the right places for growth. Big discussions around what is repatriation which doesn't really exist as we were talking before we came on camera. But really it's all about being smart and saving some of the clouds. A lot of good ways to do it. Welcome to this special presentation. Thanks John, I really appreciate the opportunity to talk about optimization. It's near and dear to my heart. I've been working on it for a long time as you mentioned. And it's been quite the journey too to see the early days of optimization where it was really just about like, doing very simple things like turning things off nights and weekends and to see the evolution of where we've come has been pretty special to be here. Well, we're psyched to do this series. It's super important. It's one of the top conversations I'm getting across the board, not just cloud, normal cloud native cloud computing but we've seen the AI side. A lot of questions around how to deploy, cost-op cost structures, how do I architect all part of the cloud architecture that's going on that we've been talking about infrastructure as code but it's getting better and better. And then next, we've got an event coming up on July 25th, we'll be in New York City for a panel which should be awesome around an AWS summit talking more about this in person in New York City. And this really kind of brings up the question especially big Vintec in New York. So, they're looking at the cost too. What's that big aha moment? Because again, you've seen the historical perspective, Rick, around what it was and where it was and where it is today and where it's going. Why is optimization more important now than ever before? Yeah, I think cloud has really excited a lot of people with the potential and the opportunity for things like designing new capabilities or launching new technology or breaking into new sectors, which is awesome. And oftentimes optimization is like this after effect where now we're looking at our cloud bills, we're realizing the cost associated with trying to invent and create new resource types and solutions. And that's very powerful. But we still have to have responsibility with how we invent. And that's not always easy to do. If it's not thought of in the original invention and creation of software and solutions, that follow-up can be very devastating or detrimental to your backlog. If you even think about like how much time does an average engineering team spend on code cleanup and repository cleanup and some of that stuff versus building new features, it's kind of never fun to sit there and spend a lot of your backlog just cleaning up. And so we have to really change the culture. We have to change the culture of how we build things, how we manage and operate cloud environments and resources. And I think it's exciting because we really have an opportunity to bake into how we build things, a new responsibility, not just for the resources we spend, but for the planet, the impact on the world around us as we figure out how to use resources efficiently. So it's an important, especially with like the economic challenges where organizations are looking to just rationalize the spend they have. It's not just about saving money, it's about is the money I'm spending on my cloud bill driving business value? And then the more we can match up those two ideas that business value comes from spending in the cloud and the value we receive and the solutions we deliver, we have this opportunity to make that ratio really something of a powerful story for cloud use. You know, the operating model of the cloud is, you know, pay as you go, pay by the drink, whatever you want to call it. We were talking about this all the time at re-invent since 2013 when theCUBE's been there, you know, it's been quoted, it's now the standard, you know, all the undifferentiated heavy lifting or now with AI and some of this automation, it's differentiated heavy lifting. So like the custodial aspect of cleaning up and doing code reviews and fixing things can be monotonous and toil. So the operating model of tuning optimization is very much part of the cloud now, right? And now as cloud goes to on-premises and edge, the cloud-native operating model of cloud is basically a distributed system. And so now you have primary cloud operations as the center point of action. This has become more important because you're always tuning and making sure things are right, not just from a cost perspective, it's an operating model. So it's not, it can't be like a bolt-on. It's not like, oh yeah, we're done, go clean that up or, you know, go do this, go check on this. It's kind of a rock fetch. It moves it from like, okay, move a rock from here to here and you kind of, it can be mundane versus a standard operating model. And you don't, it doesn't need to be either like the idea of extracting value out of cloud by being more elastic with how you treat your resources is very powerful, right? So you don't necessarily need to be in a situation where you're constantly sweeping up and cleaning up your resources. If you're really taking optimization and moving from like, hey, one-time cleanup activities, did I size things correctly? Did I schedule turn offs of my resources nights and weekends? And pivoting more towards like, how can we design systems or even take legacy systems that are still very valuable and apply elasticity concepts to them upfront? So that really turns into how are we taking elasticity as a cultural and architectural practice instead of as a cleanup custodian type activity where we're just kind of going through and, you know, doing the work that nobody wants to do of validating if all these resources are still valuable for us, right? We don't want to do that. As the old expression, it's a feature, not a bug or it should be primary, not just, you know, secondary. And let's get into it, because I think this is important. How, and I think it's not a lot of debate. Optimization is super important. I think people, it's front and center. I think the questions now are, what's the cultural shift to the new model? How do you get started with optimization? Where are we today? How does someone get involved? How do they get their arms around it? So if I'm coming into it for the first time and getting my arms around it, what do I look like if I'm in the middle of it? What's the best productivity path for me? What's the best practices? Take us through optimizations playbooks here. So we can think of optimization in four major methods of saving money in the cloud without hurting performance. The first is sizing things correctly. So we take a look at all of the resources you have in your environment. Look at CPU, memory, disk network, and we validate if we haven't revisited the size of these workloads and maybe months or years. Are there new instance families that have come out? Are there new CPU models that have come out? We can switch to that improve the price to performance ratio of your environments, right? And this, by the way, this applies to legacy as well. You don't have to rewrite your applications to pick new instance sizes. It's just a quick reboot. So this is a really important part of optimization. The next is suspension. So the suspension pillar is really where you have an ability to turn things off nights and weekends, you know, non-production workloads, environments where there's really not anybody in the office or working what have you on those resources. So they're just not needed. We can get away from the five nines culture of non-production and really just hone in on, you know, turning the lights off when you're not through. The next pillar is deleting unused resources. Oftentimes like the first for a for companies as they get into optimization, just going through and auditing and validating the resources they have and just saying, Hey, is this in use? When was the last time this volume was attached to a instance? When was the last time this IP address was used? So on and so forth and just kind of more custodian and then the last one, probably the highest ROI for the effort is purchase optimization and that's savings plans, reservations, purchasing discounts and layering them across your environment in a way that's really reduces your on demand spend for any environments, you know, we're gonna be long lasting and consistent utilization, covering them with discount management is really important. This is another really good starting spot for organizations that are not ready to really tackle engineering specific optimizations and it's more maybe like finance, FinOps type things where central finance organizations can really look at the total spend they have on their bill, optimize that just by committing for one or three years of spend to these resources which allows you to really get a more consistent bill over time and protect the costs to that. Now the real trick, John is when you combine multiple of these pillars together, that's the real trick because when you're sizing and then discounting together, they multiply and I love watching compound interest at work. I don't know about you but that's a fun one for me. Well, this is a, you know, you were talking about Corey Quinn before he came on, the cloud economist. It's an economic equation and then the low hanging fruit, I get that if you know you're using a steady state resource, you have pre-buy, makes total sense, on the integration of the pillars I want to ask you where does the cloud infrastructure costs begin to get complicated and difficult so for someone who wants to go that step what are they looking at? What kind of level of difficulties or a degree of difficulty? Where's the complexity or where do they start? Where should they go first? Can you take us through that next step of integrating in? What are some of the ways you can turn those knobs? That's a great question and depending on the company I'm talking to or the maturity that they're at, the starting stones might be a little bit different because some companies might have very strong engineering practices and the ability to validate idle and waste very quickly. Whereas other organizations have a lot of investment and capability on their finance team and they can buy savings plans or reservations a little faster. And so depending on the maturity and the comfort level of the company they might start with a different pillar. And so there's a couple of really good lower effort, low analysis starting spots, whether that's idle cleanup or savings plans and reservations. The complexity really comes into play when you're thinking about doing multiple at the same time. For example, if you want to buy a bunch of reservations or savings plans for a very specific region or whatever you might run into a situation where in a non-production region you wanna go turn all of those workloads off nights and weekends. And if you say you turn them off, I don't know 16 hours out of 24 hours you're saving two thirds of your total cost per day which is pretty significant. The discount of a savings plan might be a lower discount than the savings you get from turning it off. So you have to sort of pick and choose and look at, well, maybe I apply savings plans to all my production workloads which I know are gonna be online 24 seven but for my non-production I'm actually gonna go with suspension and turning things off nice and weekend. Right, and so mixing and matching these can get more complicated because we have to figure out what's the compatibility between the pillars. But that's also the benefit of, for example, right sizing which takes a lot more engineering effort and investment implementation and automation including but when you get it right and you right size your environments you reclaim a ton of that capacity that was left as over provisioned headroom which is a very common experience and then you apply discounts on top of it you get this benefit of having a total cost reduction of right sizing by 30, 40% and then a total cost reduction from savings plans and reservations of another 30, 40% then you're running a cloud environment really with the cost associated with the end resulting bill that you would prefer to have, right? And now you're really getting the benefit of cloud that you're scaling, you're elastic you're paying only for what you need you're getting these cloud value promises that is the whole point of how cloud was built. It's definitely totally it's in line with the key flywheel for cloud for sure. I have to ask you if you don't mind explaining what right sizing is I know a lot of people get confused refactoring applications that's the term to use replatforming that's been used right sizing is a specific definition around this piece here. When you say right sizing this comes up a lot by the way when people talk about repatriation which is really kind of not happening unless someone really has no interest in cloud at all but that's pretty much like really small percentage of anybody. But if you're in the cloud you're right sizing because you're spanning your footprint, there's more moving parts at least that's my kind of view how do you guys define right sizing for folks watching and what does it mean? That's a great question. So right sizing if we look at like individual instances you have an environment say you have an AWS environment with a lot of EC2 instances. One of the best features in public cloud in general is the ability to change the instance size you have. So say you have 10 instances their web servers you provision them with 16 cores and 32 gigabytes of memory and over time you can look at those instances and say is what's CPU doing? What's memory doing? Am I using all of these instances? Can I get by with smaller instances? Or more modernly, hey this new instance family came out with CPUs that are quite a bit faster. They went from 2.2 gigahertz to 3.3 gigahertz in the latest instance family. Let me go take advantage of that. So right sizing is the ability to turn off that instance pick a new size turn it back on. So you have a reboot of a couple of minutes that instance comes back online and boom it's almost like you just totally renovated your data center and boom you're using the latest generation hardware. Incident upgrade. Over just a few minutes it's a massive upgrade and the prices might be slightly different but when you gain the benefits of the newer generation hardware you gain the 40, 50% performance boost on the newer CPU generation. You don't need to run 16 CPUs you can run eight CPUs and surprisingly enough it's so much faster because the individual core is so much faster that customers process the API call or the website loads faster with less cores. That's why I love DevOps Rick because this is the IT that is now formalized. It's not shadow IT anymore. We talked about that 13 years ago. This is real IT and it's not the old IT. It's the new IT. Right sizing is about upgrading for the app where you need it and you're seeing that right now with a lot of the AI applications. People want to put their workloads on the fastest stuff possible if that's what it needs. So the tuning, the right sizing, they'll go hand in hand, right? Is that the right way to think about it? I think so. So there's modernization. There's scaling and elasticity. There's also the legacy thinking of IT data center provisioning where you would order hardware for a three or five year lifespan, right? So three years for making sure that you're able to replace the components and everything but you always over provision when you order new hardware because you don't want two and a half years into the life cycle of your hardware to run out of capacity. That's sort of devastating. And then you have to order new hardware at a completely new cost. And so you're in a situation where it costs you a few hundred bucks to add some more memory to your hardware that you're gonna order. And so why not? It's a cheap insurance policy but it means you're running very low average utilization. And then you have a bunch of environments, development, test, performance environments, integration test environments, user agreement environments. So you have all of these environments in your data center and then you lift and shift them to cloud and at any given time, most of them are not in use. In a data center, that's okay because it's a sunk cost. It's a capex, right? Now when you move that to cloud and now you're paying by the hour, you all of a sudden have this ability to turn those environments off. You do not need to pay for them when nobody's using them. And now we can use the resources more flexibly but they're still over provision. So we still have this problem because we got used to over provisioning our hardware and our instance sizes on-prem, we copied that behavior forward into the public cloud. We no longer need to do that. We no longer need to have eight extra CPUs for every instance, just in case. If we know that utilization will go higher, we can then just change it to a bigger instance later, no problem. Just pick a maintenance window and go change the instance type. It's great. So we have to shed these on-premise best practices and really embrace the new elasticity concepts of changing the size. It's okay to change instance sizes. If you reboot this instance, it's not going to pop the capacitors on the motherboard like it used to 10, 15 years ago. It's okay, change is good, right? And so we have to break these on-prem mentalities. And I have to say, like, that's my own fault too. I used to provision environments all the time when I worked at Microsoft. I would order dozens or hundreds of instances as servers and get them installed and racked and cabled. And I always overordered because it was safer. Like, I didn't want to get called at two in the morning. They're like, hey, the environment fell over. What are you going to do about it? So, you know, as part of moving to cloud, I'd recognized and saw so many of those instances. Well, that was the right-sizing way to do it. I mean, that's the smart play in a model where you have to right-size over buy to give you that hedge. You want to have that protection and then monitor it to know when to get that next server, right? So, and the game is still the same. It's just an old habit you got to break, right? It's like, okay, it's still the same game, different playing fields. Yeah, and the best way to manage that has just sort of flipped on its head, right? Where you would order hardware because of the long lead times for hard ordering, you would overorder. And now it's just the opposite where you can just flip a switch and all of a sudden you have more capacity. So, you just don't need to overorder anymore. It means we can run the instance size much closer to the actual required capacity you need. That tends to be a pretty significant gap because we overordered a lot. And so, we really had this opportunity to run instance sizes that are more appropriate. That drives average CPU utilization up, right? And that's a nice KPI to use to measure your success here. What's your average CPU across your environments? Is it 4%? Is it 6%? Is it 15 or 20%, right? And it's cool to see companies go through that maturation. That's a great call-out, by the way, putting down those KPIs. That's what people want to look at. What's the scorecard look like? So, that's super valuable. Speaking of that kind of approach, you've got customers, you've got partners on the cloud. So, you got AWS, you guys are the cloud. You got ISVs, you have customers in the marketplace. They're buying and they're provisioning. What do customers do to get this consumed? What do they do to get involved and with partners? I mean, obviously, you've got a multi-vendor environment in the marketplace. It's got traction in terms of how people are procuring, their cloud consumption and contracts. What's the marketplace tie-in? Where's the consumption layer? What do companies do? Yeah, there's so many amazing partners that we have that help organizations use cloud appropriately and optimize size by discounts, all of these sorts of capabilities. You know, we're talking to a few of them on the 25th at the event. We've got a spot by NetApp and we also have AppTO and the cloudability team joining us. So, they've got some pretty awesome products and capabilities to help achieve that. We also have some first-party ones. We have Compute Optimizer, which gives you right-size recommendations. We have recommendations for savings plan purchases and RI purchases, which my team owns. So, we have some first-party solutions. The partner solutions are also incredibly compelling. We love working with our partners. We know that our partners help our customers have a very successful set of practices and environments on AWS to really support their business goals. So, we love working together with our partners. We love working with our customers as well directly. So, it's a very diverse ecosystem. There's a lot of products out there right now. Optimization is really popular. I love seeing it. I love working with all of the partners that come forward and say, hey, we've got great ideas to help customers and we love sitting down and working with them. It's a data problem. I love the data business now. It's so great everywhere. You guys got CloudWatch and Cloud Optimizer too on your side and there's a lot of white space opportunities and there's a lot of complimentary action in this area. Can you just explain that real quick differences in some of the interactions and the power dynamics between some of the products? Absolutely. So, CloudWatch is a tremendous sort of default product that you have when you provision EC2 instances and various other resources that provides a lot of utilization data. So, in Compute Optimizer, we use that CloudWatch data. What's really cool is if you turn on the CloudWatch agent and start gathering even deeper metrics on your instances with CloudWatch, like memory utilization, Compute Optimizer by default will pull those memory metrics in and that'll provide even more deep and rich recommendations for right sizing. By default, Compute Optimizer does not provide recommendations to downsize memory because we do not want to hurt your application performance at all. But once you do have memory metrics enabled via CloudWatch or one of our third party providers, APM providers that we built integration with, those recommendations become significantly higher cost savings because the number of instance types that it will evaluate moving to grows pretty tremendously. So, we see a very nice uptake and potential savings when memory data is loaded in and we love working with partners and various other customers to get that memory data, unlock the potential of right sizing more completely and seeing really big numbers delivered back to the customer for them to really spend that money on more strategic investments that they are really driving their Cloud value. I think savings in the Cloud should be more like innovation in the Cloud, it's a continuation of Cloud, it's one of the pillars in the flywheel. You know, you got to kind of keep sweeping the floors and automation is your friend too. You got data coming in at more telemetry, you got right sizing, you got people investing in more of the engine of Cloud and pay for what you need is really the whole principles of Cloud. So, you know, kick the old habits, get into the new and make it operational. Great message, really excellent masterclass, Rick. Thanks for coming on the whole series, saving in the Cloud is phenomenal. Thanks for coming on this special presentation, savings in the Cloud, only pay for what you need with it is cost optimization. We'll be in New York City on July 25th, the eve before it was summit in New York City for more conversations. I'm Jonas Stam, I'm John Furrier, host. Thanks for watching.