 Hey everyone, and welcome to the Cube's presentation of the AWS Startup Showcase, Analytics and Cost Optimization. This is season three episode two of the ongoing series covering exciting startups from the AWS ecosystem. I'm your host, Lisa Martin. And today we are excited to be joined by Cube alum, Ash Munshi, the CEO of Pepperdata, and Kirk Lewis is here as well, Senior Solution Architect at Pepperdata. They're here to talk about the benefits of real-time automated cloud cost optimization. Ash, Kirk, thanks so much for joining us today. Great to have you back. Our pleasure. Thank you. All right. Ash, tell us a little bit about Pepperdata and specifically some of the problems that it was founded to solve. Sure. So Pepperdata was founded, I guess about a decade ago now. So we're sort of an old startup. It was founded by a couple of people out of Yahoo who had actually played a pivotal role in developing MapReduce and the applications of MapReduce for large-scale computation. And as some of our audience may know, MapReduce then evolved, and then we wound up getting Spark, and Spark is now sort of the infrastructure for being able to do a lot of large-scale distributed computation, batch computation. Originally, the company's intent was to make MapReduce accessible in a number of different ways. So at Yahoo, we knew how to scale it. We knew exactly how to implement it. We knew all the tricks that were there to be able to do that. Other people found it really difficult to make that happen. So what we did was Pepperdata provided the metrics that were required to run something efficiently, how to monitor it, how to actually make sure things were running properly, and then be able to do that for near mortals and let enterprises make that work. The company raised a bunch of venture capital in that day and age, and big data was the cool thing, like AI is the cool thing now. So we were the AI of that day. And raised a bunch of money. We started getting some good traction, and we had two components to our product offering. One was detailed understanding or visibility about what was going on. And at the same time, the ability to actually tune it and make it work. In those days, Cloud was still cloudy. We were still doing things on-premise. Lots of people were trying to implement this stuff on-premise. And so we focused very much on helping customers on-premise. The kind of customers that we focused on, because this was not for the faint of heart, were people who were spending millions and tens of millions of dollars on-premise building large-scale infrastructure to make it all work. And they needed two things. They need to know how to use it. And the second thing they needed was how to reduce cost on it. The how to use it part was the observability part that we built out. So collecting lots and lots of metrics to be able to show them what exactly was going on. And as a result, the company built a bunch of extremely good IP around large-scale collection of time series data and extended an open-source time series database to be able to do things that were massive scale. To give you an idea, the kind of data that we collected would be the equivalent of what Uber collects on all of its cars and all the movements of its cars today. And we were doing that roughly 10 years ago. Wow, so tremendous volume of data. Incredible volume. Over the 10 years, over the decade, since pepper data was founded, what are some of the concerns that you're seeing today from customers where cloud cost optimization is concerned? So, you know, we had this big transition. People went from on-premise to cloud. And at the beginning, everybody thought cloud was free. Why not? I mean, it's not part of your fixed infrastructure cost. It's part of my budget. Don't worry about it. Let anybody go and do cloud. Well, it didn't quite work that way because the clouds turned into a rainy day. And it wound up being that, you know, everybody was suddenly, oh my God, this is costing us a lot more than we expected. And the CIO suddenly got involved again in the discussion. And the CFO suddenly said, oh my God, everybody's budgets are going out of rummock. Let's go control the costs. It turns out that the stuff that we did on-prem for controlling costs was applicable to the stuff on the cloud as well. So in particular, one of the things that happened on the cloud was we took infrastructure that was happening in data centers and moved it into the cloud. So the big data stuff became essentially EMR. EMR is essentially MapReduce and Spark that we did on the old days in physical infrastructure now cloudified. So the same tricks that we were playing before on local infrastructure we could do, but we have one extra interesting dimension. And that dimension was the fact that we weren't constrained by the number of nodes that we had physically. We could add more. We could actually have this flexibility, this dynamic ability to grow and shrink capacity, if you will. And that added an interesting dimension to our optimization that we were already doing. So what we did was, we took what we did on-prem, which was extremely successful. And by the way, used by the world's largest companies, and I really do mean the world's largest companies, I think Fortune 10. And then we moved that into the cloud and then added that additional dimension of the scalability or flexibility that you could wind up getting. Sort of the dynamic nature of the cloud. We married the two together and we realized that we had a really good offering for people working on at least big data stack on the cloud. Then what we realized is the market was embracing Kubernetes and was also embracing microservices. And traditionally, the market for big data, which is sort of more batch oriented, was very different than the market for Kubernetes. There were two different places. There weren't, even the people running these things weren't the same. And with Kubernetes, we started seeing more and more of an intersection, more and more people saying, hey, why do we have two separate islands? Let's try to put everything together. So there has been a trend afoot, which basically says, let's take all this big data stuff, all this microservices stuff, and unify it under Kubernetes. So the logical thing for us to do is to say, hey, if that's happening, let's take a look at the optimizations that we do on the sort of the big data side and also move those into the microservices side. And that's exactly what we've done. We're about to introduce a set of products that essentially take our time-tested, proven, scalable capabilities with some of the largest companies in the world, move that into the Kubernetes space and offer the same kind of optimization, if you will. I think the distinction between us and sort of the other people that are out there is that while we provide recommendations for if you want to be able to use them and do them manually, what we recognize is that there's a lot of stuff happening on these machines. If there's a lot of stuff happening on these machines, it's impossible to do one-off tuning. And the other thing is that even if you do one-off tuning, you have this coupling thing that happens, right? I tune something, it may affect you, you tune something, it may affect me because we might be sharing resources. As a result, it's hard to actually get your hands on everything without having it happen automatically. And that's something that we do from a natural perspective because when we were dealing with large distributed systems, think about the world's largest spark jobs. You have these executors going all over the place, taking information, moving stuff around, taking resources, it's a big mess. God bless you if you think you're going to actually be able to do this by hand. We did this all automatically. Well, it turns out the same kinds of things happen in the other parts of the space as well. So that same technology we've now extended to make it so that you don't have to touch it, we actually recognize the inefficiencies that are happening dynamically, take corrective action dynamically, move things around in the right way, and then be able to deal with the scaling part of the auto scalers in an appropriate way so that not only are we taking advantage of the hardware that's already allocated, but also the hardware that you're going to allocate that you're growing into so that both of those two things are solved dynamically in real time. Got it. That means a big advantage of us. Oh, sorry, Ash. You mentioned pepper data being used by some of the world's largest companies. You mentioned Fortune 10, but I imagine that you see the imperative for cost optimization across companies of every size. Is that right? That's correct. We absolutely see it. Look, if I look, if I sort of go back a year, maybe a year and a half, even startups had plenty of money because VCs were flushed and they were just writing checks. So they didn't really care about that. What they cared about was market dominance. Let's get out there. Let's go do things. Bigger companies were starting to see issues, but financing was also relatively free for them. So there wasn't much of an emphasis on it. All of a sudden in the last year, year and a half, as we see recession coming up, bell tightening happening, we've got all kinds of people. I would say everybody across the spectrum suddenly says, hey, I can help you save money. Hey, I can help you save money. So what was interesting is we had a fantastic value proposition, but in some sense, we were the only people kind of talking in this tunnel. Because if people don't care about saving money, well, you can talk all you want. Nobody listens. All of a sudden, we were getting a lot of listens, which is great. So we're getting a lot of interest in being able to save money. I think business is coming back to what normal business should be, where you don't do everything because of cost optimization, but you make sure that you're actually managing your business in a way that it's not a runaway cost structure. So that's happening. That discipline is very welcome by us, and our mess is therefore lands on very receptive ears, and we are in fact seeing from the smallest companies to the largest companies a great desire to be able to go and save money. If you look at the whole saving the money spectrum, the large companies have incredible purchasing power. So what they can do is they can actually go squeeze. They can say, hey, you're offering me this on spot, but I wanted actually on my reserved instances. I want my pricing this way, et cetera. So they can structure their fixed cost on instances in a very, very nice way. Startups don't have the ability to do that. Some of the other people in our showcase here are actually offering effectively that type of service where you can actually arbitrage sort of pricing on the reserve instances and on spot and things of that sort. So you can lower your instance costs to be able to do that. Naturally, that happens automatically inside large companies where we play. But in both of those two cases, even if you've done that, you still have the ability to now take what you've got and run it efficiently. Running it efficiently is extraordinarily important. Either side of the coin you're on. And we make that happen seamlessly, automatically. So I would say, to a great extent, we're sort of complimentary to some of the other things that we're going to be talking about. Do whatever you can. Find the best instances. Do the stuff that you want and then turn us on. We're complimentary. We will give you extra savings above whatever else that has been contracted. Extra savings sounds good. I know we want to get Kirk in here for a demo, but just tell me really quickly about the pepper data capacity optimizer solution that's for Amazon EMR and Amazon EKS. What makes that unique? And then we'll throw it to Kirk for the demo. Certainly. So the core technology that does the cost savings is in fact capacity optimizer. It essentially has two components. One component is to deal with resources on the machine that you already are using. What we realize is that there is a gap between what is asked for in terms of resources and what is actually used dynamically. And as a result, that gap allows us to actually put more capacity effectively or create virtual capacity on a given node to be able to do that. That's our capacity optimizer. That same kind of concept happens when you try to scale. Well, when you scale, what happens is you suddenly say, I get five more instances or I get 10 more instances. Well, maybe you don't want five or 10 more instances. Maybe you want two. What's the right number to pick for you to be able to go and do so that you're incrementally growing that because your cost is a function of how many nodes you've got active at any given point in time. And so the fewer the nodes, the more packed and more efficient they are, the lower the cost is going to be. And we operate on both those dimensions. And that loosely speaking is our capacity optimizer. Kirk, keep me honest. Absolutely. And Kirk, you've got a demo here that you're going to share with us. So tell us what you're about to share. Certainly. So as Ash spoke of the components of pepper data, the way it operates is in two facets as well. We've got the components that operate on cluster to do the optimization. And then we've got a dashboard that shows you what's happening and shows you how much you can save and how much you are saving. So to start, I've got an example of our dashboard showing a plain vanilla cluster that is not being optimized. You can see components where it says capacity optimizer is not enabled. And I've got a nightly workload that runs and we're using 82 instance hours to run this workload. And this is what you pay for on your cloud bill that's going to say instance hours. So we're speaking the same language as our customers so they can see in plain English, what are we talking about here? You spent 82 instance hours to run this workload. And your core usage, meaning the compute that you spent was only spent at an average of 30%, it was only 30% efficient. And at peak, you were 77% efficient. And for clarity's sake, you're paying for 100% of the resources that you deploy. So this 30% and 77% are suboptimal. You'd like to be at 100% in a theoretical space, but you want to get as high as you can get. Memory is the same kind of story here where it's an average of 35 and you peak out at 91. And this comes from that lower level that Ash talked about where in, yes, you can have spot instances. Yes, you can get these boxes for as little costs as possible. But the applications that we're talking about here, especially for things like Spark and microservices, there's always that component where people will size what they need for that peak. And we know that there's always a usage curve there. So this chart that we have of instance hours is showing the auto scaler launching these nodes as soon as someone asks for resources. So I go from three instances up to 50 instances really quickly here. And I hover there and I do auto scale up and down as that demand sort of fluctuates. But the problem is there's always going to be that delta between what you've asked for and what you've used which quantifies the waste. So when you use capacity optimizer what does that look like? What changes? So now I've got an example of a cluster that is running capacity optimizer and you can see the full fidelity of what we're showing you here. Core hours saved. We've increased utilization across these nodes on average by about 50% for cores and about 40% for memory. The number of containers the units of work you were able to run was increased by about 30% and over 100% uplift at peak meaning there was over 50% waste that we found inside the running applications. And the big number here is we only use 35 instance hours to run that same exact workload. So there was an over 50% increase in the amount of work you got done and how we pulled that off if you look at this line here not only is your number of instances not increasing to 50 but it increases out a much less aggressive rate. And to Asha's point about how we augment the autoscaler what we did here is for every node that is launched and is running we augment the autoscaler in a way that says don't launch any more nodes until capacity optimizer packs that node to about 80% which is configurable but we shoot for about 80% efficiency at the node level. So when you do that the autoscaler doesn't have to run as many nodes to do the same work. So we can see as a product of that our average usage at the node level goes up peak goes up to 90 and 99% here. So really we've solved the problem of you know developers have to size those boxes that they're launching at peak but we know that there's always going to be some waste there but by doing things dynamically and sampling those applications every few seconds we don't have to do things like model the workload or restart the applications or retune applications. This all just happens on the fly and developers of course like it because we're not touching their code and the DevOps guys like it because we're showing them everything that's happening there's no behind the scenes magic here we're showing you what we're doing and how much you're earning or how much you're saving as a result. So that is how things operate at the node level at the autoscaler level and then at the cluster level and what we're doing now is we're serving a different audience in the realm of Phenops and the executives who are not so much concerned about the individual clusters they want to know what's the overall account look like what's the enterprise look like so this new UI treatment that we've done on top of capacity optimizer is showing me for example over the last 30 days what's the total compute cost across my organization I may have multiple cloud accounts and I rolled that all up here to about $170,000 compute cost I can see which clusters or how many clusters are managed by pepper data and we've got a couple that are not the potential savings here is savings that have not been realized yet and when I look at this next row I've got the total compute cost and then this breakdown of realized savings versus potential savings speaks to what is capacity optimizer doing and what could it be doing for you and from there when you see something like potential savings the next natural question is okay well how do I go realize those savings so we show you your top 10 clusters by compute cost and you can see the most costly cluster doesn't even have pepper data installed so this is an example where because these organizations are so dynamic someone may have spun up a cluster and not installed pepper data so we're showing you that as an opportunity even though pepper data isn't installed there yet and then on the optimized clusters I'll take for example this one is being optimized at a low level and when I click into it I can see this control plane for capacity optimizer where if I want to optimize more aggressively I can do so and the workload here and the metadata is a batch workload we know that batch workloads are sometimes not as sensitive to like transactional workloads in terms of the SLAs so I can move this up to high I can introduce the auto scaling optimization and I can apply this change and I can do that all from this panel without having to log into these machines and make changes so it's really great on the operational side and then my potential savings are from this point on because we are dynamic in real time we'll start to transfer over to this realized savings bucket so all of this happens without developers having to change anything without the DevOps guys having to get involved we do have support for change control systems on the back end so that things are also tracked but what we're doing here is we're serving that audience that wants to know at a high level you know how am I how am I doing in terms of efficiency and when I look at the product of this across the board there was that cluster that didn't have pepper data installed and I'll show an example here of a install it's very simple in that in this UI we're giving you what you need to cut and paste to install pepper data and you can see it's only three steps so there's not a drawn out install and as we mentioned once you're installed and you click apply on the button you start to realize those savings so this all rolls up from a simplicity standpoint to the FinOps and executive audiences now have this total roll up of realized savings across all of the accounts all of the different business units and then you get sort of a scorecard of which accounts are optimized or not optimized in a percentage wise basis and then you can see how their clusters are doing within their accounts so you get this entire view of the clusters rolled up to the accounts rolled up to the business units and you can also do something about it from here so with that it's really the entire spectrum from the low level to the high level to show how pepper data capacity optimizer is impacting and quantifying the ability to see and optimize what's being wasted in these environments Wow, it's a great visibility you can see core usage identifying waste with capacity optimizer I saw you mentioned core hours saved more than a 50% increase in work done and you get that whole holistic picture of what the enterprise looks like guys thank you so much for coming on the program talking to us about pepper data capacity optimizer and the huge impact that it's enabling customers to make we really appreciate your time and your insights Thank you and we want to thank you for watching and say keep it right here on the cube for more action your leader in tech coverage