 Good evening Super Computing fans, and welcome back to the Mile High City. We're here at Super Computing 2023 in Denver, Colorado. My name is Savannah Peterson, joined with my wonderful enigmatic co-host, John Furrier. Good evening, John. Savannah, great to see you. What a great day to kick off. Got four days after this wall-to-wall coverage. And it's huge in here. It's buzzing in here. Yeah, Mile High. There's even a live band playing over there. I thought that was very cool. We got to pace ourselves. Mile High, altitude. Yeah, you're feeling it going up the stairs to get on set here. Yeah, we're alpha too. Elevated game. We're elevating our game. We are, our game is elevated. This is the highest our game will ever get. All right, we'll spare you. We'll spare the puns. I'm really excited to welcome back one of our favorite guests here on theCUBE, Johnny Dallas. Thank you so much for being here with us. It's great to be here. Good to see you guys again. We've had you all over. We've had you in the studio. We had you in Amsterdam earlier this year. What has you so excited for Super Computing here specifically in Denver? This is actually my first Super Computing. I know you guys are regular guests. It's my first Super Computing. Welcome. Definitely big, much bigger than CubeCon was even just last week. But yeah, we, I mean, we work with so many AI companies and all of the new developments going on of NVIDIA, the new GPUs coming out, the new multi-cloud tech for GPUs. Very, very excited. The huge trend here is this shows always about exoscale, exoflops, quantum. A lot of, you know, how many cores can you fit on a pin, as Dave Vellante would say. But this year, Johnny, I got to get your thoughts because as an entrepreneur, and we first talked on theCUBE two years ago. Two, but two years, was it two years ago? Yeah, maybe year and a half. Your vision, we talked about this idea of specialty clouds and tier two clouds and that you're going to start to see the super cloud kind of thing happen. Here it's a cloud collision with the semiconductors. You look at all the semiconductor players out there. They got great financial leverage, great performance, NVIDIA's got their cloud, they're enabling core readers of the world to be successful. So you start to see an enablement from the semiconductors where the new middlewares emerging, new service, managed services, and then ultimately that application is going to be some sort of specialty cloud. You and I talked about this. This is the collision here. HPC with ML at scale with the semiconductor chips is going to create opportunity. How do you see this from your perspective? You are an entrepreneur. You got opportunity recognition. What's your vision? What's your view of the world out there? What's on the landscape? What are you going after? Yeah, and you hit the nail on the head. We talked about this some time ago. We predicted specialized clouds at this very table some months ago, but it's very cool to see. I mean, we're here with CoreWeave, one of the partners that we work with quite a bit, the biggest GPU cloud, and they're very focused on just GPUs. They're not trying to become a hyperscaler like Amazon or Google Cloud. They're very focused on bringing the best GPU inference to the fore. And broadly, I'd say we're seeing that with all the different cloud providers right now. Everyone's figuring out what's their niche, what's their specialty, whether it's HPC and GPUs. One of our other cloud partners is really focused on the networking stack or this set of services. But we're seeing these specialized clouds growing, and I think that's very interesting. But it's almost the precursor of the other idea that we talked about of this idea of a meta cloud. Actually, you're not really going to do one cloud in the future. It doesn't really make sense to you. You see CoreWeave building the best GPU support. You see Linarakumai building the best network support. You see AWS doubling down on their own Graviton and Tranium instances. You should use all of them. You should have something that allows you to deploy to multiple clouds and pick and choose Alucard to use all the best services. And I think that's- There's a lot of innovation. Again, a lot of opportunities. People are going to see the opportunity will go after it, but I have to ask you, how would you define in the spirit of feeding the AI good prompts? Can't imagine how you want to do that, John. How would you describe what a specialty cloud is for the folks watching? Is it a category? Do you see as a category? Is it a functionality? What is a specialty cloud? And how is that related to what's happening in the cloud computing landscape today? Yeah, so I think there's some bare bones of what a cloud is that has to be there. Every cloud's going to provide some level of compute. Every cloud's going to provide some level of networking. There's some of those basics that are always going to be there. I think what makes a specialty cloud different from say a hyperscaler or a general purpose cloud is they know their niche and they're really doubling down on that and they don't necessarily care about the workloads that are not in that niche. Maybe they can support them, like we have customers who run APIs and web services and databases on CoreWeave but what they're best at is GPUs and that's really where they want to have their focus and I would bet on them to build the best GPU support that you could find across cloud providers where the networking stack, they're going to build it but it's not going to be the best in class and that's totally fine. So I'd say whether it's, for some teams it's technology, others it's going to be around use cases like OVH clouds very much leaning into cheap bandwidth, cheap networking and enabling a bunch of companies that require lots of bandwidth to be built on top of them. Whether it's technology or kind of business model, we have a specific use case that we serve really well and we know that we're going to be the best at that. Don't have to be everything. And customers are comfortable combining clouds. You mentioned that more than 50%, I believe it's over 60% of your customers are multi-cloud. Yeah, I think this is the other big trend right now. I mean, we were at KubeCon, CloudNativeCon last week. CNCF put out a great white paper earlier this year about platforms, this idea of have some control plane or some interface above the clouds that your developers can use to actually interface with the cloud infrastructure. It's almost a prerequisite of leveraging these specialized clouds. I wouldn't have a very hard time using 90% AWS and a little bit over here if I didn't have something abstracting my workload at a higher level. I think that's a really interesting point because there was what felt like a bit of a cloud war and now we realize it is going to take all the clouds in the sky to give us this beautiful starlight future that we have. You get to see, I want to keep talking a little bit about the customer observations you get to make without obviously revealing any secrets. You get to see a lot of people in a very similar race in a hot topic right now, artificial intelligence. Can you tell us a little bit about some of the trends or some of the remarkable patterns that you're noticing across your customer base? Yeah, I'd say first off, AI is, I mean, hit the nail on the head. AI is super hot right now. Everybody's building in that. We serve SMBs and mid-market companies looking to improve their developer efficiency, but also we really serve these startups that are going from zero to one, trying to figure out how do I set up some infrastructure or how do I get this kind of platform without spending all of my time setting up CICD pipelines? Yeah. And so especially in that second group, those early zero to one teams, everybody is starting in generative AI right now. It's funny, there's many waves within that, basically as open AI releases new APIs. You see a tranche of new companies startup and then they release another new API and some of them may go away. And a lot of heartbreak. There's quite a bit of heartbreak, I think, in San Francisco these weeks. I noticed NVIDIA DGX is enabling this market opportunity for Core Weaver. I noticed also they've got a $2.3 billion debt financing package. One of that's to support the NVIDIA costs. But talk about your relationship with Core Weaver because they're showing an interesting market power dynamic, which is they're basically selling GPU virtual machines. Yeah. Okay, which are well suited for AI workloads. Which in essence, they're this tier two cloud that's emerged to enable this market sector. Very similar to like the old white box days back in the 90s or the ODM model. You're starting to see that long tail, the new market grow, this tier two, enabling people like yourselves. Talk about this dynamic. What does it mean? What does this market mean? Do you agree that this is happening? Yeah. And what does it mean for people watching, trying to understand it? It certainly is very interesting to see the fact that it's not just the hyperscalers anymore, there's actual a path to competition with these tier two clouds and all these specialized clouds. I think the other interesting thing about that that you see is these tier two clouds or whatever the market is, these tier two. Super clouds. They tend to be newer and earlier and that gives them a bit of a head start on some of the tech. So one of the really interesting things about CoreWeave is elastic GPUs. One of the other interesting things is they actually are Kubernetes first cloud. In addition to their VM product, they have a product that's just a Kubernetes interface. That's how you deal with it. There's no cloud console on top of it. You're not going to log in and deal with IAM. You're really just going to go straight to Kubernetes and they're natively built on top of that. That is something that I haven't seen a lot of. Like AWS has EKS, Google Cloud has GKE. Everybody has a managed Kubernetes offering, but being Kubernetes first is a step function in the developer experience that we can see. Do you think we'll see more of that? I hope so. I'm biased because we deploy across Kubernetes clusters and we enable multi-cloud deployments quite a bit, but it's very interesting to see these new companies, whether it's Kubernetes interface or some of the hardware that Kory is building with, they can be a step ahead of the incumbents and when the markets align, you see these explosions of innovation where they can do so much and enable so many use cases. I love that you just said that because it brings me to my next question and something I'm as curious as I am excited about. Speaking of explosion of productivity and capability, you mentioned to me earlier that you've noticed teams are varying different sizes on the engineering side in the AI space right now. Tell me a little bit more. Yeah, so I think, you know, we mentioned, we serve a lot of these zero to one startups, getting into AI, that's where everyone's starting right now and because it's such a nascent space, I think you never really know what's going to work or sometimes you'll see these extraordinary successes. I mean, we did a case study with the CoreWeave earlier this year where we had a company that was building an AI API and they accidentally had a tweet go viral because Elon Musk retweeted it and they went from basically zero GPU usage to suddenly needing 1,000 GPUs on demand and that is an insane scale up. It literally zeroed to 1,000 in five seconds. The power of a post? Power of a post and power of CoreWeave in being able to enable it. So we see that and it's funny. I mean. That's exciting. It's so exciting. Yeah. That wouldn't have been possible without these technologies and it's investment and infrastructure and I mean all the work that everybody here does that allows this small team to have this moment of extreme success and just exuberation. Well this is, you brought up the problem that we see with virality and if people with the LLM models don't understand that at any given time, Savannah, usage could pop. There's costs involved. Yes. So how do you manage that? Yeah. You shut it down. You have auto scaling for scaling up but do you got to make sure you got the cost recovery to shut it down or do you want to shut down? You got to get more cash. Yeah. It's an instrumentation problem. It's exactly that. It's instrumentation problem of you have to figure out getting your workload live is, you know, where a lot of people stop, especially startups, they're like, okay, great, it's live. It's working. Now how are you going to manage it? How are you going to operate it in that day two? How are you going to scale it up? How are you going to scale it down? How are you going to handle your costs? That's really where I mean DevOps lies. That's where all the hard part is and there's new patterns that we have to develop for GPUs. Whether it's, yeah, we're going to shut it down or load shedding. So we work across clouds and I think one interesting thing that we've seen is teams will have GPU clusters across different clouds. Maybe one is a backup and they have a more expensive elastic cluster somewhere else and they'll move workloads around depending on, you know, what the performance characteristics look like. Savannah, last week we were talking about this at KubeCon about iterating with AI workloads. When Hassan was on from Broadcom who was getting in the weeds on networking, he talked about the AI workloads, finishing your job completion, which is technical terms for the packets, whatever, moving around. Making it work, right? When you get into like this iteration world, I wanted this, and the question to you is really kind of more current problem people are facing. If you're iterating through your AI workload, model management or model governance, whatever you want to call it, you got to figure things out. It's like making that spaghetti sauce, Savannah. You put a little salt in, how many shakes did I put in? You got to understand how you got there and then when it works, do you remember what you actually did? How do you repeat it? It depends on how many glasses of wine you had making the sauce. In HPC, this has come up. In HPC, this has come up. In some of the sessions where it's like, you're trying to jam so much action on the precision and you actually achieve what you wanted and then you're like, oh shit, why did I get this? What happened? So there's logging, it's observability data. How do you repeat it? And by the way, can you repeat it? With things like Retrieval Augmented Generation, Reg is well known and VectorDays is super hot. So we're now at a new era where you can have some hardcore Broadcom stuff going on, but the software power dynamic is such that there's some new workflow or new techniques. What's your reaction to this line of questioning? Yeah, I love the line of questioning and really I see it as kind of worlds colliding in a different way of, in the DevOps world and the kind of traditional web scale SaaS, we have solved a lot of those problems, right? Reproducibility, auditability, infrastructure as code, declarative definitions, we have the technology to solve those problems and now as we move into the AI world, we're seeing a lot more of a different type of person dealing with these systems. We're seeing researchers, we're seeing data scientists who might be more used to- A lot of them are here on the show. A lot of researchers here, which is great. I mean, they're pushing the edge of science, but they are very much used to Jupyter notebooks where they just kind of go in there and make some stuff work and if it works, it works, but I think that we're going to see a lot more of these. I mean, there's a lot of MLops platforms. Generally, it's in line with this idea of have a platform through which all interactions go and it will enable us to bring the DevOps solutions that we've developed, like auditability and declarative infrastructure into this ML space. It's interesting, Savannah. The words are kind of interchanging. The word memory could be physical memory or memory of what happened in the model. Like, it was retrieval. So, proprietary versus open. Yeah, there's a lot we can say about memory. A lot of jokes I can make about memory and memories I would like to forget from this last 30 minutes. Low latency, high bandwidth conversation here. Yeah, well actually, speaking of conversation, I'm very curious since it is your first super computing and to your point, it's huge. I hope we can show folks at home just how massive it is. I did a quick lap. I think tomorrow I'm actually going to walk it like it's a hike on Strava and see just how big the show floor is to get through all of these vendors as a little bit of exercise on the show. You go to a lot of different technology events. What types of conversations are you most keen to have here and what are you excited to learn a little more about since this is your first time? Yeah, I think the most interesting thing is going on and there's a lot going on in HBC space right now. Yes, there are a lot of people talking about AI. I'm very excited for some of the new chips and some of the new hardware innovations coming out. I think the number one problem we hear from our customers right now trying to build an AI is I can't get an H100, I can't get an A100. There's just not enough capacity out there. Earlier this year we actually built a website called GPUcost.com that shows all the different GPU SKUs that are on the market today and where you can get capacity from different providers. So I'm updating my note stock of all the SKUs that we have to update from this conference. And then the software layer is on top. I mean, that's where we spend a lot of our time and so it's always interesting to see what are other people doing? What is working? What's not working? What tools should we be integrating with or how can we be helping our customers bring more of the best in class model DevOps into our platform? Well, the DGX Cloud from NVIDIA, you're going to see Broadcom probably do that with VMware. It will enable these specialty clouds. How do you see it working from a, very hard for NVIDIA to cost a million dollars to get these GPUs? That's what the service contract that you have to get cost is huge. What's the opportunity for the entrepreneurs out there or the innovators that might not have the big budgets? Is there a growth hack or is there a way to get in this game from a developer or entrepreneur perspective? Yeah, so I think when we talk about developer and entrepreneur perspective, almost inherently, there's maybe a bias towards smaller and less resources. You're early, how do you make something work hackily? By virtue of us building this website early this year, GPUcost.com, there's something really interesting we saw which was there are so much demand for A100, so much demand for H100s, and even like the B100s is quite a bit of demand. There's a lot of GPU SKUs that are actually quite powerful out there that might fall under the radar a little bit. So recently like L40s and L40s's nowhere near as much in the spotlight as the A100 and H100s, but you can run real inference workloads on top of them and we see our users doing that. You're able to get much more capacity, it's cheaper, it can conserve most of the use cases that you have. So I think that's interesting is everyone's focused on the hot new GPU. There's not a lot of them right now for the next six to 18 months, let's say. What is there and how can you leverage that? There's definitely a lot of interesting stuff there. The hot new GPU, if that doesn't tell me that hardware is sexy again, I don't know. I don't know what does, we are back, baby. I absolutely love it. Last question for you about your customers and some of the trends in there. Is there a particular category or space that has you personally excited, not just as an entrepreneur, but as an individual? I'll sound like a broken record. AI is so exciting, just I think one of the best parts about building DevTools or infrastructure and I get to thankfully be at the intersection of doing both of them is you get to be a part of all the things that customers are building on top of you. And so we get to see all of these insane applications where early this month I was talking to a school in India and they were building like a system to real-time translate content from other countries so they could turn in educational materials for their students and they were like doing real-time. Amazing application. Insane, so cool and I got to feel like we're a part of helping this school. So I think the AI stuff is very exciting, it's very interesting. I think I'm the entrepreneur in me is excited about anything that creates multi-cloud. I'm just a really big believer in this meta-cloud, multi-cloud interoperable world and so there's a lot of stuff interesting happening on the compliance and data side in Europe there too, but AI I think has my heart right now. So maybe you can help us with our specialty video cloud. Happy to help. I think you just helped with this interview. On that note, Johnny, thank you so much for being here with us at Supercomputing. I really hope that your first adventure as well as your trip to Denver is fantastic, as fantastic as your insights. John, thank you again for the commentary and color as usual and thank all of you for tuning in to this fabulous four days of coverage here live from Denver, Colorado at Supercomputing 2023. My name's Savannah Peterson and you're watching theCUBE, the leading source for technology news.