 Hello and welcome to this episode of This Week in Tech. I'm your host Shelly Kramer, Managing Director and Principal Analyst here at theCUBE Research. I'm joined today by my fellow analyst and member of the CUBE Collective Community, Joe Peterson. And today Joe and I are joined by Google Clouds Bobby Allen and Brandon Royal. Great to see you all. So our conversation today is going to focus on unpacking the new Google Cloud Hugging Face partnership, which is all about accelerating Gen AI and ML development, which of course is top of mind for everybody today, right? And so a little bit of backstory here before we dive into our conversation. In early February, Google Cloud and Hugging Face announced a partnership and one that has been touted as really a giant win for the developer community. This partnership gives developers the ability to train and operate Hugging Face models on Google Cloud infrastructure and this should go a long way toward accelerating Gen AI and ML development. So this new partnership lets developers build, train and deploy AI models without needing to pay for a Google Cloud subscription. And to me, that's a key part of this alliance here and something I think it'll make it even more incredibly attractive. So outside developers using the Hugging Face platform will have cost-effective access to Google's tensor processing units and GPU supercomputers, which will include thousands of NVIDIAs in demand and export restricted H100s. So there you go. Access to something that is hard to get access to. So before we dive into this conversation, which I know is going to be a terrific one, Bobby, I know that you've been a Google Cloud for about three years now. Will you tell us a little bit about your career journey and how you ended up here? Sure, Shelly. I'd love to. First, thank you for having me on the program. And secondly, it has been about two and a half, three years, it's been going really, really quickly. But I joined Google after 10 years of cloud computing startup. So you can believe that I was doing multi-cloud in 2012 and we had no clue what we were doing. It was a mess. There was all this new stuff. It was like exciting. It was exhilarating. It was scary. And this feels very similar to that. Before that, I spent time at Bank of America. Before that, I spent time with Intel. And if you look me up on LinkedIn, Shelly, your audience will see that I kind of made up my own title. I called myself a Cloud Paracus. And I did that because we were making it up as we go along. So I figured why not make it my own title? I love it. Outside of Google, I'm also a pastor. And so kind of when you hook all those things together, really like to talk to people. I really like to connect with people. And I really like to try to help people. And so when I think about transformation, the technology and all the things that are happening, I want to be the person who wants to listen first. I don't want to just stuff 20 pounds in a five pound bag or make you think I'm the smartest person in the room because I'm usually not. I want to be the person that listens to what you're trying to accomplish. And if I can offer something helpful, that's really what I care about. Well, you know, that's how we serve our customers, Bobby. It's not by coming, you know what I'm saying? I mean, I've had colleagues before who, you know, show up in briefing meetings or whatever and, you know, just pitching, always be selling and all that sort of thing. And really, I'm a fan of always be listening. I think that's a great strategy. I try to do that too. It's like, Brandon, I think you've been with Google Cloud for about four years now. Tell us a little bit about your journey. Yeah, just a little bit longer than Bobby, but not much. Yeah, I've been with Google Cloud for just four years, joined in 2020. I'm a product manager focused on part of our AI infrastructure portfolio, GKE. And really my background has primarily been in containers, all things containers. You'll hear me talk a lot about sort of the power of containers. And just before joining Google, I was at a small little startup called Docker. I helped the Microsoft team deliver early Windows containers capability. So I've been sort of very much steeped in all things containers and cloud-native infrastructure for a very long time. But really come more from an application background, more of a developer just out of school, hands-on keyboard doing development. That's what I did in the beginning of my career. And so I really like to, when I talk to customers about their AI journey how customers are adopting ML, I really take the lens of what would a developer do with AI? And what does it mean to infuse AI into development, into applications? So while my recent experience has very much been AI and ML heavy, I do take very much of a container-centric view and helping customers to go through this journey together. So yeah, so super excited to have the conversation today. Awesome. Well, you two together are kind of a whole package. Forget that a lot, Shelley. Yeah. Well, you know, right? And you know what? That's how Joe and I are too. We laugh about that sometimes because Joe brings, you know, Joe brings her engineering brain and her tech analyst brain and I bring my tech analyst brain and my years of marketing brand strategy focus. And so, you know, a lot of things like customer behavior and messaging, all those sorts of things are things that I think about. And you know, she thinks about how things are put together. And so, you know, we make a good pair. So it sounds like you do as well. I like it. I like it. So in case you're not familiar with hugging face, it's more, it's one of the more popular AI repositories. It's a platform for viewing, sharing and showcasing machine learning models, data sets and related work. Its goal is to make neural language models accessible to anyone building applications powered by machine learning. And I think that, you know, kind of a quick summary of the services that hugging face provides are, you know, it hosts an open source pre-trained, can host open source pre-trained machine learning models. Users get easy access to these models through various environments, for instance, Google collab or Python virtual environment. They provide tools for adjusting machine learning models, an API that offers a user friendly interface for performing tasks, machine learning related tasks, and then community spaces for collaborating and sharing and showcasing work. So it is a really, it is a really awesome platform. And so the partnership between the two of you, I think is an exciting news in the industry as a whole. And Joe, I know that you had a little bit of, I know that you had a little bit of kind of information on hugging face in the transformer model library. I think that was, that you thought was particularly interesting. Yeah. You know, the core of hugging face is the transformers model library dataset library and the pipeline. So hugging face has a library in case folks don't know this, of 495,000, that's right, I said thousand models. And they're grouped, I'm right, crazy. And they're grouped into data types called modalities. So some of the tasks that customers can perform, like you were mentioning, Shelley, are object detection, question answering, summarization, text generation, translation and text to speech. So I think the key here is to have this tool set in a customer's back pocket that really shaves off time to market and helps them leapfrog the competition. And, you know, guys, keep me honest here, but Google Cloud customers are going to be able to deploy hugging face models within GKE and Vertex AI. And that's the company's ML platform offering Gemini, which is a multimodal platform from Google DeepMind. And it's expected that the Vertex AI and GKE will be available on the hugging face platform in the first half of 2024. Did I say all that right? Did I get that right? Yeah. Your own tractor. Sounds good. All right. All right. I'm just checking. Excellent. So here's what I'm interested in. So Bobby, this new partnership with hugging face, you know, it's all around making it easier for developers to build and train and deploy AI models in the open ecosystem. This is kind of a big deal for AI development, isn't it? It is. It is, Shelley. So it's a huge deal. And before I go directly into that question, I want to thank you for explaining what hugging face is, because we don't want to leave any of the listeners behind. There's so much stuff happening in AI right now that I think is really important to get people on ramps, because most folks in their day jobs don't have time to keep up with all that that was an announcement to come out every day. So thank you for doing that. I want to answer your question, but I want to also put AI in its proper context. Because one of the things that I think is happening is AI is becoming this junk drawer of all these different things that are happening. So I want to make a statement. Joe and I have talked about things like this before. I love food analogy, Shelley, to try to explain things very simply. So I have one for you. I'm going to make a statement. Then I want to give you a visual. Okay. And so here's the statement. My statement, Shelley, is that AI is not the thing. AI is the thing that makes the thing better. So let me kind of unpack. So in terms of what the partnership means, AI, in my opinion, Shelley, is the sauce for the spikes that enhances the flavor of the dish. It's not the dish itself. The dish is the application. And so what does this mean? I think the Google Hugging Face partnership means that essentially the AI cooks, if you will, now have access to this huge pantry of spices and sauces. And so as they want to taste and experiment with different applications, they can pull what they want off the shelf and use that to make the application better. That's essentially what it means. I love that analogy. What you don't know about me is every minute of every day in my life is spent thinking about my next meal. So that is a wonderful analogy. Thank you, Shelley. You know what some people may have missed in the announcement is that the word application has mentioned no less than five times. Yeah. Oh, I don't know. It's also models in the AI, but again, AI has got to be used to enhance, accelerate, or improve an application because that's really where people are going to get value from. Yeah, absolutely. So Brandon, I'm guessing that you may spend some of your days experimenting with AI and maybe even doing some hands-on development with large language or generative models. What is the hugging face Google Cloud partnership mean to you? Yeah. Yeah, it's a good question. So maybe I'll take a step back and kind of look at the last 12 months, if you will. There's been an unprecedented amount of innovation in AI models. And you couldn't agree more, by the way, with Bobby's point. And it's a fantastic analogy there as well about really thinking about applications role in AI. But even if you look in the last 12 months of AI innovation, so much of that innovation has been happening in the open ecosystem. And hugging face has been a huge player, a big part of that ecosystem in general. And if you follow things like the hugging face leaderboard as an example, anyone's familiar with that, and you might know what I'm talking about, it's essentially a leaderboard of all the top models that are happening. You can see changes of new models every week, new data sets, new research. I mean, the pace of innovation is absolutely staggering. And so while the imagination, I think of, a lot of folks, a lot of the listeners have likely been capsified by very powerful models like Gemini from Google and Claude and Claude II and ChatGPT and a number of others. It's really important to call out that a lot of this groundswell of innovation is actually happening in the open, with open models and open data. And that's really, really huge. So my background being an open source, this is really an exciting time to be in AI because you have just an amazing access to innovation that's happening out in the ecosystem. And a lot of players are playing a big role here. Google, of course, is playing a big role in this partnership. We've been very open with models all the way back to BERT and open research from Google Brain and a bunch of others. That's ever led a lot of this innovation. But there are other players in the ecosystem that are also contributing. If you look at what Meta's done with Lama and Lama II, that's been huge in creating new models for innovation that the community has really ran with. So I think it's a really exciting time to be in the open ecosystem. And I think from the partnership perspective, this gives us great new opportunities to engage developers, things like better playgrounds for experimentation. You want to experiment with models. You want to try something out. You have an idea. You want to prototype it quickly. This is going to allow for even easier experimentation with a large library of models. It's also going to mean a faster path from prototype to production. And that's a big deal. You talk about this panel all the time, this idea that data scientists or ML engineers are building these models, but they don't deliver a value until they're actually in production, until they're actually consumed or they're integrated with an application. So we're really excited about that. We believe we have a great opportunity to serve our customers, to serve the community, create a lower friction path to production, and help developers build their ideas, prototype them, and ultimately leverage Google's leading infrastructure and Kubernetes-based platforms like GKE to help them deliver those production-ready products faster. Well, and I think the fact that you don't have to be a Google Cloud customer is a huge, huge assist that you're providing. I mean, I think opening the doors and saying, everybody's welcome to come and play here in this sandbox, that's how you spur innovation. Yeah, experimentation, it's a big part of how these models are becoming so popular. And we've done things before, if there's some great playgrounds for image generation with stabilities, partnership also with stability and stability AI and hugging face. So you can actually use TPUs, Google's Tensor Processing Units, to build images really quickly. And that's all in a playground. That's completely free to experiment. So it really gives developers the ability just to try things out, to test out their ideas and to build that prototype ultimately faster. And when customers are ready to get to production and start to scale things out using the infrastructure, we're here and we're ready to help them through that journey. I get it. Yeah, well, Bobby, I love your food analogies, of course. So I want to talk about maybe a different spice cabinet with you. I want to talk about the Vertex AI products suite. There's 130 prepackaged foundation models. So talk about some spices to choose from. Why is this an important piece of news for developers in particular? I think this is, let's extend the food analogy a little bit, Joe, but also mix in some of the kind of computer stuff. So when I used to go over my grandmother's house and you tell grandma you're working in the cloud, that doesn't mean anything. Baby, can you look at my printer while you're here? Yeah. You're just your computer person, right? That's right. You're a technology generalist, right? Here, fix my remote. Fix my remote. Look at my TV, reprogram my whatever, right? But the reality is there are specializations in technology now, right? And so what's interesting is that we know that if we think about builders as kind of a comprehensive term, they're developers, they're platform operators or infrastructure people. What's nice about this is Vertex allows people to kind of focus on the part of the technology piece that they're experts in. You may be a person who wants to just deal with the stuff above the infrastructure layer and you want to be able to pull some of those recipes that are already in Vertex or supplement those with some of those open source spices that Brandon talked about in Huggingface. You can do that without dealing with all the other stuff underneath of that. So typically you'd have to deploy a platform to be able to roll out a model or an application. This kind of keeps you at the level that you want to play at because some developers may not be platform experts and may not have a platform team. And so this is exciting because you can stay at the right level, focus on the things that you want to play around with to kind of get value from that quickly. So the other way that I would say this show is Vertex comes with a lot of stuff baked in out of the box, right? If you think about something like an airport that knows how to heat up pizza or, you know, make chicken nuggets, which is a big fan of my doors or tater tots, my personal favorite. Imagine if you could pull down another type of recipe or another type of action that that air fryer could take. If you could pull down a way for that air fryer to make tacos or to do something else. Vertex is like that preset platform that allows you to do things quickly, but now we can learn from Huggingface to pull other models off the shelf that could be deployed and managed in Vertex. You don't have to build an oven to cook a meal. I like it. Yeah, it's kind of it's kind of cool. Brandon, question for you. So if I'm getting my information correct, there's going to be a similar integration that we talked about with Vertex AI that's going to roll up for GKE. And again, developers can, you know, conceivably use that service to run AI workloads such as open models from Huggingface in containers. Can you talk to us a little bit about that integration? Did I get that right? And then if that did, can you talk to us about that integration? Yeah, absolutely. You know, and as I sort of mentioned in my introduction, you know, people typically think of containers and Kubernetes in its sort of historical context. And by the way, this year marks 10 years of Kubernetes. So, you know, talk about a battle tested platform that has grown to support just about every workload you could imagine for the cloud and on-prem. And so, you know, I think historically, a lot of these workloads have been sort of thought as this as application workloads. But, you know, it turns out, if you actually look at the data, more and more ML workloads are being posted and deployed directly into Kubernetes. So, you know, to Bobby's point earlier about, you know, folks having different specialties, you know, there are a set of customers and organizations and individuals within organizations who really focus on that lower level, that level of control that's needed for a lot of specialty workloads, you know, it's the nature of the model, the requirements of the organization, clients, data sovereignty, whatever it happens to be. You know, there are organizations for a number of reasons that want that lower level of control, but they also want the scalability that Kubernetes delivers. And Kubernetes is pretty unique in its ability to efficiently manage all kinds of infrastructure and all kinds of different workloads. So, now you can start to think of Kubernetes as the orchestrator for your ML models. And you can now orchestrate those ML models not only over a traditional compute, but also across all kinds of different accelerators. So, whether those are GPUs, CPUs, or even TPUs, Google's Tensor Processing Units, those can now all be orchestrated using Kubernetes. And, you know, for us, it's really exciting, right? Because we can now deliver sort of the Kubernetes experience that we're really proud of, that we're really excited about, you know, which is Google Kubernetes Engine, to those customers, and really just lower that path for lower the sort of friction of deploying those models. We just want to make it super fast and super easy. You have a model that's in hugging face, you want to get that thing to production as quickly as you can, but you also want that lower level of control. So, that kind of integration is going to offer that. So, yeah, we're super excited. And, you know, for those of you out there, for listeners that are maybe thinking, you know, Kubernetes, it's not an ML platform, or maybe containers aren't an ML platform, I also want to sort of remind you that containers have been here for ML for a very long time. You've used, say, a Jupyter Notebook as an example, or maybe you've done some kind of machine, you know, maybe you've done some kind of machine learning with like a deep learning image, that's likely being served directly from a container. So, a lot of this stuff in ML has actually been happening within containers, has been happening within Kubernetes, and even some of the largest organizations of the largest foundation models that we're all familiar with today, most of those are also being orchestrated with containers. So, long story short, you know, containers and Kubernetes is really a battle-tested platform for managing AI and ML workloads, and we're really just excited about that hoogie face partnership, it's just making that easier and easier for customers to deploy those in production while having control and flexibility and efficiency that they need. You know, I feel like that the theme song of 2024 is ease of use, simplicity, speed, you know, all the things. And I think that that's really sort of personifies the industry, certainly the four of us work in, right? We all want it fast and we can, but I think what we're seeing is that, you know, as the technology space has evolved and complexity has risen so much, so any kind of partnerships and any kind of solutions that brands bring to the table that make complex things less complex, I think is a big deal. I think we're seeing a lot of that and I know that customers, that's really important to customers, you know? Yeah, yeah, absolutely. Yeah, it was, you know, quite popular at the end of each year to do sort of predictions for the next year. And it was, of course, no different, right? The number of blocks for the four of us have read about 2024 predictions. But you're right, Shelley, that was a big one. It was all about ease of use. We know that we now have this huge opportunity to take these AI models and all this innovation and now infuse it into our applications, infuse it into our services to deliver new value and do it in a way that's easy and safe and efficient. Like that's a huge deal and really I think that's what 2024 is really going to be about. Yeah, I agree, I agree. So I want to dial in just a second to TPUs and so, you know, most folks following AI developments are likely familiar with GPUs being a critical accelerator for AI model training and inference. But this announcement particularly calls out TPUs and you just spoke to that just a moment ago. But tell us a little bit more about exactly what a TPU is and why does it matter so much for hugging face users who are developing and deploying AI models? Yeah, yeah, thanks for the question. You know, I think, you know, first and foremost, Google has had a long-standing partnership with NVIDIA. You know, going back quite a long ways and we're really excited about the GPU capability that we're offering, not only through this partnership but also just across the entire platform. You know, leadership and things like, you know, NVIDIA L4 is where the first market with those capabilities, you know, huge investments in each 100s and a number of others make GPUs as easy to consume and powerful to consume with NGCP as possible. That said, you know, I think we also have this huge opportunity to leverage a lot of the innovation that's been happening within Google with TPUs. So TPUs, again, are tensor processing units. That's our own custom silicon that we've been building over many years to serve our own machine learning needs. So, you know, Google obviously has a very long-standing history of building machine learning, machine learning models, AI. We build it into all of our largest products and we need specialized infrastructure. We have needed and we continue to need specialized infrastructure in order to serve the unique needs of our AI models. So, you know, we've been bringing that to our customers through what we call cloud TPUs, basically the same infrastructure. We now bring it directly to our customers in NGCP, giving them the ability to leverage this specialized infrastructure that's really designed for AIs. It was designed for generative AI from day one. So, you know, we really want to, you know, make that more accessible to warming up customers so they can start to sort of see the power that TPUs can bring to their own AI development, you know, whether it's for price performance, it's really important for them to increase the margins and sort of the economics, if you will, behind their AI models, or whether it's, you know, lightning past latency, whatever it happens to be. We want to make those technologies available to more and more customers. And so the Hugging Face partnership's an ability for us to do that. We want to help more and more of those models be sort of compatible with the TPU ecosystem through things like PyTorch XLA, which is something that we continually invest in, that is basically a way of using PyTorch, which is an incredibly popular framework for ML models, and allowing those models to then run on TPUs. And of course, lots of investment happening in JAX, which is another framework that's increasingly popular, designed by Google, available for TPUs. So, you know, a lot of this investment is really happening in the ecosystem with the core focus of helping more and more developers build their models and run their models on TPU, really get the most value out of cloud. So we're really excited about that. Yeah, well, it would end with good reason. Yeah, I mean, it makes perfect sense. So as we wrap the show, I would love to hear one piece of advice from each one of you on one developers and GCP customers or non-customers should be thinking about as they hear about what's possible here and as they kind of prepare to dive in and get started. What's your best advice, Bobby? So my best advice is always going to be a little bit non-technical for a second. So I want to leave you with the Bobbyism. I try to leave this insight with folks, just things I've cleaned over the years. And so the short version is technology is the easy part of tech. The longer version is tech is the easy part. People are the best part. Behavior is the hard part. Humility is the worst part. And so you may not like the fact that someone with Google shirts admitting that I don't know everything about AI because I don't. So what I did is a hired or smart guy like Brandon had brought him on the car back. When I tried to model Shelley and Joe for the audience, it says, it is really hard for us to know everything about everything. We have to work with people. We have to work together. Maybe someone teaches you about Kubernetes and you teach them about notebooks or other concepts. So we can share the knowledge. Like the community aspect of this is what's powerful. And if we don't embrace the humility of the fact that we can't do it all, know it all or learn it all, it's an island we're going to struggle. And so that's what I hope people get from my part of this, Shelley, is that I would smart people like Brandon, if you don't know everything. Literally, I'm Brandon's manager, but I've learned something from him every week. And I hope it's okay for your audience to hear that. We need to learn from each other. We need to be humble enough to admit that we don't have all the answers. So I would say do that. Don't leave people behind in the rush for technology. Right. Remember, the people are the best part. And I'll give you one more kind of hackable baptism. Right. We talk about people process technology. Let's get back to people process problem. Who are we making something better for or solving something for? The technology can drown out the people, see someone's face when you're working on that project or exploring that topic. I'm going to make this better for grandma or so and so. Or a child or an elderly person, like make something better for somebody or a person. So that's my advice, because that's how we translate Shelley and Jody's science fair projects into something that'll actually make the world better. Yeah, I know. I think that's wonderful advice. And I am going to, fair warning, I'm going to now use that Bobby ism from now until eternity, because that's what I do. But that is wonderful. And, you know, I've been working in the digital transformation space as all of us have for a very long time. And I'm regularly beating the drum that it's not technology alone. It's not the answer. It is people and it is processes and it is technology. But I love the way that you morph that. I think that and I love the humility add. I think that all of us really need to not at all be reluctant to admit, I'm not the smartest person in the world. I love learning. I want to be a sponge. And that's really, I think that's the path to success today. Thank you. Yeah, that's really Brandon. What's your piece of advice? Yeah, along the similar lines is don't be afraid to experiment. Can you look back in this, we're in the beginning of this incredibly exciting time. That's not unlike however many years ago, Edison and Westinghouse talking about, you know, this electricity thing, it's going to be groundbreaking. Look, we can have a light bulb. How exciting is that? You know, if they had only imagined the different applications for electricity that we now completely take credit today, you know, those early inventors, experimenters, they couldn't even imagine that it's only through iterative experimentation, trying new ideas. Don't be afraid to fail. Throw something out there. See if it sticks. And I think we'll be really amazed to see in just five years' time what all this foundational work we've all done as an ecosystem. So don't be afraid to experiment and try something new. Great advice. Well, Bobby and Brandon, thanks so much for joining Joe and me today. I knew this was going to be a fascinating conversation. And so thrilled that we were able to kind of unpack this and really show our listening and viewing audience what this partnership means. And hopefully the developer community is excited about this because I think it's a tremendous opportunity to get in and build things and do good things. So with that, we're going to wrap this week in tech. Thank all three of you for joining us today. And we will see you again next time. Thanks, guys.