 Welcome back everyone, the CUBE's coverage here at Cisco Live Mandalay Bay in Las Vegas. The theme is super cloud, future cloud, networking cloud, collaboration cloud, Serena cloud, Cisco building the platforms, tying unified experience together. Love to see the networking prominent, becoming a platform enabling all kinds of action. AI is a big part of the story and our next guest is CUBE alumni, Thomas Scheiba. Welcome back, VP of product management for the networking group, big responsibility. Thanks for coming on. Thanks for having me again. I love the conversation we've had with you before, networking core product, the revenues are there, other groups now coming in, all unified platform, big story, I like the story, Cisco's all hangs together, now it's got to work. But one of the flavors this year is AI. Yes. So I'm imagining there's a lot of things that can be automated, AI could be a great co-pilot, whatever you want to call it. What are you guys doing with AI and networking? Let's get into it. Yeah, great, great, and you can't get away from AI these days. And so you pointed out, I mean, I'm the networking guy, so really the question I get is, hey, Thomas, we know we want to use AI, actually most of them going down this path for, you know, accelerating how do you suffer development, how do you support, every enterprise looks at this. But the question then comes saying, hey, do we need to do something with my network to make this work? And the answer is we have a great blueprint which we actually released this week, how to actually get the network in or expand what you already have to support AI workloads. So it's really the question around, I said something special you need to have around the network. So it's called AI networking blueprint. What are the salient aspects of that? So here's the interesting discussions going on. There's two things. One is, this discussion, hey, I need GPUs that do the work for AI, right? And they need a mesh and lossless behavior on the network to get the best performance of it. So the question is, can I use a decent network or does it have to be in front of it? And the answer is yes, of course you can do the decent network. The second question everybody asked, and I was saying, okay, do I have to have a separate network for my GPUs versus my CPUs? And the answer is no. You're going to have one standard data center, Ethernet network and we have a blueprint how to actually make this work. So that's the discussion I'm having literally every day. So you're saying the conventional wisdom is you're going to have to separate that traffic, but you're saying that's not necessary. Why is that a perception and how are you able to address that? The conventional wisdom is really more is that how GPUs that is sold today was Infiniband Nix. And it's just in the past, people looked at saying, hey, I need an Infiniband dedicated networks to connect GPUs. The answer is you really don't. You can also use Infiniband clearly, but you can also use Ethernet because Ethernet has evolved over the years to actually have a lossless character. That gap is closing you're saying. Is it closed? Is it, where are we at? I mean, is it close enough? So there's always see, can you optimize if you need a dedicated fabric just for GPUs? There's a little gap and that will close. If you have a network where you say, I want to optimize as a common infrastructure for my GPU cluster, for my high performance computer, for my normal workloads, Ethernet is the answer and will be from an operational TCO perspective much better than operating multiple networks that it takes more power that nobody have. And it's just cost money because you have to operate two different technology networks. And the economics are so much more attractive if you can use the same network. Absolutely, absolutely. My question on the AI is, I'm sure everyone will ask, what's the low hanging fruit for use cases? Where do you see it going? What's the trajectory? What's the, where do you land? And where do you adopt? And where do you expand? Yeah, I mean, I think we've bounced a little bit off. I mean, you probably should talk about what you think the use case is now, but the thing I see, I mean, the office want to say it earlier is anybody that has product with the support, you should be able as a company you have all this data from your customers over years to use AI, I'm generated for AI to basically have faster answers, response on customer increase, using this as a model, how to automate this, how to get faster responses to customers on the support side. That to me is a very, very obvious one. The next one I think there's a lot of discussion around in development, right? If you do software development, can you use the experience of your code that you have written or others have written and saying, hey, I need to solve this problem. Are there some examples, right? The other one, if you go out there and I see this for customer that has Cisco networks, the question, how do I configure this? You can go into a model today, some of the externally available models that use just published data, what we have with data sheets and deployment guides, you can start Googling this, how do I configure an endpoint group on an ACI network? And you will get an answer. Probably close enough, but you can get going. And so I think these are some of the ideas where I see every customer can use this. I mean, I can see the prompt engineering really being like a new kind of discipline for a network operator because I'm just imagining them on top of my head, like, okay, network routing, policy, what are conditions over the internet? What's the best practice? Is there playbooks? I mean, just to kind of get that architectural thinking or even patch management, all kinds of things that were mundane tasks. Yeah, absolutely, as I said, I mean, the way I really look at it is I think where AIM all shines is there's a wealth of information out there either in the public domain, you can leverage to get good answers, or as I said, in companies itself, there's a wealth of information internally that you now can leverage to these models to make the right answers available faster. So most companies that we talk to say, even though it's the year of AI, we've been working on this for a long, long time. I presume the same is for you. So take us back. When did your AI journey start and then really start in earnest? And then if you could help the audience understand how your AI, large language models are different than what we're seeing with, for instance, the ChatGPT, the AI heard around the world. Yeah, so just to come back on my end, it's really on the networking side. Yeah, right. But where does it start? And I think when we started this, it was probably five, six years ago. We were not thinking about the specific use case of AI. We were thinking about how do I have a network that can deal with different workloads, with different performance characteristics. At that point, it was really round. High performance computer was around, how do I have databases that have a lot of node-node interconnectivity? How do I connect this over an Ethernet network while still having other traffic running this as well? And so this is where we started. We actually worked with, can't give the name, but very big companies in the space that have these clusters deployed as racks, that actually five years ago, replaced Infiniband with Ethernet to have one network in these racks. And so this is where we have a lot of these experience. How do you tune, how do you do an dynamic congestion award into these networks to deal with segmentation between different workloads? And so now with GPUs coming around, which is just another endpoint for specific workloads, which is AI workloads, it's the similar problem area. I have a certain workloads hanging off that network with different bandwidth requirements. Now we're talking 200 gig instead of 100 gig, right? But beyond that, it's the same idea, how do I prioritize correctness to get losses behavior for connectivity in these workloads? And so that's why when you asked me, how long are you working on this, we didn't start this three months ago. We started this five or six years ago. That's why we are really very, very confident with the solution that we have that this will work for our customers. But when the chat GPT hype came out, did that change your thinking at all? Did it accelerate your urgency? Or did it have no effect? It accelerated. I think what really happened, what most enterprise customers think about at this point, when I talked to them, six months ago they were saying, wow, do I really need a high performance Eastern network? Do I really need 400 gig? Do I really need 100 or 200 gig on the edge, right? And it was more like customer splitting head for the next five to 10 years thing, I got to put this investment in. What changed is the urgency really on the networking side because everybody says, if I have to put more and more of these GPU clusters in my data center, I need a network that can deal with this, right? And I have a choice that can be either an Eastern network or by a dedicated separate infinite network. Any of these answers from, as I said, operational total cost, I can use my Eastern network, I add additional 100, 200 gig capable leaves and I have a 400 gig core, and there we go. And so that's really what changed. And what's the basis of your, what's the corpus of your data that you're training on? Where are you getting, is it the entire internet? Is it, you know? So this is, I think, where it comes down is depending on, I see there's a lot of data that's available on the public internet. It's like everybody publishes things that's public, but then there's also corporate data, and I'm just looking at Cisco, right? I mean, we have a wealth of tech information, we have a wealth of internal information when we do testing that we can leverage now to expose and give better guidance. When I say expose in our own models, to then expose answers faster, right? And I think, I think we talked a little bit about what you're looking at. I mean, you have a wealth of past interviews, right? That now out there, but how can I use that and get faster answers to some questions? Can I say that? It's a data asset that is harnessed with AI. It's actually creating value from a dormant asset. So you've got unique Cisco data assets and a broader, in context, set of assets from the broader internet, right? And you're training on that corpus? And so that's what we use for us, right? But from a network perspective, so I want to make sure that's what we can use as a company, use this to get better information to our customers. But there's also enabling everybody else with the right network to, they can do this themselves. How do you deal with IP, John and I get so many questions to happen to each other. I want to ask the next one. How do you deal with IP leakage? What's the, what's the- Well, this is the network piece separate, right? But on the data, I mean, this is like seeing where the companies, every company is going through this. Do you want to have your own LLM setup in your enterprise because you want to avoid it? Or some of this, the training, maybe you do in a public cloud and use leverage there to large models existing. And I think this is going to be play out, right? I don't think, I pretend I have to answer to this, but I think this is playing out. Some customers are comfortable to do the learning and public clouds using the GPUs there. Some customers will say, you know what? I don't trust this. What about Cisco? I want, we're looking at this. For some, I think we are comfortable leveraging. For some, where it's the data we don't want to leave, we would keep it in terms of- Okay, so both. It depends on the use case. And I interviewed Matt Garmin who runs Amazon. He used to run their EC2 group. There's some silicon advantages and they're in the cloud, what they're doing is there. They can keep all the data on a VPC. All within the cloud. And then if they're going to go on-premise, this is a networking thing, right? This is all about why I love this platform, bitch. This Cisco live because it's kind of clicking, okay? And so that to me is like, these use cases are going to emerge. So I'm super excited and enthused about that. The question I want to get more at is the question you and I were just having about unlocking this value of data that's sitting there. Now, we've always reported about the cloud and Andy Jassy's comment about, oh, the cloud takes care of all this undifferentiated heavy lifting. The muck and the toil, all that good stuff. Automation, we all kind of talk about. You're getting at, what we're talking about is automating differentiated, enabling differentiated assets. That were either too slow or no mechanism to get out there. Or too expensive. Or too expensive. This is where I think AI could actually identify and allow for agility. And then just watching these key points about IP, it's just more risk management, I think. But I think this thing is a phenomenal wave of something we've never seen before at this level, of value extraction. You're spot on. I look at this really as, there's a tremendous productivity efficiency gain. You can unlock because of that capability being available. That's, and there's so many use cases. I don't think we have scratched the surface. Some office icing, but there's a lot of more things you can do. In the customer base, are we beyond the point of concern about, oh, this is going to take away my job? And we certainly saw that with RPA. But every time you talk to an RPA customer, they're like, oh, this is so great, these software bots. There's always that sort of underlying tension. Is that now dissipated largely, or is it still there? I personally think you will always get sometimes that reaction, because it's more, the way I really think about it is, you're going to unlock options you have instead of working on maintaining the status quo and trying to figure out how to keep it running, you actually can focus on accelerating, unlocking new value or capabilities, right? And depending on who you ask, the answer will be depending on the skill level. What you have to do clearly is you have to retrain, reskill, to take advantage of it. But I don't look at it as, this is like to save cost as, you know, getting rid of position. This is really, unlock capabilities that today I can get to because I'm busy running the current job. Yeah, yeah, I think that's a mindset issue. And we saw that early days of networking with Cisco, we saw it early days of every trend, even cloud. Do you go offence and build and scale, or manage your costs? Early day of automation was precisely this, oh, you just do this, and then the answer is no, by automating this, now I can do other things I couldn't do before. How important is scale? This is sort of a relative question. I mean, of course, scale is important, but you have electrical engineering background, you've got experience with neural networks in a previous, you know, your education, I know. So, but- You've got pedigree. But the availability of massive GPU clusters and supercomputers 10, 15 years ago was limited. As well, it made it hard to really scale from a data standpoint. And obviously open AI and chat GPT is massive scale. And I think generally the engineering community, maybe understated the importance of scale. Now it's very evident. How important is scale to your mission and your customer's mission? Right, I think the answer is always it's somewhere in the middle, right? Because if you look at this, if you look at AI ML, there are really two different phases. There is the learning phase, where you have a very large set of data and you actually want to train or learn the model. And then there is the actual used phase, what people call the inference phase. For that you need actually much smaller cluster. And, but frankly, a lot of those will probably be at the edge because you want them closer to where the customer does the inquiry to get an answer faster, right? And so number one, inference model tend to be smaller. Learning models seem to be larger. And then on the learning model side, there really is the question, how big is the data set, right? Chat GPT, I mean it's basically the internet is the learning set. Yes, huge. You need a large model to compute that data, right? So if you're saying, hey, my learning set is all my videos that you guys have ever put out, you probably don't need the same learning model that you need for the internet, right? And so I think it comes down to, what's the scope of data you want to learn on? And that will drive how big your learning cluster needs to be. And then the second one is, how fast in the inference phase do you need to get answers back to a customer? Those tend to be much smaller and quite frankly closer to the edge because the response time is important, right? People always talk about guardrails and let's put some definition about what that means. An obvious one is that if the AI doesn't know the answer, it actually will tell you that it doesn't know the answer instead of making stuff up. Or it'll ask you more questions, more clarifying questions. Are those, is that the right way to think about guardrails? How do you think about them? That's a long discussion. I think that's really, if you look at Cisco, we actually put a whole positioning paper out and it's an ethical AI, which I think is really, really important. And the way I think about it is in the end, you learn the model based on the dataset and what you get as an answer after you learn is based on the dataset that went in. It's like there's this famous thing, data in. The quality of data in defines the quality of data out and I try to avoid the real time. I don't buy the whole product. That is what it comes to. So the importance really from a guardrail is sorry if it's just too close on us. I think the importance from a guardrail is you need to make sure that you have a good hygiene on the data you use to learn the model because that controls what your answer's going to be. And that's what I'm saying. I don't buy some of this guardrail conversation when it's skews to be political. I think the guardrail is exactly that, the data conversation, the IP leakage, quality of data relative to what you're optimizing for and then making that available for other prompts. And I'd like to hear your thoughts on this, being a neural network, kind of a DNA in your mind from college, we've been seeing the data fusion because a prompt is a call. When I prompt, I'm asking it a question, that's a prompt, that's a query, that's a parameter procedure in my mind. As data merges together, making data connect, so if we have a small language model like theCUBE, it's small, we tune it, I can intersect that into another model. So I can control the hallucinations because it's not going to hallucinate because I'm giving it the parameters. I'm passing my data parameters into the large language model and it's highly functional. That's data interaction. And that's the thing where a lot of the work needs to happen, the way I think about integrity of data, right? If you have your own data center, I think where most, at least I think will start, harvesting the internal experience and data set because you know what you have and build answers around that. If you don't experience it, I want to mesh it over sources that you might not control. This is, I think, where a lot of work needs to happen. It's like, how do I ensure I have good data sets that I know I can guarantee quality on the outcome? Yeah, but I want to take it away from that and talk about chat GPT in particular because we all have experience with it. My understanding is it's designed to predict the next word and to predict the next word, it does a really good job of that and it's got the entire internet at its disposal to really try to understand the human condition. And as a result, it predicts what you're going to say and it makes a lot of mistakes. My point is, is it the right way to think about it, is have the, and it's not designed to do this today because it doesn't do it. Have the AI ask you, like what Jason, it said Jason Calcana started the cube with me. It should ask me, well, is Jason Calcana's involved in the cube and just clarify? Because that's the internet they're scraping. Yeah, but being smart enough to know, to ask questions when it doesn't actually know, like it's a probability thing, is what's the probability of this being the right return and if it's below some threshold, ask more questions and get more, that to me is an obvious guardrail. Yeah, probably a much longer discussion. I think I'll always come back to first points around this one. In the end, if you look at these models, how this is built, there's a learning phase that relies on the data set that's available and you need to have a good knowledge of the quality of that data, right? Because as I say, if there's noise in the data, since that's your tagline, right? There's a lot of noise in the internet. And you're going to get noise out because you train on noise, right? And so I think that's really the challenge. I do think it's actually for all enterprises or a larger customer that leverage these models. If you can control the input data, you will have a really good output. If you don't, it's challenging. And also, if you can have your own data set, I'd imagine, you mentioned earlier, Cisco has its own data that you're going to expose publicly. I'm sure you have a lot of traffic data. You probably have DDoS signaling when threats are coming in. So like, I mean, I'm making this up, but I'm oversimplifying it. But the point is, if you have that data exposed, you can train the AI to identify, hey, alert. So it's almost reverse policy. You already know policy scenarios. Why not make that extensible? So that's where it was going, right? I think the value, and I just picked as an idea what you could do, but that's what I do think for a lot of customers, the value sets. They have this data sets today that they already use for support, but it takes them probably a day, two days to get to the right answer with the IML models on top. You can get to answers much faster. Do you consider what Cisco is building a learning system? I think any enterprise will do this, right? I personally actually think the advantage of companies with large set of data will be very, very successfully leveraging AI-ML because you get to quality answers much faster than without. And whether you call it a learning system or not, I think it's somewhere because it has expert systems, but every large customer that has, or an in-customer customer at enterprise that has such data sets will be able to leverage AI-ML to get to better quality answers faster. Does it write your PRDs for you now in the product management team, AI? We're not quite there yet, but hey, it's an interesting thought. The reasoning is coming, the meta-reasoning. Final question for me is more on the product side. You run product management, it's a discipline that's highly important in Cisco. You shape the product, you got to look at the engineer, you got to look at the customers. What is AI going to have impact on, on the acceleration of how products are built? Do you see that AI could assist the acceleration of product market fit? And then once you have product market fit, can AI help you guys go to market faster? How do you see that? I know you might not have a direct answer now in the longer conversation, but what's your thoughts, what's your reaction to, okay, does it change the operating cycle of how you do business? I absolutely think there's an opportunity to do that. Because in the end, the way it is, I mean, what product management is, or product management, you listen to customers, you get feedback, you try to derive what customers really want based on their use cases, and then you build a product around that. So can they use the AI-ML from getting this wealth of information as an input and, you know, aggregate faster? Absolutely, I can see opportunities there, yes. Awesome. Thomas, thanks for coming on theCUBE. I know you're super busy. We've got Audi coming up next. We've got Dr. Henning coming on. Head of Audi Production Lab. That's going to be a good conversation. Love driving those cars. You know anything about Audi's? I love them. All we have, we're an Audi family. Love, love, love, love. Thomas Shiber here on theCUBE. This is day two coverage. This is the pop-up queue. Go to siliconangle.com. You'll see all the stories from that are going to come from here. The video's going to be there. And theCUBE.net is the catalog where all the videos will sit. And after the show's over, we're live right now on the stream. Have a great day. We'll be right back after this short break.