 Welcome back, we're live here from Supercomputing 22 in Dallas, Paul Gillan for Silicon Angle in the Cube with my guest host, Dave. Excuse me. Our guest today on this segment is Kim Lanar who is a storage performance architect at Broadcom. And the topic of this conversation is networking, it's connectivity. I guess how does that relate to the work of a storage performance architect? Well that's a really good question. So yeah, I have been focused on storage performance for about 22 years. But even if we're talking about just storage, the entire, all the components have a really big impact on ultimately how quickly you can access your data. So, you know, the switches, the memory bandwidth, the expanders, just the different protocols that you're using. And so, and the big part of it is actually ethernet because as you know, data's not siloed anymore. You have to be able to access it from anywhere in the world. So wait, so you're telling me that we're just not living in a CPU centric world now? Because it is sort of interesting when we talk about supercomputing and high performance computing, we're always talking about clustering systems. So, how do you connect those systems? Isn't that kind of your, your wheelhouse? It really is, it is Broadcom's wheelhouse. We are all about interconnectivity. And we own the interconnectivity. You know, you know, years ago it was, hey, you know, buy this new server because you know, we've added more cores and we've got better memory, but now you've got all this siloed data and we've got, you know, we've got this, this stoffer defined kind of environment now, this composable environments where, hey, if you need more networking, just plug this in. Or just go in here and just allocate yourself more. So what we're seeing is these silos really of, hey, here's our compute, here's your networking, here's your storage. And so how do you put those all together? The thing is interconnectivity. So that's really what we specialize in. I'm really, you know, I'm really happy to be here to talk about some of the things that we do to enable high performance computing. Now we're seeing, you know, new breed of AI computers being built with multiple GPUs, very large amounts of data being transferred between them and the internet really has become a bottleneck, the interconnect has become a bottleneck. Is that something that Broadcom is working on alleviating? Absolutely. So we work with a lot of different, there's a lot of different standards that we work with to define so that we can make sure that we work everywhere. So even if you're just a dentist's office that's deploying one server, or we're talking about these hyperscalers that are, you know, that have thousands or, you know, tens of thousands of servers, you know, we're working on making sure that the next generation is able to outperform the previous generation. Not only that, but we found that, you know, with these siloed things, if you add more storage, but that means we're going to eat up six cores using that, it's not really as useful. So Broadcom's really been focused on trying to offload the CPU. So we're offloading it from, you know, data security, data protection, you know, where we do packet sniffing ourselves and things like that. So no longer do we rely on the CPU to do that kind of processing for us, but we become very smart devices all on our own so that they work very well in these kinds of environments. So how about, give us an example. I know a lot of the discussion here has been around using Ethernet as the connectivity layer. Yes. You know, in the past people would think about supercomputing as exclusively being Infiniband based, but give us an idea of what Broadcom is doing in the Ethernet space. What are the advantages of using Ethernet? So we've made two really big announcements. The first one is our Tomahawk 5 Ethernet switch. So it's a 400 gigi Ethernet switch. And the other thing we announced too was our Thor. So we have, these are our newer controllers that also support up to 400 gigi each as well. So those two alone, it's amazing to me how much data we're able to transfer with those, but not only that, but there's super, super intelligent controllers too. And then we realized, hey, we're managing all this data. Let's go ahead and offload the CPU. So we actually adopted the Rocky standards. So that's one of the things that puts us above Infiniband is that Ethernet is ubiquitous. It's everywhere. And Infiniband is primarily just owned by one or two companies. And it's also a lot more expensive. So Ethernet is just, it's everywhere. And now with the Rocky standards, we're working along with, it does what you're talking about, much better than predecessors. Tell us about the Rocky standards. I'm not familiar with it. I'm sure some of our listeners are not. What is the Rocky standards? So it's our DMA over-converged to Ethernet. I'm not a Rocky expert myself, but I am an expert on how to offload the CPU. And so one of the things it does is instead of using the CPU to transfer the data from the user space over to the next server when you're transferring it, we actually will do it ourselves. So we'll handle it ourselves. We will take it, we will move it across the wire and we will put it in that remote computer. And we don't have to ask the CPU to do anything to get involved in that. So it's a big savings. Yeah, I mean in a nutshell, because there are parts of the Infiniband protocol that are essentially embedded in RDMA over-converged Ethernet. So if you can leverage kind of the best of both worlds, but have it in an Ethernet environment, which is already ubiquitous, it seems like it's kind of democratizing super computing and HPC. And I know you guys are big partners with Dell as an example. You guys work with all sorts of other people. But let's say somebody's going to be doing Ethernet for connectivity. You also offer switches? We do, actually. I mean, that's another piece of the puzzle? That's a big piece of the puzzle. So we just released our Atlas II switch. It is a PCIe Gen 5 switch. What does that mean? What is Gen 5? What does that mean? Oh, Gen 5 PCIe. It's a magic connectivity right now. So we talk about the Sapphire Rapids release as well as the Gen OI release. I know that those have been talked about a lot here. I've been walking around and everybody's talking about it. Well, those enable the Gen 5 PCIe interfaces. So we've been able to double the bandwidth from the Gen 4 up to the Gen 5. So in order to support that, we do now have our Atlas II PCIe Gen 5 switch. And it allows you to connect, especially around here, we're talking about artificial intelligence and machine learning. A lot of these are relying on the GPUs and the DPUs that you see a lot of people talking about enabling. So by putting these switches in the servers, you can connect multitudes of not only NVMe devices, but also these GPUs and these CPUs. So besides that, we also have the storage component of it too. So to support that, we just recently have released our 9,500 series HBAs which support 24 gig Sass. And this is kind of a big deal for some of our hyperscalers that say, hey look, our next generation, we're putting a hundred hard drives in. So we're like, you know, so a lot of it is maybe for cold storage, but by giving them that 24 gig bandwidth and by having these mass 24 gig Sass expanders that allows these hyperscalers to build up their systems. And how are you supporting the HPC community writ large? And what are you doing this exclusively for supercomputing? Exclusively for, so we're doing the interconnectivity really for them. You know, you can have as much compute power as you want, but these are very data hungry applications. And a lot of that data is not sitting right in the box. A lot of that data is sitting in some other country or in some other city or just the box next door. So to be able to move that data around, you know, there's a new concept where they say, you know, do the compute where the data is. And then there's another kind of, you know, the other way is move the data around, which is a lot easier kind of sometimes, but so we're allowing us to move that data around. So for that, you know, we do have our Tomahawk switches. We've got our Thor, Knicks, and of course we got, you know, the really wide pipe. So our new 9500 series HBA and RAID controllers not only allow us to do, so we're doing 28 gigabytes a second that we can trans through the one controller and that's on protected data. So we can actually have the high availability protected data of RAID five or RAID six or RAID 10 in the box, giving in 27 gigabytes a second. So it's unheard of. The latency that we're seeing even off of this too, we have a right cash latency that is sub eight microseconds. That is lower than most of the NVMe drives that you see, you know, that are available today. So, you know, we're able to support these applications that require really low latency, as well as data protection. So often when we talk about the underlying hardware, it's a game of, you know, whack a mole, chase the bottleneck. And so you've mentioned PCI-E five. A lot of folks who will be implementing five, gen five, PCI-E five, are coming off of three, not even four. I know. So they're not just getting a last generation to this generation bump, but they're getting a two generations. They are. How does that, is it the case that it would never make sense to use a next gen or current gen card in an older generation bus because of the mismatch in performance? Are these things all designed to work together? That's a really tough question. I want to say no, it doesn't make sense. It really makes sense just to kind of move things forward and buy a card that's made for the bus. And however, that's not always the case. So for instance, our 9500 controller is a gen four PCI-E. But what we did, and we doubled the PCI-E, so it's a buy 16, even though it's a gen four, it's a buy 16. So we're getting really, really good bandwidth out of it. As I said before, we're getting 28, 27.8 or almost 28 gigabytes a second bandwidth out of that by doubling the PCI-E bus. But they work together. It all works together. It all works together. You can put our gen four and a gen five all day long and they work beautifully. We do work to validate that. We're almost out of time, I want to ask you a more nuts and bolts question about storage. And we've heard for years of the aerial density of hard disks has been reached and there's really no way to make the disk any denser. What does the future of the hard disk look like as a storage medium? Multi actuator, actually. We're seeing a lot of multi actuator. I was surprised to see it come across my desk because our 9500 actually does support multi actuator. And so it was really neat after, I've been working with hard drives for 22 years. And I remember when they could do 30 megabytes a second and that was amazing. That was like, wow, 30 megabytes a second. And then about 15 years ago, they hit around 200 to 250 megabytes a second and they stayed there. They haven't gone anywhere. What they have done is they've increased the density so that you can have more storage. So you can easily go out and buy 15 to 30 terabyte drive but you're not going to get any more performance. So what they've done is they've added multiple actuator. So each one of these can do its own streaming and each one of these can actually do their own seeking. So you can get two and four. And I've even seen a talk about eight actuator per disk. I don't think that that, I think that's still theory but they could implement those. So that's one of the things that we're seeing. Old technology somehow finds a way to remain current even in the face of new alternatives. Kim Lay and our storage architect, storage performance architect at Broadcom. Thanks so much for being here with us today. Thank you so much for having me. This is Paul Gillan with Dave Nicholson here at Supercomputing 22. We'll be right back.