 Hi, I'm Peter Burris and this is Wikibon's Action Item. We're joined here in the studio by David Floyer. Hi, David. Hi there. And remote, we've got Jim Kabilis. Hi, Jim. Hi, everybody. Now, Jim, you probably can't see this, but for those of you who are watching when we do see the broad set, notice that David Floyer's got his Game of Thrones coffee cup with us. Now, that has nothing to do with the topic. David and Jim, we're going to be talking about this challenge that businesses have, that enterprises have, as they think about making practical use of AI. The presumption for many years was that we were going to move all the data up into the cloud in a central location, and all workloads were going to be run there. As we've gained experience, it's very clear that we're actually going to see a greater distribution of function, partly in response to a greater distribution of data. But what does that tell about the relationship between AI workloads, storage, and hybrid cloud? David, why don't you give us a little clue as to where we're going to go from here? Well, I think the first thing we have to do is separate out the two types of workload. There's the development of the AI solution, the inference code, et cetera, the dealing with all of the data required for that. And then there is the execution of that code, which is the inference code itself. And the two are very different in characteristics. For the development, you've got a lot of data. There is very likely to be data bound, and storage is a very important component of that, as well as compute and GPUs. For the inference, that's much more compute bound. Again, compute neural networks, GPUs are very, very relevant to that portion. Storage is much more ephemeral in the sense that the data will come in and you will need to execute on it. But that data will be part of the set, the compute will be part of that sensor. And you will want the storage to be actually in the DIM itself, or non-volatile DIM, right up as part of the processing. And you'll want to share that data only locally in real time through some sort of mesh computing. So very different compute requirements, storage requirements, and architectural requirements. Yeah, let's come back to that notion of the different storage types in a second. But Jim, David described how the workloads are going to play out. Give us a sense of what the pipelines are going to look like, because that's what people are building right now is the pipelines for actually executing these workloads. How are they different? How do they differ in the different locations? Yeah, so it's the entire DevOps pipeline for data science, data analytics, AI in other words. And so what you're looking at here is all the processes from discovering and ingesting the data to transforming and preparing and correcting it, cleansing it, to modeling and training the AI models, to serving them up for inferencing along the lines of what David's describing. So there's different types of AI models that one builds from different data to do different types of inferencing. And each of these different pipelines might be highly, all it is, highly specific to a particular use case. You know, AI for robotics, that's a very different use case from AI for natural language processing, embedded, for example, in an e-commerce portal environment. So what you're looking at here is different pipelines, but all have, all share a common sort of flow of activities and phases. And you need data scientists to build and test, train and evaluate and then serve out the various models to the consuming and devices or applications. So David, we've got 50 or so years of computing where the primary role of storage was to persist a transaction and the data associated with that transaction, that has occurred. And that's, you know, disk and then even all the way out to tape if we're talking about archive. Flash changes that equation. AI actually demands a different way of thinking. Here we're not talking about persisting the data, we're talking about delivering the data really fast. As you said, sometimes in very ephemeral. And so it requires a different set of technologies. What are some of the limitations that historically or that storage has been putting on some of these workloads and how are we breaching those limitations to make them possible? Well, if we take only 10 years ago, the start of the big data was Hadoop and that was spreading the data over very cheap disks, hard disks with the compute there and you spread that data and you did it all in parallel on very cheap nodes. So that was the initial but that is a very expensive way of doing it now because you're tying the data to that set of nodes. They're all connected together. So a more modern way of doing it is to use flash, to use multiple copies of that data but logical copies or snapshots of that flash and to be able to apply as many processes, nodes as is appropriate for that particular workload. And that is a far more efficient and faster way of processing that are getting throughput through that sort of workload. And it really does make a difference of 10 fold in terms of elapsed time and ability to get through that. And the overall cost is very similar. So that's true in the inferencing or I'm sorry, in the modeling, what about in the inferencing side of things? Well, the inferencing side is again, very different because you are dealing with the data coming in from the sensors or coming in from other sensors and smart sensors. So what you want to do there is process that data with the inference code as quickly as you can in real time, most of the time in real time. So when you're doing that, you're holding the current data actually in memory or maybe in what's called a non-volatile dim, NV-dims which gives you a larger amount but you almost certainly don't have the time to go and store that data and you certainly don't want to store it if you can avoid it because it is a large amount of data and if I open my... It's limited derivative use. Exactly. So you want to get all the, or quickly get all the value out of that data, compact it right down using whatever techniques you can and then take just the results of that inference up to other ones. Now, at the beginning of the cycle, you may need more but at the end of the cycle, you'll need very little. So, Jim, the AI world has built algorithms over many, many, many years, many of which still persist today but they were building these algorithms with the idea that they were going to use kind of slower technologies. How is the AI world rethinking algorithms, architectures, pipelines, use cases as a consequence of these new storage capabilities that David's describing? Yeah, well, AI has become widely distributed in terms of its architecture increasingly and often increasingly it's running over containerized Kubernetes orchestrated fabrics. And a lot of this is going on in the area of training of models and distributing pieces of those models up to various nodes within an edge architecture may not be edge in the internet of things a sense but widely distributed, highly parallel environments as a way of speeding up the training and speeding up the modeling and really speeding up the evaluation of many models running in parallel in an approach called ensemble modeling to be able to converge on a predictive solution more rapidly. And so that's very much what David's describing is that that's leveraging the fact that memory is far faster than any storage technology we have out there. And so being able to distribute pieces of the overall modeling and training and even data prep workloads is able to speed up the deployment of highly optimized, highly sophisticated AI models for the cutting edge challenges that we face like the Event Horizon Telescope, for example, that we're all aware of, when they were able to essentially make a visualization of a black hole, that relied on a form of highly distributed AI called grid computing, for example. I mean, challenges like that demand a highly distributed memory-centric orchestrated approach to tackling. So you're essentially moving the code to the data as opposed to moving all of the data all the way out to the one central point. Well, so if we think about that notion of moving code of the data, and I started off by suggesting that in many respects, the cloud is an architecture approach to how you distribute your workloads as opposed to an approach to centralizing everything in some public cloud. I think increasingly application architects and IT organizations and service providers are all seeing things in that way. This is a way of more broadly distributing workloads. Now, as we talk a little bit about the relationship between storage and AI workloads, but we don't want to leave anyone with the impression that we're at a device level. We're really talking about a network of data that has to be associated with a network of storage. Now, that suggests a different way of thinking about data and data administration and storage. We're not thinking about devices. We're really trying to move that conversation up into data services. What kind of data services are especially crucial to supporting some of these distributed AI workloads? So there is the standard ones that you need for all data, which is the backup and safety and encryption, security, control. Primary storage allocation. All of that, you need that in place. But on top of that, you need other things as well because you need to understand the mesh, the distributed hybrid cloud that you have and you need to know what the capabilities are of each of those nodes. You need to know the latencies between each of those nodes. Let's not be here for a second. When you say you need to know, do you mean I as an individual need to know or the system needs to know? It needs to be known and it's too complex, far too complex for an individual ever to solve problems like this. So it needs, in fact, its own little AI environment to be able to optimize and check that the SLAs for that particular inference coding can be achieved in the way that it's set up. So it sounds like- It's a mesh type of computing. Yeah, so it sounds like one of the first use cases for AI, practical commercial use cases, will be AI within the data plane itself because the AI workloads are going to drive such a complex model and utilization of data that if you don't have that, the whole thing will probably just fold in on itself. Jim, how would you characterize this relationship between AI inside the system and how should people think about that and is that really going to be a practical, near-term commercial application that folks should be paying attention to? Well, yeah, well, looking in the cloud native world, what we need and what we're increasingly seeing out there are solutions, tools, really data planes that are able to associate a distributed storage infrastructure of a very hybridized nature in terms of disk and flash and so forth with a highly distributed containerized application environment, so for example, just last week at Red Hat, I met with the folks from Robin Systems and they're one of the solution providers providing those capabilities to associate, like I said, the storage cloud with the containerized, essentially application clouds or cloud applications that are out there. You know, what we need, like you've indicated, are the ability to use AI to continue to look for patterns of, look for performance issues and bottlenecks and so forth and to drive the ongoing placement of data on storage nodes and servers within clusters and so forth as a way of making sure that storage resources are always used efficiently in that. SLA's, as David indicated, are always observed in an automated fashion as the data placement and workload placement decisions that are being made, so ultimately that the AI itself, whatever it's doing, like recognizing faces or recognizing human language, is able to do it as efficiently and really as cheaply as possible. All right, so let me summarize what we got this far. We've got that there is a relationship between storage and AI, that the workloads suggest that we're going to have centralized modeling, large volumes of data, we're going to have distributed inferencing, smaller volumes of data, more complex compute. Flash is crucial, the mesh is crucial, and increasingly because of the distributed nature of these applications, there's going to have to be very specific and specialized AI in the infrastructure, in that mesh itself, to administer a lot of these data resources. So, but we want to be careful here, right David? We don't want to suggest that we have, just as the notion of everything goes into a centralized cloud under a central administrative effort, we also don't want to suggest this notion that there's this broad heterogeneous, common democratized, every service available everywhere. Let's bring hybrid cloud into this. How will hybrid cloud ultimately evolve to ensure that we get common services where we need them and know where we don't have common services so that we can factor those constraints? So it's useful to think about the hybrid cloud from the point of view of the development, which will be fairly normal type of computing and be in really large centers and the edges themselves, which will be what we call autonomous clouds. Those are the ones at the edge which need to be self-sufficient. So if you have an autonomous car, you can't guarantee that you will have communication to it. And most, a lot of IOT is in distant places which are gained on ships or distant places where you can't guarantee. So they have to be able to run much more by themselves. So that's one important characteristic. So that autonomous one needs to be self-sufficient itself and have within it all the capabilities of running that particular code and then passing up data when it's... Now you gave examples where it's physically required to do that, but there's also OT examples, operational technology where you need to have that air gap to ensure that bad guys can't get to your data. Yes, absolutely. I mean, if you think about a ship, it has multiple very clear air gaps and a nuclear power station has a total air gap around it. You must have those sort of air gaps. So it's a different architecture for different uses, for different areas. But of course, data is gonna come up from those autonomous upwards, but it will be a very small amount of the data that's actually been processed, the data. And there'll be requests down to those autonomous clouds for additional processing of one sort or another. So they still will be a discussion, a communication between them to ensure that the final outcome, the business outcome is met. All right, so I'm gonna ask each of you guys to give me a quick prediction. David, I'm gonna ask you about storage and then Jim, I'm gonna ask you about AI in light of David's prediction about storage. So David, as we think about where these AI workloads seem to be going, how is storage technology going to evolve to make AI applications easier to deal with, easier to run, cheaper to run, more secure? Well, the fundamental move is towards a larger amounts of flash. And then the new thing is that larger amounts of non-volatile dim, the memory in the computer itself, those are gonna get much, much bigger, those are gonna help with the execution of these real-time applications. And there's gonna be high-speed communication between short distances, between the different nodes in this mesh architecture. So that's on the inference side, there's a big change happening in that space. On the development side, the storage will move towards sharing data. So having a copy of the data, which is available to everybody, and that data will be distributed. So sharing that data, having that data distributed, will then enable the sorts of ways of using that data, which will retain context, which is incredibly important, and avoid the cost and the loss of value because of the time taken of moving that data from A to B. All right, so to summarize, we've got a new level in storage hierarchy that puts between flash and memory to really accelerate things. And then secondly, we've got this notion that increasingly we have to provide a way of handling time and context so that we sustain fidelity, especially in more real-time applications. Jim, given that this is where storage is gonna go, what does it say about AI? What it says about AI is that first of all, we're talking about, like David said, meshes of meshes. Every edge node is increasingly becoming a mesh in its own right with disparate CPUs or GPUs and whatnot, doing different inferencing on each device. But every one of these, like a smart car, will have plenty of embedded storage to persist a lot of data locally that may need to be kept locally for lots of very good reasons, like a black box in case of an accident, but also in terms of e-discovery of the data and the models that might have led up to an accident that might have caused fatalities and whatnot. So when we look at what AI is going, and AI is going into the mesh of meshes, where there's AI running in each of the nodes within the meshes, and the meshes themselves will operate as autonomous decisioning nodes within a broader environment. Now, in terms of the context, the context increasingly that surrounds all of the AI within these distributed architectures will be in the form of graphs, and graphs are something that distinct from the statistical algorithms that we've built AI on. We're talking about knowledge graphs, we're talking about social graphs, talking about behavioral graphs, so forth. So graph technology is just getting going and for example, Microsoft recently had built, they made a big continued push into threading graphics, contextual graph technology into everything they do. So that's where I see AI going. It's up from statistical models to graph models as the broader metadata framework for binding everything together. Excellent, all right guys. So I'm gonna, Jim, I think another topic, another time might be the mesh mess, but we won't do that now. All right, let's summarize very quickly. We've talked about how the relationship between AI, storage and hybrid cloud is going to evolve. Number one, AI workloads are at least differentiated by where we handle modeling, large amounts of data, still need a lot of compute, but we're really focused on large amounts of data and moving that data around very, very quickly, but therefore proximate to where the workload resides. Great, great application for clouds, large, public, as well as private. On the other side, where the inferencing work is done, that's going to be very compute-bound, smaller data volumes, but very, very fast data, lot of flash everywhere. The second thing we observed is that these new AI applications are going to be used and applied in a lot of different domains, both within human interaction, as well as real-time domains within IoT, et cetera, but that as we evolve, we're going to see a greater relationship between the nature of the workload and the class of the storage, and that is going to be a crucial feature for storage administrators and storage vendors over the next few years, as to ensure that that specialization is reflected in what's needed. Now, the last point that we'll make very quickly is that as we look forward, the whole concept of hybrid cloud where we can have greater predictability into the nature of the data-oriented services that are available for different workloads is going to be really, really important. We're not going to have all data services common in all places, but we do want to make sure that we can assure, whether it's a container-based application or some other structure, that we can assure that the data that is required will be there in the context form and metadata structures that are required. Ultimately, as we look forward, we see new classes of storage evolving that bring data even closer to the compute side, and we see new data models emerging such as graph models that are a better overall reflection of how this distributed data is going to evolve within hybrid cloud environments. David Floyer, Jim Kobielus, Wikibon Alice. I'm Peter Burris. Once again, this has been Action Item.