 Power your data center in the Internet of Everything. Urgent. OK, we're going to get going. The first panel, and these are panels, which means this is, why is it that whistling does better than speaking? A question for, there you go. What's that? All right, so if we can start finding some seats again or getting ourselves organized, this next section is the audience participation section. So it's an opportunity for you to ask questions. The next two panels are going to be presented by or moderated by George Gilbert, who is Wikibon's big data analyst and has written recently on a number of different topics. At Wikibon, we are fascinated, as is everybody else, about the technologies that are part of the big data ecosystem, but we're especially interested, and I feel like I'm on a beach as the wind is blowing through my hair up here, we are especially interested in how over the course of the next few years, these big data technologies are going to be made available to developers to create applications. And so our big concerns are how are we going to make things simpler, how are we going to make things better from machine learning, et cetera, and very importantly, what will be the application forms that these take over the next few years? So don't forget, this is an opportunity for you to ask questions. Each of the panel members will be introduced by George, and then we'll spend some time having some kind of pre-formatted conversations, and if you have something you want to say, raise your hand, and we'll make sure we get to you. So with that, I'd like to introduce George Gilbert. Thanks, Peter. Which one's me? Okay, just I had my cheat sheet with all my names, because whenever I get up in front of a group, I'm pretty much blank. And I'm using Google Docs, and the names disappeared. Oh, right, okay. Okay, so let's bring our speakers out one by one. So coming first, Jim McHugh, who was prompting me. Jim, come on out. Once everyone's up here, I'm going to ask them to introduce themselves. Randy Swanberg from IBM, Distinguished Engineer, Power Systems, and Ram from Databricks, and then of course, Josh from SkyMind. All right, I just want to kick things off on a sort of big picture way, following up on the context that Peter set, which was that we're about helping people understand how to use data to create and sustain customers. And the big advances in computers come every X many years when there's a big relative shift in the price performance of compute network or storage. We went through the commodity direct attached storage, and that was, you know, our data lakes. And we went to moderately multi-core CPUs that helped us crunch through those. But the real big advance that's really impending right now is just the orders of magnitude greater number of cores that we have through GPUs as auxiliary sort of processing units, and that's going to change the type of computations we can do. Just by way of contacts, I want to say that in machine learning, many of us, especially humanities majors like me, think you drop a couple numbers in and get an answer out. The reality is there are many different levels of good answers. And when we have more compute power coming from GPUs, we can either look inside that black box better or we can have that black box search over a greater space and come out with better answers. So hopefully that's not too techy. And I did graduate in the top two thirds of my class. So having said that, let's start from the far end and have Jim introduce himself. So I'm Jim McHugh. I'm the vice president and GM of our DGX1 platform, but also responsible for driving really a lot of the analytics processing and things we're doing at NVIDIA. Okay, great. All right, and I'm Randy Swamberg. I'm a distinguished engineer with IBM, specifically in our power systems. Work a lot with open power partners like NVIDIA. And I work a lot around anything GPU accelerated from accelerated databases to deep learning and context today, Spark that we'll talk about. Hello, everybody hear me? Okay. So I'm Ram Sriarsha. I'm the product manager for Spark Databricks. So I lead the open source efforts around Spark Databricks. Prior to Databricks, I was a research scientist at Yahoo working in machine learning algorithms. That's the song. Yeah. I'm Josh Patterson. I'm the director of field engineer for SkyMind. I'm also the co-author of the deep learning book with O'Reilly with Adam Gibson right over there raising his hand. And together he and I have been working on DL4J for about three years now. In a previous life I was worked in the field for Clutter. Okay. So we're going to make this interactive kind of like a town hall, which is what Hillary did really well when she ran for Senate, but not quite as much in this race. So why don't we kick it off with Jim? At the most fundamental level, what's NVIDIA doing that enables us to rethink all the machine learning technology? Or if not to rethink it, to rethink how we apply it? Well, that's pretty much an easy question. I think reality, if you think about what's going on in machine learning, deep learning and AI, it's really changing rapidly. I mean, that's frankly why we're up here together, talking about Spark and deep learning. Most people in the room probably like, wow, I didn't know the guys at Databricks were even considering GPUs and acceleration. And it's because the industry's changing so fast, it's bringing us together, right? At Spark Summit, a lot of talk about deep learning, a lot of talk about AI is coming together. But the reality is, Spark is the platform that most people are using for their data management. It really facilitates it. So I see really customers, and I was actually really happy to see Lumiata call it out, but Spark and GPU right next to each other because they're two technologies that need to work together. So we're working through those kind of processes. Okay, so Jim, why don't you sort of pick on our panelists one by one and tell us, or ask them to explain where you see them fitting in the ecosystem. Okay, that's a good point. I'll start with Randy, because he's distinguished. Um. Ha ha ha ha. Extinguished. No, I actually think it's really interesting because we're talking about an ecosystem here tonight and that's really what we are. We have a lot of people that are working together and to have the DGX guy and the guy who's running Power, people ask, well why are you guys working close together? Because we have a lot in common. We actually see the ecosystem together. We know what the technologies are that need to be optimized. So we're kind of the guys that see most eye to eye, right? We're just a little bit different on what our products are but we're looking at it. So when you're looking at the industry, when you're looking at some of the technologies and things you're seeing in Spark or other, what do you see that needs to accelerate or the benefits you can bring your customers? So I think, I mean you said it when you talked about Spark. You know, the value proposition of Spark is really about simplification, unifying your data sources, time to value, productivity, you know, for somebody who's wanting to do distributed computing, it abstracts away a lot of the complexity to do that. You can actually focus on the workload that you're wanting to distribute at scale and a lot of the work that we're doing around GPUs in Spark is really trying to follow that paradigm with respect to transparency and give you an example. So some of the work we're doing is trying to accelerate Spark machine learning APIs without the Spark programmer even knowing it's happening because it comes down to a skills set. You don't have to learn CUDA, no offense, I mean. But you don't have to be a GPU programmer. You don't have to learn CUDA if we can transparently accelerate Spark machine learning libraries underneath the covers. And that's, you know, again, that's to advantage productivity to make it simpler to use and really make GPU exploitation, especially for something as popular Spark, more widespread. So let me jump in, Jim and Randy. Rum, as sort of sitting in the epicenter of all this and seeing how people are trying to use Spark machine learning right now, what qualitative change in the types of problems do you see them attacking now that not only can they scale out to a cluster, but they can really turbo charge that cluster? So that's a very good question. I think, you know, so Spark already has a lot of support for machine learning. A lot of it is distributed machine learning. Where I see GPUs fit really well is there are types of the certain classes of problems where, A, on a GPU, you can train much, much faster. Deep learning is a classic example of that. It's just a problem that's very well suited for single instruction multiple data, SIMD kind of processing, which is very much bread and butter for GPUs. But where I see Spark fit really well is also, even if you're learning on a single, even if you're learning a neural network on a single machine and if you have enough hardware to be able to train a neural network on a single machine, neural networks are actually fairly complex. What happens is that the size of the neural network is big and that's how they're able to learn on big data. But the problem is that neural networks require a lot of tuning parameters. So one obvious way you can speed up training is you can train multiple neural networks in parallel and actually pick the best set of hyperparameters. So this is called hyperparameter tuning and Spark is extremely good at that. And some of the logistics that go into figuring out the best hyperparameters is actually hidden from you by libraries like TensorFlow and so on. These are higher level libraries built on things like TensorFlow on GPUs and stuff like that, which we are working on. The other aspect of Spark and deep learning is also, let's say you learned a deep learning model on a single machine. Now you have to apply it to data. You're either applying it to data for doing prediction or sometimes they're applying it to data to come up with new features. This is called feature extraction. So think about looking at images and coming up with a feature representation of images. Your end goal could be just image classification, in which case you're applying the model to data to classify an image as cat or dog or whatever it is. But sometimes they're actually taking the model and applying it on an image to get an intermediate feature representation, which is used for a different task. In either of these scenarios, the inference part of it or the scoring part of it is actually completely parallelizable. So it fits extremely well in the framework of Spark, right? So you have a very nice mixed workflow here where even certain algorithms which can benefit from single machine massive bandwidth of CPUs is still amenable to the framework of Spark. So that's kind of why we are excited about it, right? So it brings the best of both worlds of Spark as well as GPUs. Okay, Josh, you're up. Some great answers up here. I want to tell my story from the perspective of someone who stood up a lot of Hadoop clusters back in the early Hadoop days. I ran Hadoop before there was distros and then this little shop called Clare came along and I'm like, this is a good idea. I don't know how we're going to make money, but I think this is a good idea. And we stood up a lot of Hadoop clusters and found out what the Fortune 500 wanted to do. It was Google-like, not quite Google. Adam comes along and I was like, we need to do this again, but with deep learning. These research things are coming out and they're good, but we've got to make this in a way just like HDFS and MapReduced it. And so we came up with DL4J and I'm coming right to Spark. And then he said, we need to make the back ends agnostic and we need to make the JVM fast. So he came up with what's called ND4J, which is our, we can run our linear algebra on the CPU or GPU, right? So that makes it interesting because the same JVM code runs fast on the CPU or the GPUs. And then we get the parallelization from Spark, Databricks, Clare, and Horton. And so now we have a much cleaner interface to run in the Fortune 500. This is where it gets interesting if I'm an AE and I've spent a lot of time with AE's because now companies can build nice applications like they did on Hadoop with DL4J and ND4J. They leveraged Spark very easily and then they say, our use case can benefit from going faster. GPUs let them do linear algebra the same things, but faster. And so they don't change their code and an AE for a GPU company can say, if you just plug in, I don't know, a lot of GPUs in that Hadoop cluster, don't change your code, change your Maven profile. Now they have a reason to sell a whole lot of GPUs into the data warehouse. And I guess at an NVIDIA event that might be a little bit interesting, but, oh. Okay. All right, so like I said, we wanna do Hillary style and I know all of you are gonna be busting down the doors to see how the debate went and I almost played hooky. So why don't we take some questions from the audience to try and keep people here and engaged while hopefully politically incorrect. I'm gonna say I hope she's winning. Anyone have any questions for our panelists? That's a thundering affirmative. I have a question. Okay. So Ram, what are we doing to work together? So first of all, you know, Databricks, if you're not aware of us, we are the primary contributors to Spark ecosystem. We were started by the company, I mean, we came out of the AMP Lab which created Spark. Now, we are a SaaS platform, so we wanna make it really simple for people to explore big data. Our motto is basically big data simple. And part of big data analytics, increasingly a part of big data analytics is deep learning, right? So there's a lot of interest in using GPUs for deep learning. So we are actually working very closely with NVIDIA to actually build in deep integration with Databricks to offer GPU support. And by this, I don't just mean making GPU instances available, but making them available with NVIDIA drivers, the runtime and things like TensorFlow compiled on GPUs and so on. So if you are prototyping deep learning algorithms, you can get started on it very, very easily, right? The ease of use of platforms is very important for us. But we are also doing things like figuring out, looking across a section of our customer use cases and so on. We're trying to figure out what are the types of problems or use cases that are really benefit from GPU acceleration, right? It's kind of, today it's very well known that image classification and to an extent, language translation and things like that are very nice use cases for deep learning. But we are just getting started with GPU acceleration. So there may be use cases out there which are still amenable to the GPU acceleration that we have not yet discovered. So part of what we are doing in this process is actually figuring that out. So one more question, Ram, and not to pick on you, but the initiative to take advantage of GPUs and storage class memories was this project called Tungsten. How far along are we with Spark 2.0 and what might we see in exploiting GPUs further over the coming releases? Yeah, that's actually a very good question. So the history of Tungsten was really, when Spark started off, the API was actually RDDs, Resilient Distributed Data Sets. The nice thing about RDDs was that it was a very fluent API, so you could really think about high level abstractions and not worry about how the actual data was distributed. But one of the things that it sacrificed in that process was optimal representation of data. So you want your memory representation to be as tight and optimal as possible and Tungsten was the effort to make that happen. And to do that, we had to introduce this concept of data frames. So data frames are the level of abstraction kind of between the execution engine as well as the programmer that allows us to actually have things like Tungsten behind it. Now, I briefly talked about tensor frames. So tensor frames is the effort to bring GPU acceleration to Spark and it actually works on top of data frames. The reason why it's written on top of data frames is to actually eventually use Tungsten. So Tungsten allows you to manage memory really well. Now, when you're thinking about taking data and running portions of computations on GPU, you want to minimize the amount of data transfer that's happening from the system memory into the GPU. If you're doing this on top of RDDs today, there's already a serialization that goes on between RDD data structures to native memory to what's the local memory on GPU. So there's a lot of layers of kind of serialization that are going on. With Tungsten, the hope is that there's only one level, one layer of serialization that goes on, basically moving data from system memory onto the GPU and that's unavoidable, right? And how much more is there left to do? We are set up to do that. So basically with Spark 2.0, everything is set up with tensor frames. We have the right level of abstraction. The next thing to do is actually columnar support. So basically if you operate on a set of rows at a time, instead of a row at a time, that's the right level of abstraction that's needed and that's coming. That's what will make all this kind of tie together. All right, guys, I think we're... Question? Oh, okay. All right, this will have to be our first and our last as we're getting a hook. So one question about computer vision and data frames. So how do you do select star from pixels? Because one of the problems with the data frame abstraction is SQL. How do you do select star from pixels? NVIDIA's primary use case has been self-driving cars and computer vision, which doesn't map all the SQL. So how do you combine those things and what are you doing to fix the data frame abstraction so that it can work well with GPUs and computer vision? So that's a good question. I would actually say you don't want to do select star on pixels, right? If you think about it, pixel is just a vector, right? So if I simplify it and I have only three colors, I can think of a pixelated image as three colors at every point, every pixel on an image. That's a vector in certain number of dimensions, right? So your data representation is actually just a vector. It's not any new data structure. It's a data structure that pretty much every database knows about. So you can certainly do SQL on top of that, but the goal of deep learning is not to do SQL on top of this. And in fact, data frames are actually an abstraction that's SQL-like, but they are very much beyond SQL. Because data frames allow you to do user defined functions, aggregate functions, TensorFlow and data frames and so on. So you can actually do fairly complicated DAGs of operations on top of data frames. You shouldn't really be thinking of them as just traditional vanilla SQL, right? But that said, a pixel is from our perspective just a vector, right? And on that vector, you can do some transformations to make it lower dimensional or you can do some transformations to figure out what is this vector map to as a label, right? So that's where the deep learning part of it comes in. All right, guys. We're gonna have to pause it there until next year at this time. So Jim, Ram, Randy and Josh, thanks very much.