 Hi, this is Peter Burris. Welcome to Wikibon Research's Friday research meeting on the Q. Today we're going to talk about something that's especially important given the events of this week. As many of you know, Apple announced the new iOS 11 and a whole bunch of new devices. Now, we're not going to talk about the devices so much, but rather some of the function that's being introduced in iOS 11, specifically things like facial recognition. An enormous amount of processing is going to go into providing that type of service on devices like this. And that processing capability, those systems capabilities are going to be provided by some new technologies that are related to artificial intelligence, big data, and something called deep learning. And the challenge that the industry is going to face is where will this processing take place? Where will the data be captured? Where will the data be stored? How will the data be moved? What types of devices will actually handle this processing? Is this going to all end up in the cloud? Or is it going to happen on increasingly intelligent and smart devices? What about some of the different platforms? And ultimately, one of the biggest questions of all is how are we going to bring some degree of consistency and control to all of these potentially distributed architectures, platforms, and even industries as we try to weave all of this into something that serves all of us and not just a few problems. Now, to kick this off, Jim Kobielus, why don't you start by making a quick observation on what we mean by deep learning? Yeah, thank you, Pierre. Deep learning, the term has been around for a number of years. It's actually machine learning, but with more layers of moron and able to do higher-level abstractions from the data. Abstractions such as face recognition, natural language processing, and speech recognition and so forth. So when we talk about deep learning now as a client, to what extent can more of these functions, face recognition as in iOS 11 or iPhone 10, to what extent will this technology, this capability be baked into all edge endpoints now? Jim, I'm having a little bit of a problem hearing you. So maybe we can make sure that we can hear that a little bit better. But very quickly and very importantly, it suggests that the term deep learning suggests something a little different than I think we're actually going to see. Deep learning suggests that there is going to be a centralization of function for some process. It's going to be the ultimate source of value. And I don't think we mean that. When we talk about deep learning, let's draw a distinction between deep learning as a process and deep learning as a set of systems and designs and investment that's going to be made to deliver on this type of business function. Does deep learning fully capture what's going to happen here? Is this for me, Peter? Can you hear me, Peter? I can hear you better now. I can hear you better now. I'm a little bit sassy. Okay, I got my earbuds in. Yeah, essentially the term deep learning is a broad paradigm that describes both the development pipeline functions that quite often will, more often than that, will be handled in the cloud among distributed teams. And those functions of deep learning that can be brought to the edge, to the end devices, the mobile devices, the smart sensors. When we talk about deep learning at the edge, as enabled through chipsets, we're talking about functions such as local sensing, local inference from the data that's being acquired there, local actuation as we're taking actions like an autonomous vehicle steering right or left based on whether there is an obstacle in their path. So really in the broadest sense, you need that full infrastructure to do all the building and the tuning and training of deep learning models. And, of course, you need the enabling chipsets and tools to build those devices, those functions, deep learning functions that need to be pushed for local, often autonomous execution at end points. So, David, that strongly suggests that, in fact, deep learning is suggestive of a new system architecture model that is not going to be largely centralized but rather is going to be dependent upon where data actually exists and how proximate it is to the set of events that we're both monitoring and ultimately trying to guide as we think about new automation, new types of behavior. Take us through our thinking on some of these questions of where the data is going to reside, where the function is going to reside, and ultimately how the architecture is going to evolve. I think you're on mute, David. Yes, I would put forward the premise that the majority of the processing of this data and the majority of the spend on equipment for this data will exist at the edge. And Neil brought forward a very good differentiation between second-hand data, which is where big data is today, and primary data, which is what we're going to be analyzing and taking as decisions on at the edge. As sensors increase the amount of data and smart sensors come, so we're going to see more and more of the processing shift from the traditional centralized to the edge and taking Apple as another example. They are doing locally all of this processing of data. Siri itself is becoming more and more local as opposed to centralized. And we're seeing the shift of computing down to the edge. And if we look at the amount of computing we're talking about, we're talking with the Apple 10, 600 billion operations per second. That's a lot of computing power. We see the same thing in other industries, the self-driving car. If you take the Nvidia Drive 2, it has a huge amount of computing power within that to process all of the different sources of data in a device which is costing less than $1,000, $600, $700. So much lower pricing of processing, et cetera. Now, the challenge of data, the traditional model, is that all of the data goes to the center. Is the cost of all this data moving it from the edge to the center is just astronomical. It would never happen. So only a subset of that data will be able to be moved. And people who develop systems, AI systems, for example, at the edge, will have to have simulation factories very local to them to do it. So car manufacturers, for example, having a small city, if you like, where they have very, very fast communication devices. And the amount of data that can be stored as well from this new primary source of data is going to be very, very small. So most of that data either is processed immediately or it disappears. And after it's processed, in our opinion, most of that will disappear. 99% of that plus will disappear completely. So the traditional model of big data is being turned upside down by these new and prolific sources of data. And the value will be generated at the edge. That's where the value is in recognizing a bad person coming into a building or recognizing your friends or recognizing that something is going wrong with a smart center locally. The vibrations are too high. Or whatever the particular example is, that value will be generated at the edge by new classes of people and new classes of... So one of the interesting things we're talking about here is that we're talking about some pretty consequential changes in the nature of the applications and the nature of the architectures and infrastructures that we're going to build to support these applications. But those kinds of changes don't take place without serious consideration of the business impacts. Is this something that companies are going to do kind of willy-nilly? How deeply are companies going to have to think about? How deeply are users going to have to think about deploying these within their business? Because it seems like it's going to have a pretty consequential impact on how businesses behave. Well, they're going to need some guidance because there just aren't enough people out there with the skill to implement this sort of thing for all the companies that they want to do it. But more importantly than that, I think that our canonical models right now for deep learning and intelligence at the edge are pretty thin. You know, we talk about autonomous cars or facial recognition, something like that. There's probably a lot more things we need to think about and from that we can derive some conclusions about how to do all this. You know, when it comes to the persistence of data, there's a difference between a B2C application where we're watching people click and deciding, you know, next best offer and anything that happened a few months ago was irrelevant. So we can throw that data away. But when you're talking about monitoring the performance of an aircraft in flight or a nuclear power plant or something like that, you really need to keep that data, not just for analytical purposes, probably for regulatory purposes. In addition to that, you know, if you get sued, you want to have some record of actually what happened. So I think we're going to have to look at this whole business and all of its different components before we can categorically say, yes, we saved this data, here's the best application, everything should be done in the cloud. I don't think we really know that yet. But the issue that's going to determine that decision is going to be a combination of costs today, we know that those costs are going to change over time and knowledge of where people are and the degree to which people really understand some of these questions and ultimately what folks are trying to achieve as they invest to get to some set of objectives. So there's probably going to be a difference in the next few years between, in which we do a lot of learning about deep learning systems and some steady state that we get to. My guess is that the ecosystem is going to change pretty dramatically between now and then. So it may be the teleco think that they're going to enjoy a bonanza on communications costs over the next few years as people think about moving all this data. If they try to do that, that's going to have an impact on how Amazon and Google and some of the big cloud suppliers invest to try to facilitate the movement of the data. But there's a lot of uncertainty here. Jim, why don't you take us through some of the ecosystem questions what role will developers play? Where is the software going to end up and to what degrees is going to end up in hardware and is this going to lead to or catalyze kind of a renaissance in the notion of specialized hardware? Yeah, that was a great question. I think most of this functionality, meaning the local sensing in inference and actuation is inevitably going to end up in hardware and highly specialized and optimized hardware for particular use cases. In other words, smart everything, smart appliances, smart clothing, smart lamps. What's going to happen is that what we call deep learning will just be built in by designers and engineers of all sorts regardless of whether they have a science or a computer background. So it's becoming part of the material fabric of reality for bringing intelligence in. With that said, then, if you look at the chip set architectures and if we can use this from a chip set here, that will enable this vast palette of embedding of this intelligence in physical reality. The jury is really out about whether it will be GPUs like NVIDIA, of course. This is the power up I and GPUs versus CPUs versus FPGAs, ASICs. There are various neuromorphic chip sets from IBM and others. It's clearly going to be a fairly very innovative period of great ferment in innovation in the underlying hardware substrates, the chip sets, to enable all these different use cases in embedding of all of this. In terms of developers, triple software developers, definitely they're still very much at the core of this phenomenon. When I say they, data scientists in the new era who are the ones who are building these convolutional neural networks and recurrent neural networks and long short term and so forth, all these PL algorithms very much are the province of data sciences for the new generation of data scientists who specialize in those areas and who work hand in hand with traditional programmers and so forth to put all of this intelligence into a shape that can then be embedded or might containerized or whatever and brought into some degree of harmonization with the physical hardware layer into which hardware can be used for terms like smart clothing. What we're getting is that now we have a new era where the collaborations are going to be diverse among nontraditional job or skills categories who are focused on bringing AI into everything that touches our lives. It's wide open now. So David, let me throw it over to you because Jim's raised some interesting points about where the various propositions value propositions and how the ecosystem is going to emerge. This sounds like once again going back to the role that consumer markets are going to play from a volume cost and driving innovation standpoint is are we seeing kind of a repeat of that or the economics going to of volume going to also play a role here? Judith? Yes, I believe so very strongly. If you look at technologies and how they evolve, if you look for example at Intel and how they became so successful in the chip market they developed the chips with Microsoft for the PC that was very very successful and from that they then created the chipset for the data centers themselves. When we look at the consumer volumes we see a very different marketplace. For example, GPUs are completely winning in the consumer market. So Apple introduced GPUs into their ARM processes this time around. NVIDIA have been very very successful together with ARM in producing systems for self-driving cars. Very very powerful systems. So we're looking at new architectures we're looking at consumer architectures that in NVIDIA's case came from game playing and ARM has come from the distributed ecosystem, the client, etc. or ARM base. We're seeing that it's likely that consumer technologies will be utilized in these ecosystems because volume wins. Volume means reduction in price and when you look at for example the cost of an ARM processor within an Apple iPhone it's $26 90 cents. That's pretty low compared with the thousands of dollars you're talking about for a processor going into a PC. And when you look at the processing power of these things in terms of operation they're actually greater power. And same with NVIDIA with the GPUs. So yes, I think there is the potential for a big big change and a challenge to the existing vendors that they have to change and go for volume and pricing for volume in a different way than they do at the moment. So that's going to have an enormous impact ultimately on the types of hardware designs that we see emerge over the course of the next few years and the nature of the applications that the ecosystems willing to undertake. I want to pivot and bring it back to the notion of deep learning as we think about the client because it ultimately describes a new role for analytics and how analytics are going to impact the value propositions the behaviors and ultimately the experience of consumers and everybody has with some of these new technologies. So Neil, what's the difference between deep learning related analytics on the client and a traditional way of thinking about analytics? Take us through that a little bit. Deep learning on the client? You mean at the edge? Well, deep learning on the client, deep learning on the edge, deep learning out away from the center. When we start talking about some of this edge work what's the difference between that work and the traditional approach for data analytics, data warehousing, etc. Well, you know, my naive point of view is deep learning involves crunching through tons of data in training models to come up with something you can deploy. So I don't really see deep learning happening at the edge very much. I think David said this earlier that deep learning is happening in the big data world where you have trillions of observations to use. Am I missing your point? No, we talked earlier about the difference between deep learning as a process and deep learning as a metaphor for the process of systems. So when we think about utilizing these technologies whether it's deep learning in AI or whatever we call it and we imagine deploying more complex models close to the edge what's that mean from the standpoint of the nature of the data that we're going to use the approach, the tooling that we're going to use the approach we're going to take organizationally and institutionally to try to ensure that that work happens. Is there a difference between that and that? Well, there's a difference in terms of the technology. I think that ten years ago we were talking about complex event processing. The data wasn't really flowing from sensors it was scraping web screens and that sort of thing but it was using decision making technology to look for patterns and pass things along. But you have to look at the whole process of decision making if you're talking about commercial organizations commercial organizations it requires complex real time decision making except you know making decisions about supply chain or shop floor automation or that sort of thing but from a management point of view it's not really something that you do. The other part of decision making that troubles me is I wrote about this ten years ago and that was we shouldn't be using any kind of computer generated decision making that affects human lives and I think you could even expand that to living things or harming the environment and so forth. I'm a little bit negative about things like autonomous cars. It's one thing to generate a decision making thing that issues credit cards and maybe it's acceptable to have 5% or 3% of the decisions just completely wrong but that many wrong in autonomous driving especially trucks the consequence is a disastrous. We have to be really careful about this whole thing with IoT. We've got to be a lot more specific about what we mean, what kinds of architectures and what kind of decisions we're trying to make. I think it's a great point Neil. There's a lot that can be done and then the question is we have to make sure that it's done well and we understand some of the implications and I think there's a difference between a transition period in the steady state. We're going to see a lot of change over the next few years. Technology is making it possible to do so but there's going to be a lot of social impacts that ultimately have to be worked out and we'll get to some of those in a second but George Gilbert I wanted to give you an opportunity to talk a little bit about the way that we're going to get this done talk about where's this training going to take place for what Neil said. Is the training going to take place at the edge? Is the training going to take place in the cloud? Institutionally what is the CIO and the IT organization have to do to prepare for this? I think the widespread consensus is that the inferencing and predicting for low latency actions will be at the edge and some smaller amount of data goes up into the cloud training but the class of training that we will do over time changes and we've been very fixated on the data centricity like most of the data at the edge a little bit in the center and Neil has talked about secondary or reference data to help build the models in the center but the models themselves that we build in the center and then push out will change in the sense that we look at the compute intensity the compute intensity of the cloud will be will evolve so that it's more advantageous there to build models that become rich enough to be like simulation. So in other words it's not do I, if I see myself drifting over the lane marker on the right, do I correct left? But you have a whole bunch of different knobs that get tuned and that it happens over time so that the idea of the model is it's almost like a digital twin but not of let's say just an asset or physical device but almost like a domain and that that model it's very compute intensive it generates a lot of data but then the model itself can be distilled down and pushed out to the edge or essentially guiding or informing decisions or even making decisions with a lot more knobs than you would have with a more simplistic model. So Ralph I know that we've spent some time looking at some of these some of the market questions of this based on this conversation can you kind of give a summary of how much data volume we think is happening data movements happening what's the big broad impact on some of the segments and opportunities over the course of the next couple of years. I think if you think about maybe 10 years the amount of unstructured data and that was out there was not all that great obviously in the last 10 years of war there's a lot more of it so the growth of data is dramatically increasing most of it is going to be in the mobile area so there's just a lot of it out there and I think fishing for where you derive value from that data is really really critical for moving optimization of processes forward but I think I agree with Neil that there's a lot of work to be done yet about how that actually unfolds and there's also a lot of work to be done in areas like what will the role of who's going to help define how a lot of these platforms are going to be integrated together what's the role of standards what role will government play there's an enormous number of questions here but one thing we all agree on ultimately is that this is an emerging source of or this technology is an emerging source of dramatic new types of business value taking on problems that we've never thought about taking on before and it's going to have an enormous impact on how IT organizations work with business how they work with each other how businesses work together this is the centerpiece of the new digital business transformation alright so let me summarize this week's findings the first observation we make is that this week Apple introduced facial recognition directly in iOS 11 and it wowed much of the industry and didn't get a lot of people excited for a variety of reasons but it does point to the idea that increasingly we're going to see new classes of deep learning, AI machine learning and other big data type technologies being embedded more deeply in systems as a way of improving the quality of the customer experience improving operational effectiveness and efficiency and ultimately even dramatically improving the ratio between product and service revenue in virtually everything that we can think about now that has led folks to presume that there's again going to be this massive migration of workload back into the cloud both from a data standpoint as well as from a workload standpoint but when we stop and think about what it really means to provide this value it's pretty clear that for a number of reasons including real time processing to provide these services, the cost of moving data from one point to another and that the characteristics of the intellectual property controls etc restricts the pressure to try to move all this data from the edge, client and device back into the cloud and if the new architectures increasingly are going to feature a utilization of dramatic new levels of processing on devices we observe for example that the new iPhone is capable of performing 600 billion instructions per second that's an unbelievable amount of processing power and we're going to find ways to use that up to provide services closer to end users without forcing a connection this is going to have enormous implications overall in the industry questions for example like how are we going to institutionally use the development flow we think we're going to see more model building at the center with a constrained amount of the data and more execution of these models at the edge but we note that there's going to be a transition period here there's going to be a process by which we're learning what data is important what services are important etc we also think it's going to have an enormous impact for example on even describing the value proposition if everything is sold as a product of moving the data the cost of liability etc on these devices is going to be extreme it's going to have an enormous impact on the architectures and infrastructures we use but we think in terms of services that might have a different or lead to a different set of ecosystem structures being put in place because it will change the transaction costs the service provider perhaps is going to be more willing to move the data because they'll price it into their service ultimately it's going to have a dramatic impact on the organization of the technology industry the past 25, 30, 40 years have been defined for the first time by the role that volume plays within the ecosystem where Microsoft and Intel were the primary beneficiaries or were primary beneficiaries of that change as we move to this notion of deep learning and related technologies at the edge providing new classes of behavior it opens up the opportunity to envision a transitioning of where the value is up and down the stack and we expect that we're going to see more of that value put directly into hardware that's capable of running these models with enormous speed and certainty in execution so a lot of new hardware gets deployed and then the software ecosystem is going to have to rely on that hardware to provide the data and build the systems that are very data rich to utilize and execute on a lot of these mainly ARM processors that are likely to end up in a lot of different devices in a lot of different locations in this highly distributed world the action item for CIOs is this this is an area that's going to ensure that a role for IT within the business as we think about what it means for the business to exploit some of these new technologies in a purposeful and planful and architected way but it also is going to mean that more of the value moves away from the traditional way of thinking about business systems with highly stylized data to a more clear focus on how consumers are going to be supported devices are going to be supported and how we're going to improve the definition of more distributed high quality processing at the edge utilizing a new array of hardware and software within the ecosystem alright so I'm going to close out this week's Wikibon Friday research meeting on the Cube and invite you back next week where we'll be talking about new things that are happening in the industry that impact your lives and the industry