 from San Jose, in the heart of Silicon Valley. It's theCUBE, covering Big Data SV 2016. Now your hosts, John Furrier and Peter Burris. Okay, welcome back everyone. We are live in Silicon Valley for Big Data Week, which is comprises of Big Data SV, our event, and Strata Hadoop going on right across the street. We've been here every single year for Hadoop World, this is theCUBE. This is SiliconANGLE Media's flagship program. We go out to the events and extract the signal from the noise. I'm John Furrier, my co-host this segment, the analyst segment, the power analysts here, analyzing all the Big Data, George Gilbert, Wikibon analysts, and Peter Burris, head of researchers at SiliconANGLE Media, Wikibon and guys, the analyzing, plenty to analyze here at Big Data as this transformation happens in real time. It's streaming out there. All the data is real time. We're seeing that right now. This industry is exploding. Guys, I want to get your thoughts on that. Peter, first I want to get your take on it. I'll see you're new to the team. Congratulations, welcome aboard. But you're not new to the industry, you have a lot of experience. You've seen this movie before, okay? We've seen transformations, this game changing, these inflection points. These are the moments that everyone's watching because this is where the rubber hits the road. This is where the weed is separate from the shaft. This is where the people put their big pants on, they go to the party. This is where the value is created. This is what's happening. This is the show that will be the bellwether for the future of innovation and wealth creation from entrepreneurs and customers, their customers, customers, your thoughts. What are the key trends that you're seeing here and analyze the market? Well, I think you said where the wealth gets created or where the value gets created or not. Cause that's the other side. I think one of the big transformations that we're witnessing now in the Big Data universe is a not so subtle shift from talking about the technology to talking about the problems. And I think that most of the folks who have been at the vanguard and cutting their teeth and suffering the failures associated with some of these initial forays in the Big Data are now helping to articulate what is that next class of use case going to be as we think about applying the technology to the business problems, they're gonna drive business change and improve customer engagement. That is a crucial junction point in any marketplace when folks, especially a technology marketplace, when folks pivot from worrying about the technology in a narrow sense to worrying about the problems that the technology's trying to solve. And it's crucial because on the one hand, that's how we learn about how to build the next round of technology. But it also means that it becomes incumbent upon the enterprise to start thinking about how to generate more consistent, reliable value out of what they already have, wherever it might be. And one of the real bellwothers of that is people start to see technology fall into different patterns so that we can start to say, oh, okay, well before we just kind of deployed it and we put it out there in this way, now we're starting to see, again, at the vanguard, those leaders start to identify patterns of usage that in between the technology and in between the business problems, so those patterns are enabling us to go after new classes of business problem. That's where we are right now is where the, what are those application patterns? So let's get the top three things that we've been seeing. I want to get your thoughts on this. The maturation, the maturing of the technology, you're starting to see full stack developers are emerging as essentially the developer of choice and open source has been a great driver of this trend and also this notion that the maturing tech combined with the cloud and combined with the applications is the perfect storm for innovation. Talk about your take on that one point of the maturing of the technology and what is that impact to the vendors out pitching stuff and their customers, because they're the ones who are thinking about their business. You know one buys products because of the vendors. People want to know what's going on with their business. Right, and so one of the things that I think we're seeing is that that first round of, well let's step back even a little bit further. Data warehousing was a relatively successful way of going about being able to answer new classes of questions. So the data was being generated by these systems of record. We loaded it up in data warehousing with the right set of models and it allowed individuals to be able to quite frankly answer questions that before nobody could even think of answering. But we identified and found the limits to how that was gonna work. You had to know something about the questions in advance. You had to be able to, because you had to be able to model it. A new class of technology was created, Hadoop, and related stuff within that stack to be able to then take on the questions that we might not have known in advance with data just coming in from a lot of different locations and then using that as a basis to discover new insights. That's what we've been doing for the past 10 years. One of the last guests just mentioned that Hadoop is 10 years old this year. Well, we're not going into the teenage years, right? And you're a father. When you start thinking about teenagers, you start thinking about how do I allow them to go their own way while at the same time I bring enough control and discipline on them so that they don't go kill themselves. And I think that's where we are in a lot of this place. And it's leading to a new way or new classes of technology to fill in the problems associated with it. There's so many puns I could put on this. Frontal loaves not developed in this industry. Bad judgment. But they're growing. But again, George, I wanna get your take because there's two other points. There's maturing the tech as one. And one thing that's coming out of the Wikibon research is two other major points I wanna get your take on. A new kind of data, engagement data that's moving to a precognitive or cognitive or machine learning phase. And then three, the operationalizing, the business path to value. So second one, George, on engagement data, because the data is where the action is. We're seeing all the top minds in the industry talking about this data layer being the new middleware, glue. And we've been saying, David, I've been talking on theCUBE since we started that this is where the battleground's gonna be. And software, certainly, we're seeing that play out. But you are looking at the data. What is this new engagement model? What's the intelligence behind it? How do companies figure this out and what are vendors offering? Well, I guess, backing up to reframe before diving in to your question. For 50 years, we've had these systems of record which were automating more and more of the internal processes in companies. The processes were well known. They were often regulatory or they were so repeatable that the vendors could codify them. And there was essentially no advantage in doing them any differently from your competitors so that we could have packaged apps. And if people customized a little around the edge, that was fine. But the new class of apps relates increasingly to external facing things. Jeffrey Moore's been talking for years about systems of engagement. For instance, so when we've got a customer facing application and we're taking them through, say a multi-channel engagement model, we don't know how to codify exactly how that should work. And so we rely on data to tell us not a definitive answer, not to come up with a definitive answer from an application, but to anticipate what's likely to happen and to influence them. And so rather than the applications being really process heavy where the process was baked in in advance, we have data that we accumulate in each enterprise and that data guides then how a firm might interact and personalize with each individual customer engagement or employee interaction or partner engagement. So that's why we don't see at a big picture level packaged applications accelerating and taking this market off on a new growth curve, at least not yet. We don't know enough about them. As Peter says, invoking that illustrious Defense Secretary, Donald Rumsfeld, we don't know our unknowns yet. So we can't quite codify them. Yeah, it's one of those things. It's like, Peter, we were talking about completeness, right? How that doesn't really mean anything in this new era because the data's out there, we don't know what we don't know. So that's an interesting dynamic. But you talk about that because I've been asking the guests about completeness and I'm kind of baiting them a little bit because what does that mean? I mean, it doesn't really mean anything because if data is coming in so fast, the old model of data warehousing and business intelligence makes sense of the data. Okay, it makes sense to the data, check the box, we are complete for a nanosecond, then boom, more data comes in. So you're never gonna be fully complete. Notions of perishable data's coming out, we're hearing birds like that. Is it all BS or what's going on with this notion of completeness? It's not BS, but it really, what it really brings out, John, is that it's essential that we recognize the relationship between data arriving and insight and learning and then people going out and saying I need more because I just discovered what I don't know. And I think that's one of the biggest challenges. I like to talk about, as George mentioned this notion of the Rumsfeld model of analytics where I know what I know is kind of reporting. I know what I don't know is kind of data warehousing where I can model something in advance but I know what I'm trying to model even if I don't know exactly what the questions I'm gonna ask are. And then I don't know what I don't know is really what data warehousing or what this big data is all about. And as I discover what I don't know, I'm constantly looking for new sources of data from unanticipated sources and finding ways to bring it together and turning it into those crucial business nuggets they're gonna drive difference in how customers behave. So is that the notion what people talk about with machine learning, surfacing insights? Even though before that it was, when I was listening to Peter describe it and you're about describing the machine learning, Hadoop actually took shape at Yahoo. At Google it was really developed for searching or crawling the web and indexing that but it took shape at Yahoo when they had these huge data warehouses and they had questions they couldn't answer. And so they had to unravel essentially the pipeline that fed the data warehouse then they had to redesign the data warehouse and someone said, well, can't we take this Hadoop technology that Google's written about and do a better job of our unknown unknowns? And that's where it started and that's what most people are experimenting with in data lakes today. Well, let's take that example because Yahoo was kind of an outlier. All these web scale companies that build their own systems we know about Facebook and these guys all doing it. They didn't have a lot of, they had a lot of cash and a lot of expertise and they did it in-house, they didn't go out and buy off the shelf. And that's a historical thing but that was an operational challenge and this brings up the point about, okay, I can replicate some sort of web scale environment where I can have a data lake and put stuff in whether it's a data warehouse model for known queries that can make sense of it but now you've got real-time data, I got insights that need to be serviced and now I have impact to my business. This is where the top line revenue could grow, efficiencies could be established. Peter, operationalizing it seems to be the challenge. That's the common thread in all of these examples whether you go prehistoric big data which I would say is the Yahoo era and then now the man is standing up tall go from cavemen to humans. That's where we're at now. Peter, your thoughts on this because at the end of the day if you can't operationalize it doesn't exist. And if you can't operationalize you don't turn it into business and that because really the whole point is to can we make new and improved and extending commitments to business that all of this or to customers that all of this ends up supporting? And so when we think about this John what we're really talking about is the idea that data is becoming an asset. We like to say that in the realm of digital business data is your capital. So data is digital capital. And to put in place a set of capabilities that allows the business to operate consistently for clients and for partners and within employees utilizing some of these new technologies is going to require an enormous amount of effort to start bringing some disciplines in in ways that harken back to the era of data warehousing and good data hygiene and whatnot without undermining the fundamental proposition that we are always trying to create new insights with this data. So that's going to be the tension over the next few years is how do we bring new disciplines that we've learned from using from data warehousing as well as from things like a dupe as we start to bring in streaming technologies and new classes that we haven't even thought about or talked about yet. And in a way that facilitates more individuals entering into this ecosystem to create business value out of the results. Talk about let's shift gears to the suppliers out there and their vendors, suppliers, what do you want to call them? And then their customers, people who take the solutions and put them into practice. What are you guys seeing out there? Because this is again what customers care about what they can do with the products. As things are being operationalized, can you share what you've found, what's your analysis of the market, both the vendors slash suppliers and then the people who put them into practice for operational benefits for their business? Thoughts on the market? Analyze some of the horses on the track. Yeah, so tomorrow, Wednesday, we're actually going to introduce a whole slew of kind of a collection of reports on what we think is the forecast for the big data marketplace. George has been the primary author for a lot of those and it goes into this notion of that we do see these different patterns emerging and these patterns provide clues about what disciplines are going to be required and what tooling is going to be supporting of those disciplines, even as we understand the nature of the challenges that we face. So George, I know you like to talk about the two big ones is how do the analytic change and how does the preparation data chains evolve as a consequence of trying to take on more complex problems with more sources of data while sustaining that business discipline necessary so that it becomes a regular ongoing feature of the value proposition. So it sort of builds on the data lake because in the sort of first phase of people getting their feet wet with data lakes, it's an ad hoc process to collect the data, clean it because the data, it's sort of like mining for gold is just for diamonds, there's a lot of crud you got to clean off and then you have to prepare once you figure out what the data means you have to prepare these models and you have to figure out do they predict well and then you have to embed them in the operational apps and so that's the design time and there's an analogous one that works at runtime where it's sort of like maintaining this but as we get further down the experience curve, there was this wonderful digression, wonderful and I say that tongue in cheek study by McKinsey where they said we're going to be short by two million data scientists in five years and then IBM would say if you're going to need you're going to need to work with us if you need any of these but they're missing the whole point that the tools themselves are evolving to have the intelligence to run these pipelines and I'm talking now sort of- And they're going to be easier to use. I mean, when you say that, it reminds me of the idea that everybody in the world was going to be a telephone operator back when the telephone systems first started. The tools improve. But it's also, once those tools improve then we can worry about the next problem which is let's build better apps. Let's tie these predictions deeper into our operational apps. Like even SAP, let's put some real supply and shade smarts in there where it's not just allocating your sort of manufacturing schedule and your inventory placement. You're actually collaborating with people upstream in the supply chain or downstream. That's the sort of smarts you want to start thinking about. You mentioned SAP. Let's bring in Oracle in there. Oracle SAP, you got IBM, you have the big vendors. Informatica on. I want to ask you guys in our final couple of minutes here what is the length to the path to digital value because you mentioned if data is the digital capital which we agree then there has to be a clear path to that value or the straight and narrow that can start sprinting to putting out solutions. So what is that link? That's the key thesis of your research. And it's something that we're going to have to spend, we are going to invest an enormous amount of time and try to answer that question with some clarity. I think that it's still someone up in the air, John. We were talking earlier that every single significant period of transformation in the tech industry has infrastructure or tools or other companies often created the new thing but it was the developer community that grabbed it and ran with it to create new levels of value. So I think one of the most interesting issues over the next couple of years is how are developers going to respond and what will the tooling look like to create value out of these tools. So we were another Cube segment that we were on. It was the, we talked about this notion of the traditional developer approach is to come at it from an application tool standpoint looking at the screens, imagining the behavior and the data was modeled to serve that. Now we're being driven by the data itself. How is that going to change the way development happens? So it's the next couple of years, one of the crucial things to unfold will be the role that developers play in creating new methods, new approaches and new tooling so that we can get more people in locking value out of these very, very powerful sets of technologies. And open source has been a big driver and killer point. Now let's go to the other side of the spectrum of the business value. The CXO, the CIO, the people writing the checks for their businesses growth. They want to see a new kind of reporting. We were speculating before we came on camera here about the lens of reporting, short term versus long term. So this will change some of the tactical and also execution aspects of the impact statements to businesses. Now I have new data sources and new insights. So that drives the question of does that change the value chains of the business side thoughts? Oh absolutely, it's going to change the value change of the business side. In, you know, you can take it in simple terms where companies are no longer selling aircraft engines. They're selling flight time on those aircraft engines because they are selling insight into the analytics associated with how the aircraft engine works. So the company, so their customer no longer is buying the asset. They're buying the outcome of the asset. And clearly that's going to be a regular feature of how business operates. And big data along with IoT, along with new ways of building software, new ways of thinking about analytics, new algorithms that we're going to discover over the next few years is absolutely essential to understanding how we can move into this world where almost anything can be measured, modeled, benchmarked as long as we have access to the data. And this brings to the conscious we're having within SiliconANGLES teams which is this lens of short-term view and transparency because if you have data and you have transparency and you have open source, you have this notion of the lens of real time changes the aspect of how people are judged. And you hear that term all the time. We're in, we're playing the long game. Right, right. Currently on Wall Street you've seen some of that data. And so this is going to be a major change on what the data means, how people react to it more importantly, the business models. Well, what we were talking about earlier with one of the guests, John, was the idea that the stock market in many respects is one of the most mature, big data systems out there. Where you have literally millions of people adding information, taking action, and watching how that all turns into signals through the stock price. And you used to be a stock picker, so you know this really well, George. It was punctuated by, it was punctuated periodically by a company event where they released their financials on a quarterly basis. And everybody geared up for that, and there was this enormous amount of importance. And so the question is, can we turn that around entirely? Can we flip that on its side? So the company is in a position to start participating in the community that's looking at valuation and what not. Valuation's a huge impact. Yeah, and by releasing data in different ways and participating more in some of this transparency, can we flip that around? Now that I guess leads to one of the biggest challenges here is that the technologies making it possible, our understanding of the business problems are making it possible, but there's still a lot of social change that needs to happen before privacy, for example. What we were just talking about, what it means to report on earnings and what not. A lot of social change is gonna be associated with some of this. And the keyword you mentioned is things are being flipped around and flipping things upside down is this opportunity certainly we are seeing it here on theCUBE, we're seeing it at SiliconANGLE and on our Wikibon teams and we're certainly seeing it in the data. The opportunity for entrepreneurs and businesses out there is to flip things on their head and take advantage of the new arbitrage opportunities and get a valuation for the long game. So I think that's a great analysis, great point guys. Peter Burris, head of research at SiliconANGLE Media, Wikibon. George Gilbert, big data analyst at Wikibon. I'm John Furrier. We'll be back with more CUBE extracting the signal from the noise after this short break. Mr. Liner. Great. Cool. Really interesting. Oh. Good. Peter Burris.