 The Cube at Hadoop Summit 2014 is brought to you by Anchor Sponsor, Hortonworks. We do Hadoop. And headline sponsor, WAN Disco. We make Hadoop invincible. Okay, welcome back everyone here at Hadoop Summit. This is The Cube, our program. We go out to the events and extract the signal and noise. I'm John Furrier, the founder of Silicon Dance. Wikibon, big data analyst, Jeff Kelly. And of course, we have Cube alumni, good friend of ours, Mervagebian, industry analyst, big data with Gardner, Magic Quadrant, sizing up the landscape. You got the survey from Wikibon, two of the best analysts in the business, right here inside the Cube. I'm proud to sit next to and discuss what's going on at Big Data. Merv, welcome back. Thanks, good to be here. You have a busy day up on the stage there. Great to see you. Thanks for taking the time. Give us the Merv Quadrant action going on here. Where's the state of the Union of Big Data? Hadoop, obviously, Horton works with their big move on the M&A Front, Cloudera, Natchez Today, Kim Stevenson, Cube alumni, women in tech on the board. Certainly looks great, epic board there. Marketplace, where are we? We are at a significant tipping point. Last year, my message was primarily it sounds a lot better than it is so far, so hang on, this year it's about if you're not moving, you're behind. We have arrived at the place where if you haven't gotten started yet, it's past time to do so. Hadoop too just really turned a lot of things around and we are moving from a blunt instrument for batch processing to a sophisticated nuanced stack of technologies that allow us to do many, many kinds of use cases. Many of those are in fact, of course, still immature, many of them are just getting going, but it's time. If you're not on board already, the senior guys are getting up on stage here at the summit and telling you that what we had last year was traditional Hadoop. I love that, that was great. By next year, when we get to 3-0. Legacy Hadoop. Legacy Hadoop, there you go. Right on. It's not backwards compatible, oops. Let's just get into that. So obviously, this is all the hype there. I mean, I hate that jargon, honestly. Legacy, come on, it's still early days. It gives us all something to do. Big date is happening. You mentioned Internet of Things prior to discussing. Huge tsunami, the marketplace is still in massive flux. You got some steady winners on the table. You got Cloudera, Hortonworks, MAPR, all the players out that we had win. Disco on earlier. Red hats out there in the open source world. Open source is dominating. What is the foundation? And you see things like security as the key effort here as the table stays. What is being done in your mind that's moving it faster? What accelerants need to be applied this year? Is it security? Is it high availability, disaster recovery? What are the things that you're saying? All of the above. I quoted my colleague, Mark Byer, from Gartner this morning, who has a theory about the maturation of technologies that roughly takes about two decades. It's an interesting point and a lot of us have been of the opinion that the cycle has been shortening of late and certainly that open source and irrational aduberance in the funding market perhaps are accelerating that. But the fact is it still takes a long time for technology to get mature. It's not just about how many engineers you can throw at it. It's about using it in the real world, experiencing the problems that it encounters, remediating them, finding out what the users really need. And it turns out, if you look back through the history of Hadoop, that it was about 2005 that Doug Cutting put the first piece out there. We are about nine years in. Usually about 10 years in is when you start talking about all those mainstream enterprise-class features like governance and security and availability. Guess what? We are right on schedule. So security is necessary as you go to the mainstream because the early adopters didn't really worry about it too much. For one thing, they were isolated. They weren't doing their own special project. They were somewhere off of the mainstream of IT. So I got to give Jeff Kelly some props here. He was the first industry analyst to have a market sizing of the big data market. He congratulates Jeff, Nimble, Wikibon, Gardner. So you guys have the magic quadrant. I want you guys to answer one question for me because when you talk about maturization, the 10-year cycle, you start to see different metrics. And this is something that came up at OpenStack was what kind of metrics do you need to see? Not commits, total addressable markets. So when you look at that process, once you guys have talked about it, you've got the scoreboard now, Jeff. You put out the study. You guys do a ton of work at Gardner. You know, great job. It's an evolution, it's a continuum. But what metrics do we need to see when you go, there we go, is it TAM, is it market share, is it all the above? Can you guys comment? Well, so one interesting thing we found in our survey that we just conducted was only about, of the folks that have actually deployed Hadoop, whether it's a proof of concept or in production, only about 25% are actually paying customers of important works or a clod era. So for me, that means, and this is just early adopters and only 25% of the early adopters are paying customers. So for me, when that number starts to climb, that indicates that we're moving to a lot more production deployments. You pay money, too. You know, people got to eat, right? The market's going to start actually growing and some real revenue's going to start flowing through these companies like Hortmorks and Clod Arrow. For me, that's one of the key metrics. I'm right there with you. My favorite metric is money. There is a phenomenon in the post open source world that says I can spend a lot of time becoming familiar and testing and proving value before I have to start committing the dollars. We've got numbers very similar to talk about how many of the people who are out there piloting are actually spending money. And this morning in my keynote, I talked about the size progression from pilot to production. You know, pilots are 10 nodes or less, sometimes a lot less, and sometimes they're not on premises at all, they're in the cloud. But when people go to their first project, they're getting up into 50 node territory pretty fast. And when you're charging for your software by node, the numbers start to cycle up pretty quickly. Are the numbers there though? Look at Clod Arrow, they're quote IPO, I mean investment by Intel, many even in Clod Arrow are saying it's an IPO kind of their liquidity because it was liquidity while people got out of the deal. Is that proof that they're not ready to go public? They don't have the revenues or is that just more of an opportunity to take advantage of a great valuation? You want to go first? I'll let you take a stab at that. So if I have a very attractive, very interesting product that is starting to get mainstream acceptance and I want to go global, I need musculature. I need a support organization. I need a pre-sales organization. I need people in China, in South America, in Eastern Europe. I need them all over the world. That takes time to build out. That is the one piece of the technology cycle that really has shrunk in the last few years. Companies that get some early success are global in minutes, it seems. The big win for I think Clod Arrow in the Intel deal was the infrastructure they got that Intel had in some of those enormous markets, the world's largest market, China, already had some Clod Arrow customers, some Intel customers, excuse me. Those people become available now. Clod Arrow is very strong on the sales and marketing side too. They have feet on the street. Let's get one more quick thing. Let's put it in perspective. Jeff pointed out only a small percentage of people are actually paying money right now for this. We think there are fewer than 800 companies today paying subscription prices for Hadoop distributions. Total, across the independence. There's some IBM customers, there's some EMC customers, there's some mega vendor customers. They're a lot harder to see because they tend to be opaque in their reporting. But add up the independence and you've got 700 or 800 customers. The future is right in front of us. It hasn't hit yet, but it's coming. 7,000 customers, 70,000, 700,000. Certainly in the thousands within the next two years. Absolutely. For me, just back to the Clod Arrow Intel, the Intel investment, I think you're right on from a technology perspective. It's a great move for Clod Arrow. They're going to basically inherit all the work around security and some of the performance optimizations that Intel has leveraged and now that team is going to work together. I think the challenge is going to be from a sales perspective and remember I'd love to get your opinion. I mean, what does Intel know about selling a data platform or data management? Well, I think the answer to that is they got out of the business and gave it the Clod Arrow. I think that tells you something. But I think people in the Western Hemisphere don't actually recognize the role that Intel plays in other parts of the world. In some of those other geographers in the world. Well, they're an OEM company. They're Intel inside and they need an inside right now. They realize they needed this now. They're also an enterprise partner in the two biggest markets in the world, India and China. Those people look to Intel as a technology partner, not just as a supplier. And so that role for them there is not the one that we're used to seeing here. Yeah, and Intel's investments are awesome in system on a chip and with internet of things and devices on the edge becoming small and small and not just mobile, I mean just everything. They have an opportunity, in my opinion, with Clod Arrow to take the distraction away from the big data market, get something in the software to find data center positions, space there, have the internet of things market for a data ingest system for big data and then take the pressure off so they can fight the arm in the enterprise battle. Own the edge and make money at the center. And in many ways that is really the iTunes story told in the enterprise marketplace. You know, if you think about it, Apple gave away the client because they made money back in home, right? Back in the data center where they were serving the content to people. Microsoft's doing exactly the same thing. This is why Microsoft is giving away windows on devices nine inches or smaller because they want the footprint. And if those people come back to Xbox for their consumer life or to Active Directory for their enterprise life, Microsoft has got a real- This is an exciting time in tech. I get just jazzed up just thinking about all the disruption, consequences, competition, opportunity, value creation, wealth creation. So great to have you guys on, Mer, in particular, taking the stage to take the time. I'll give you the final word. Tell the folks out there why this event is so important this week around this ecosystem. Well, what it's done here is it's focused the attention of the big data community again on what's needed to go to mainstream. And the pieces are in place. We heard Aaron talk about what's next with Yarn. And it's all about really managing resources effectively. That's what enterprises want to be able to do. They don't want any more rogue projects. They've had the rogue projects and they've proved that there's value there. Now it's time to move on to real exploitation. Connect this with the rest of the fabric. The synergy that Hadoop brings to the table is most powerfully magnified when it's connected to what's already there. That's what's next. I love it. This is all about big data in the future. Cloud Mobile, social big data is going to make a trend we've been following. This is theCUBE. Great job, Mervage. I'm one of the best analysts in the business. Jeff Kelly with the Big Data Survey releasing our tidfithing out. Go to crowdchat.net slash Hadoop Summit. Look for the commentary. We're leaking out some key data points. Thanks, Merv, for coming on. We'll be right back for our next guest after this short break. Thanks.