 Yeah, Omar Awadallah, the other co-founder, back to back. This is theCUBE, SiliconANGLE.com, SiliconANGLE.tv's production of theCUBE, our flagship telecast, we go out to the event. That was a great conversation. I was really just, just cool. I could have, we could have probably hit on a few more things. Obviously, well-read, awesome. Co-founder of Cloudera, Omar, you were, you did a good job teaming up with that co-founder. Not bad on theCUBE, huh? He's not bad on theCUBE, isn't he? He reads the internet, that's what we say. Anything is going on anywhere. He's a cute star, you know? Technology, Jeff knows it. We tell you, I'm smarter just by being in Cloudera all those years and that actually was following what he was saying, sad. Didn't dust my brain out. So, okay, so you're back. So, we were talking earlier with Michaels and about the relational database thing, so I kind of picked that up when we left off with you around, he was really excited, he's like, hey, we saw that relational database movement happen, he was part of that, that generation, and then, but things were happening, are kind of happening the same way, in a similar way. Still early, so I was trying to really peg with him, how early are we? Like, so, as the curve, this is 1400, it's not the Javits Center yet, maybe the Duke World next year, it might be at the Javits Center, 35,000 people. Just don't go to Vegas. So, I'm trying to figure out where we are on that curve. Are we on the upward slope, down here, not even hitting that? I think we're moving up quicker than previous waves, and actually, if you look, for example, Oracle, I think it took them 15, 20 years until they really became a mature company. VMware, which started about, what, 12, 13 years ago, it took them about maybe eight years to be a big company, a mature company, and I'm hoping we're gonna do it in five, so a couple more years. Highly accelerated. Yes, but yeah, we see, I mean, I've been surprised by the growth. I have been, I've been warned about enterprise software and that it takes long for adoption to take place, but I've been surprised. But the consumerization trend is really changing that. I mean, it seems to be that the enterprise has always lasted. Why the shorter cycle? I think the shorter cycle is coming from having the right solution for the right problem at the right time. I think that's a big part of it. For luck, definitely is a big part of this. Now, in terms of why this is changing compared to a couple of decades ago, why that option is changing compared to a couple of decades ago, I think that's coming just because of how quickly the technology itself, the underlying hardware is evolving. So right now, the fact that you can buy a single server and it has eight cores to 16 cores, has 12 hard disks, two terabytes each, is something that's just pushing the limits of what you can do with the existing systems. And hence, making it more likely for a new system to disrupt them. Yeah, we can talk about a lot. It's very easy for people to actually start a big data project, for example. And the hardest part is, okay, what problem do I need to solve? How am I going to monetize it, right? Those are the hard parts. It's not the underlying technology. Yes, yes, that's true, that's true. I mean. You're saying, yeah, you're saying because things are so much. Because I'm seeing both, I'm seeing both. And I'm seeing cases where, you're right, there's some companies that's like, oh, this Hadoop thing is so cool. What problem can I solve with it? And I see other companies, like, I have this huge problem and they don't know that Hadoop exists. It's so obvious. And once they know, they just jump in it right away. It's like we know when you have a headache and you're searching for the medicine in aspirin. Wow, it works. I was talking to Jeff Humberbacher before he came on stage and I didn't even get to it because we were so on a nice riff there, right? A bunch of musicians playing the guitar together. But we talked about the IT and dynamics. And he said something that I thought's right in the money. And SAP is talking the same thing. It said, they're going to the lines of business. Because IT is the gatekeeper that's, it's like selling mini computers to a main frame. We're selling client servers for a mini computer team. We're seeing both, we're seeing both as well. So, more likely the former one. Meaning that, yes, line of business and departments, they adopt the technology. And then IT comes in and they see there's already these five different departments having it and they think, okay, now we need to formalize this across the organization. So what happens then? What are you seeing out there? Like when that happens, I mean, perhaps people get their hands on, hey, we got a problem this time. That's what it comes down to. Well, Hadoop exists and go get Hadoop. Oh yeah. They plop it in there and what does IT do? So they pop it into their own installation or on the cloud and they show that this actually is working and solving the problem for them. And when that happens, it's a very easy adoption from their own because they just go tell IT we need this right now because it's solving this problem and it's going to make us this much more money. Moving it right in, no problem. Is that another reason why the cycles compressed? I mean, you know, you think client service, there was a lot of resistance from IT. Yes. It's the same thing with mobile. I mean, mobile is flipped, right? I mean, it's okay, bring it in. We got to deal with it. I would take the same thing. We have a data problem. Let's turn it into an opportunity. Yeah. And it goes back to what I said earlier. The right solution for the right problem at the right time. Like when you have larger amounts of unstructured data, there isn't anything else out there that can even touch what Hadoop can do. So, Omar, I need to just change gears here in a minute. Yeah, yeah, the gaming stuff. So, we have, we're featured on Justin.tv right now on the front page. Oh, wow. But the numbers aren't coming in because there's a competing stream of a recently released Modern Warfare 3 feature. Yes, yes. So, we have to compete with Modern Warfare 3. So, can we talk about Modern Warfare 3 for a minute and share with the folks what you think of the current version, if any. Have you played it yet? Unfortunately, I'm waiting to get back home. I don't have my Xbox with me here at the conference. I'm talking about modern Warfare 3, it's like a Christmas tree. You know, I love, I'm a big gamer, I'm a big gamer. At Cloudera, we have every Thursday at 5.30 in the office, we play Call of Duty version 4, which is Modern Warfare 1, actually. And I challenge people out there to come challenge our team. Just ping me on Twitter and we'll do a Cloudera versus- Let's reframe that. Let's keep them out there. Let's keep them out there. Let's keep them out there. Let's keep them out there. Let's keep them out there. Let's keep them out there. Let's keep them out there. Let's keep them out there. Let's keep them out there. These guys are gamers. So all the young gamers out there, I'm saying they're gonna challenge you. Which version? Modern Warfare 1. Modern Warfare 1. Yes. How do they firewall in? Can you set up an external- We'll figure it out. We'll figure it out. Okay, just ping me on Twitter. We'll carry it live. Actually, we can stream that. Yeah, that'd be great. That'd be great. Yeah, so I'll tell you some of our best Hadoop committers and Hadoop developers. Pitch in the picture of Modern Warfare 3 going on. Now, Modern Warfare 3, very excited about the game. I saw the trailers for it. The graphics look just amazing. Graphics are amazing. I love the series since the first one that came out, and I'm looking forward to getting back home to playing the game. I can't play. My son won't let me play. I'm such a flumbler with the new. I'm a keyboard controller. I can't work the Xbox controller. I have a coordination problem at age, and I'm just a klutz, and like dad, sorry, charity's over, and I can, I play with my friends now. You can get the box back. I'm around adult, big gamer. But in terms of, I mean, something I wanted to bring up is how to link up gaming with big data and analysis and so on. So, I'm a big gamer. I love playing games, but at the same time, whenever I play games, I feel a little bit guilty because it's kind of like wasted time. So it's like, I mean, yeah, it's fun, and I'm getting lots of enjoyment on it. It makes my life much more cheerful. But still, how can we harness all of these hours that gamers spend playing a game like Modern Warfare 3? How can we collect instrument, all of the data that's coming from that, and coming up, for example, with something useful? This is exactly the kind of application that's mainstream is gaming. Danny at Riot Games is telling me we saw him at Oracle Open World. He's up there for the Java one. He said that they don't really have a big data platform, and their business is about understanding user behavior, wrap tons of data about user playing time, who they're playing with. They want to get into currency trading. I can't mention the names, but some of the biggest gaming companies out there are using Hadoop right now, and depending on CDH, for doing exactly that kind of thing. Creating a good user experience. Today, they're doing it for the purpose of enhancing the user experience, and improving retention. So they do track everything, like every single bullet you fire, everything you baseball, head you get, everything, home run you do, and in a football type of game. Every consecutive head shot you get. Everything. Everything is being, yeah, head shot you get, and so on. But as you said, they are using that information today to sell more products and retain their users. Now, what I'm suggesting is that how can you harness that energy for the good as well? I mean, for making money, money is good and everything, but how can you harness that for doing something useful so that all of this entertainment time is also actually productive time as well? I think that'd be a holy grail in this environment if you can achieve that. It used to be that porn, used to be the telegraph of the future of applications, but gaming really is, if you look at gaming, you get the headset on, it's a collaborative environment. You got unified communications. Yeah, and you see our teenager kids, how many hours they spend on these things. You got play, it's a play environment, it's very social collaborative. Some say, we're saying, what I'm saying is that that's the future work environment with Skype evolving. Our multiplayer game is called Our Job, right? So, I'm big on gaming, so all the gamers out there, Homer has challenged you. Yeah. Got a big data example. What else are we seeing? So let's talk about the software. So one of the things you were talking about that I really liked, you were going down the list. So on Mike's slide, he had all the new features around the core. Can you just go down the core and rattle off your version of what it means and what it is? So you start off with, say, HBase. We talked about that already. What are the other ones that are out there? So the projects that we have right now? The projects that are around, those tools that are to be built. Yeah, so the foundational one, as we mentioned before, is HDFS for storage, map-reduced for processing. And then the immediate layer above that is how to make map-reduced easier for the masses. So how can he, not everybody knows how to learn map-reduced Java. Everybody knows SQL, right? So one of the most successful projects right now that has the highest attach rate, meaning people usually, when they install Hadoop and install it as well, is Hive. So Hive takes SQL, and to Jeff Hammerbacker, my co-founder, when he was at Facebook, his team built the Hive system. Essentially, Hive takes SQL, so you don't have to learn a new language, you already know SQL, and then converts that into map-reduced for you. That not only expands the developer base for how many people can use Hadoop, but also makes it easier to integrate Hadoop through ODBC and JDBC, integrated with BI tools like MicroSherry, and Tableau, and Informatica, et cetera, et cetera. He mentioned R, too. He mentioned R programming. R is one of our best partnerships. We're very, very happy with them. So that's one of the very key projects, is Hive. A sister project to Hive is called PIG. PIG Latin is a language that Yahoo invented that you have to learn the language, but it's very easy to learn compared to map-reduced. But once you learn it, you can specify very deep data pipelines. SQL is good for queries. It's not good for data pipelines because it becomes very convoluted, it becomes very hard for the human brain to understand it. So PIG is much more natural to the human brain. It's more like Perl, very similar to Perl scripting kind of languages. So with PIG, you can write very, very long data pipelines. Again, very successful projects doing very, very well. Another key project is HBase, like you said. So HBase allows you to do low latency. So you can do very, very quick lookups, and also allows you to do transactions. So you can do updates, inserts, and deletes. So one of the talks here at Hadoop World, which I recommend people watch when the videos come out, is the talk by Jonathan Gray from Facebook. And he talked about how they use HBase. Jonathan. He's coming on here in the Cube later. Yeah, so you should grill him on that. So they use HBase now for many, many things within Facebook. They have a big team now committed to building and improving HBase with us and with the community at large. And they're using it for doing their online messaging system. A live mail system in Facebook is powered by HBase right now. Again, eBay, the Cassini Project, they gave a keynote earlier today at the conference as well, is using HBase as well. So HBase is definitely one of the projects that's growing very, very quickly right now within the Hadoop ecosystem. Another key project that Jeff alluded to earlier when he was on here is Flume. So Flume is very instrumental because you have this nice system Hadoop, but Hadoop is useless unless you have data inside it. So how do you get the data inside Hadoop? So Flume essentially is this very nice framework for having these agents all over your infrastructure inside your web servers, inside your application servers, inside your mobile devices, your network equipment that collects all of that data and then reliably and materializes it inside Hadoop. So Flume does that. Another good project is Uzi. There's so many of them. I don't know how long you want me to keep going here, but Uzi is a workflow processing system. So Uzi allows you to define a series of jobs, some of them in PIG, some of them in Hive, some of them in MapReduce. You can define a series of them and then link them to each other and say only start this job when these other jobs finish because I'm waiting for the input from them before I can kick off and so on. So Uzi is a very nice framework that will do that. We'll manage the whole graph of jobs for you and retry things when they fail, et cetera, et cetera. Another good project is WIR, W-H-I-R-R. And WIR allows you to very easily start a Hadoop cluster on top of Amazon EC2, on top of Rackspace, virtualizing time. It's more for kicking off Hadoop instances or H-Base instances on any virtual infrastructure. VMware vCloud, so it supports all of the major vCloud, sorry, all of the major virtualized infrastructure systems out there. Eucalyptus as well and so on. So that's where, W-H-I-R-R. Avru is another key project. It's DuckCutting's main kind of project right now. None of DuckCutting came on stage yet with you guys. So Avru is a project about how do we encode with our files the schema of these files, right? Because when you open up a text file and you don't know what the columns mean and how to parse it, it becomes very hard to work with it. So Avru allows you to do that much more easily. It's also useful for doing RPC, well we call RPC, we remove procedure calls for having different services talk to each other. Avru is very useful for that as well. And the list keeps going on and on. MAHAUT, yeah, which we just add, thanks for reminding me of MAHAUT. We just added MAHAUT very recently. What you said, I'm not familiar with it. So MAHAUT is a data mining library. So MAHAUT takes some of the most popular data mining algorithms for doing clustering and regression and statistical modeling and implements them using the map-reduced model. Does it have machine learning in it too? Yes, yes. So that's the machine learning. So yeah, state vector machines and so on. What's SCOOP? So SCOOP, you know all of them. Thanks for feeding me all the names. The ones I don't understand. There's so many of them, right? I can't even remember all of them. So SCOOP actually is a very interesting project. It's short for SQL to Hadoop, hence the name SCOOP, right? So SQ from SQL and OOP from Hadoop. And it also means SCOOP as in scooping up stuff when you scoop up ice cream. And the idea for SCOOP is to make it easy to move data between relational systems like Oracle, Teradata, Netiza, Vertica, and so on. And Hadoop. So you can very simply say SCOOP, the name of the table inside the relation system, the name of the file inside Hadoop, and the table will be copied over to the file. And vice versa, you can say SCOOP, the name of the file in Hadoop, the name of the table over there, it will move the table over there. So it's a connectivity tool between the relational world and the Hadoop world. Great, great tutorial. And all of these are Apache products. They're all projects built within the Apache. This is not part of your unique proprietary. But these are things that you've been contributing to. We're contributing to the whole ecosystem. Yes. And you understand very well. Yes. And contribute to your knowledge of the marketplace. Absolutely, we collaborate with the community on creating these projects. We employ committers and founders for many of these projects, like Doug Cutting, the founder of Hadoop, he works at Cloudera. The founder for the Uzi project, he works at Cloudera, for Zookeeper, works at Cloudera. So we have a number of them on stuff. So we had Arunon from Hortonworks. Yes. And it was really good because I tell you, I walk away from that conversation and I got to say for the folks out there, there really isn't a war going on in Apache. In Apache, there isn't. I mean, there isn't. But we'll be honest, like in the developer community, we are friends, we're working together, we want to achieve the ultimate goal. It's all kumbaya, understands the rising tide, floats all boats, they're all playing nice in the sandbox. It's just a competitive landscape in Hortonworks. In the business? Yeah, the business side is competitive. There's this PR. Not PR, we're trying to be friendly as friendly as we can. Yeah, no, they're hyping it up. But he was cool, he was like, hey, we know each other. We all know each other and we're just going to offer free and charge our support and so are they. And that's okay. And they got other things going on. But he brought up the question. He said they're launching a management console. So I said, Clutter's got a significant lead. He kind of didn't really answer the question. So the question is, that's your core bread and butter? That's your... Yes and no, yes and no. I mean, if you look at Cloudera Enterprise, and I mentioned this earlier when we talked in the morning, it has two main things in it. Cloudera Enterprise has the management suite, but it also has the support and maintenance that we provide to our customers and all the experience that we have in our team. It's part of the description. Yes, part of the description. And I want to stress the point that the fact that I built a sports car doesn't mean that I'm good at running that sports car. The driver of the car usually is much better at driving the car than the guy who built the car. So yes, we have many people on staff that are helping build Hadoop, but we have many more people on staff that helped run Hadoop at large scale at financial industry, retail industry, telecom industry, media industry, health industry, et cetera, et cetera. So that's very, very important for our customer, all that experience that we bring in on how to run the system effectively within these verticals. But their strategy is clear. We're going to create an open source project with an Apache firm management console. Yes. And we sell support too. Yes. So there'll be a free alternative to management. So we'll have to see about, I mean, we look at the product. I mean, our product. It comes down to product differentiation. Our product has been in the market for two years. So they just started building their product. It's just alpha. It's just alpha. Their product is out in alpha right now? Yeah. Well, the Apache product. Yeah. Yeah, the Apache project is out in alpha. So we'll see how does it compare to ours, but I think ours is way, way ahead of anything else out there. Yeah. And for people to try them for themselves. Essentially, John, when I asked Arun, why does the world need Hortonworks? Eventually the answer we got was, well, It's free. Hadoop needs to be more open. No, there's a, I mean, that's not really the reason why Hortonworks. No, they want to go make money. Right, exactly. Exactly. We're not going to say that to you. Yeah, yeah, yeah. But I kept pushing and pushing, and that's ultimately the closest. Yeah. Because you just listed the 12 open source projects. Yes. I mean, you can't get much more open. Yeah. Look at the management console. But Clottier's not contributing on all those. I mean, not only. We are. No, no, we absolutely are. No, you are contributing, you're not, but that's not all your projects, it's other people involved. Yeah, we didn't start all of these projects. Yeah, that's true. You're contributing heavily to all of them. Yes, we are contributing to all of them. Yeah, and that's clear. And Todd Lipkin said that he contributed his first batch to HBase in 2008. Yes. So, I mean, you go back through the ranks of your people. And Todd now is a committer on HBase, he's a committer on Hadoop itself. So, on a number of them. So, you're clearly the lead. And, you know, and... But there is a concern, but we've heard it, and I want to just ask you. I don't know, please. So there's a concern that if I build processes around a proprietary management console, I'm going to end up being locked into that proprietary management CA all over again. Now, this is so far from CA. Yes. Right? But that's a concern that some people have expressed. And I think one of the reasons why Hortonworks is getting so much attention. So, talk about that. It's a very good observation to make. So actually, there is two separate things here. There's the platform where all the data sits, and then there's this management console beside the platform. Now, why did we make the management console? Why the cloud didn't make the management console? Because it makes our job for supporting the customers much more achievable. When a customer calls in and says, we have a problem, help us fix this problem. When they go to our management console, there is a button they click that gives us a dump of the state of the cluster. And that's what allows us to very quickly debug what's going on. And within minutes, tell them, you need to do this, you need to do that. Without that, we just can't offer the support services. There's real value there. Yes. So now, a year from, but you have to keep in mind that the underlying platform is completely open source and free. CDH is completely 100% open source, 100% free, 100% Apache. So a year from now, when it comes time to renew with us, if the customer is not happy with our management suite, is not happy with our support, they can go to Hortonworks. People are afraid of Oracle. They can go to IBM. You can take the data. Yeah, you don't even need to take the data. You're not going to move the data. It's the same system, the same software. Everything in CDH is Apache. We're not putting anything in CDH which is not Apache. So a year from now, if you're not happy with our service to you and the value that we're providing, you can switch. There is no lock-in. There is no lock-in. And your argument would be the switching cost. The only lock-in is happiness. The only lock-in is happiness. The only lock-in is happiness. Customer delay, which was... By the way, we just wrote a piece about those wars and we said the risk of lock-in is low. We made that statement. We got some heat for it. Yes. And this is sort of... At scale though, what the people are saying, they're throwing the tomatoes and saying, if, this is again, in theory, at scale customers are so comfortable with that, the console that they don't switch. Now my argument was... Yes, but that means they're happy with it. That means they're satisfied with it and happy with it. And it's more economical for them than going and hiring people full-time on stuff. So you're always on check. As long as the customer doesn't feel like Oracle. Yeah, see, that's the difference. Oracle is very different. Oracle is different, right? Here, it's like Cisco routers. They get nested into the environment, provide value. That's just good competitive product strategy. Yes. If they're happy. It's called open-washing it with Oracle. I mean our number one core attribute in the company, the number one value for us is customer satisfaction. Keeping our customers happy with the service that we provide. So differentiating the product, keep the commanding lead, that's what's happening. That's your goal. Yes. That's what's happening. Absolutely. Okay, I'm Ronald Adala, co-founder of Cloudera. Always a pleasure to have you on theCUBE. We really appreciate all the hospitality over the year and a half. And personally thank you for letting us sit in your office. And we'll miss you. We'll miss you too. We'll see you at theCUBE events, off swing by. Thanks for coming on theCUBE. And great to see you. Congratulations on all your success. Thank you. And thanks for the review on Modern Warfare 3. Yeah, yeah, if there's any gaming stuff, you know I love this. Yeah.