 Live from New York, it's theCUBE covering Big Data NYC 2015. Brought to you by Hortonworks, IBM, EMC, and Pivotal. theCUBE, theCUBE goes out to the event. Live from New York. We're here at Stratoplus of Duke World. This is our event, within the event, Big Data NYC. Craig Steele is here. He's the General Manager of Global Strategic Alliances at Pivotal, and he's joined by Chris Harold, who's the global field CTO for Big Data Solutions inside of EMC. Gentlemen, welcome to theCUBE. Thanks. Thank you. Craig, I want to start with you. A lot of talk about the Federation last several months, you know, last couple years really. I mean, Joe Tucci has stood up and he said, we asked him one day at an analyst meeting, you know, and he regrets, and he goes, you know, I wish I had done a better job of bringing the Federation together sooner. You know, it's a complicated arrangement, but you guys have laid out, you know, a strategy. Talk about the Federation play as it relates to Pivotal and EMC relationship. Certainly, so I think we all wish we would have started a little bit sooner. EMC is a very large organization, VMware, as well. And, you know, as Pivotal was spun out, and we started working on enabling each other, if you will, to build joint solutions to take to our customers, you have to take into consideration the multiple solutions and technologies that are available from all these different organizations, and come up with winning solutions that customers will actually be willing to invest in. And I think, recently, we've seen a lot of momentum, specifically to the Federation Business Data Lake. We've got some wins under our belt with some, I would say, significantly large customers, and we're in the various phases of deploying those options for customers now, and we've got a pretty significant pipeline lined up together as well. So, I think for all intents and purposes, things are going very, very well, and we expect that to continue. So, Chris, I wonder, one of the things that we talk about all the time is the difficulty of using Hadoop, and you're in the solutions group, so part of your job is to make it easier. Talk about the mission of your part of the organization. Sure, so, we started with a very simple premise about, really, I started about four or five years ago working in the Hadoop space, trying to help enable customers using the EMC sort of core offering storage and compute and things like that from a VMware and block and file storage perspective. What I realize, just like you said, is that everybody has a challenge with that. It's tough, it's hard to get environments up. Once they're up and running, people basically just, I don't want to touch it, and that makes it very hard to innovate, it makes it very hard to iterate and make changes rapidly, and about a year and a half ago when I came into the Solutions Organization, I came in specifically with the goal of designing a platform that allowed our customers to get out of the plumbing business, right? And the adage I always use is that if I buy a house, I don't want to actually have to do all the plumbing work, I just want to walk in and take a shower. And when we built this platform, that was the whole goal, is we want customers to have a push button, get cluster experience, Hadoop as a key technology, but across the board, right? And so, Pivotal brings to the table a number of analytics technologies outside of traditional MapReduce that are also complex, are can also be difficult to deploy and integrate. And instead of spending a lot of time on the plumbing, on the making bits work together and actually run, we've said our stated design goal for our solution is, let's simplify that, and let's get to the really hard work, which is the actual data science, which is the actual analytics, right? We want to bring those things to the table as fast as possible. So one of the solution areas that we look at a lot is governance, you know? It's kind of in the boring but important category, but it's becoming more and more important and less boring for a lot of organizations. It's interesting because Pivotal, you think of Pivotal Labs as like this, maybe oftentimes shadow IT, you know, organization, because somebody in marketing or sales says, hey, let's call it Pivotal Labs, these guys, we know how to get stuff done, you know? Our IT guys, we don't want to work with. And so, they'll perpetuate some big data analytics project, right? And they'll pop up all over the place, but then there's the governance gap. How are you guys, first of all, is that a fair representation? And how are you guys addressing that? Yeah, I think it's a fair representation, I'll start and then you can probably pick up the details there. What we're seeing more and more is, there is a definite link from an application to actually big data analytics and understanding whatever databases you're attaching to that. And then making sure that those interface with the application in a way that can scale, and it's, in some cases, it needs to be very, very fast when you start to think about machine learning and other capabilities that we bring to bear. I think if you look at it holistically like that, it is challenging, but I think we worked on together and Chris can expand on this, a way to make sure that the technology pieces are, you're aware of what data is going to be part of that ecosystem. And then I think the gravity of that data as well goes into the decision-making process with what you bring in. And I think as that starts to expand and it moves into different business units that want to take advantage of this technology, that's where the governance model starts to really show its importance. And then Chris, you can probably tell us what we're working on. No, and that's absolutely, that's exactly our model in governance. And I think you see a lot of activity right now, specifically, certainly pivotal and through the ODP initiative, you guys are part of some of that activity. I know Cloudera and Matt Barr are working specifically around governance. And for us, as much as we 100% wholeheartedly support that effort, it's a very HDFS-focused, it's very focused on Hadoop as a microcosm of a much bigger ecosystem, right? And when we think about data governance from a federation level, from a solution level perspective, our goal is not just to protect HDFS, it's to protect anything you're running inside that framework, right? And so the key point for us is, we want to get to policy-driven data governance that allows you to govern that data regardless of what you happen to store it in, what tools you happen to consume it with, and most importantly, who you happen to be, right? Because you could be a data scientist, a BI person, whatever it is, and we realize that, I mean, Wikibon just put it out this morning, right? You guys just put out the report that the number one and two, or code number one, is Google and Azure. And what is Azure analytics based on SQL? That is what predominantly most of our customers are using. If we're protecting Hadoop, but we're not doing anything above that layer where we're addressing the governance and policy and access control into SQL, all of the advanced tools like machine learning and stuff that we want to bring to bear from the pivotal side of the Federation relationship, we're in the wind, right? Now we're trying to stitch together this little bit of open source over here and this little bit of a consumer product over here and whatever it might be to make that patchwork work. And that's just not a good approach when you're thinking about a larger, broader solution set, and customers are coming back to us and saying, we can do that. We'll go stitch together the open source stuff, what we really want from you is a bigger view into enabling all this new machine learning and all of this stuff with our existing people, with their existing skill set, without having to teach them Java. So especially your large customers have the capabilities to do that. They want to help in those areas that they just aren't really focused on, is that? Yeah, I was just going to add that that's SQL, there's a very large ecosystem of folks that know how to develop code with that framework. And so the parts of the product portfolio from Pivotal that are also with the Federation Business Data Lake, we see that as a trend that's continuing. That's why Azure and Google are having success in that area because the intellectual property is there and folks that are dealt with business intelligence tools in the past, they don't want to learn a new language, just don't have the scale to ramp up as quickly as they should. So I've always been a huge fan of the Federation when EMC announced that I was actually down here at the financial analyst meeting and when they sort of laid out what they were going to do with the Federation, so it made a lot of sense. You had all these sort of piece parts but you had enough critical mass to create Pivotal. Obviously you had VMware growing like crazy and you had sort of EMC providing the steady hand. What struck me at the time was, okay, this changes the game for EMC in particular because it allows you to become more strategic as opposed to just talking to storage admins which is kind of, you know, the guys do in Lund Management and sort of a dying breed anyway. At the same time Pivotal, this startup has this distribution channel and then you've got this strategic component of VMware which everybody has VMware. So it sort of made sense that, okay, this can organically grow but you guys have now more deliberately, we were talking about at the top of this set, deliberately sort of developed capability here, I don't know, the Tiger team and overlay. I wonder if you could talk about that a little bit more. That's the color. Yeah, sure, we both have comments on this but I'll tell you a couple of things that we're doing. We're focused on accelerating what we see as initial success with the Federation Business Data Lake. And what we're doing is we're going to double down and we're going to invest, actually we've already made the decision in our signing contracts now to build a center of excellence. And that center of excellence will be focused on differentiated technologies between Federation entities. So you look at the portfolio again, from a Pivotal perspective, it's a lot of what's happening now and in the future and as we build and integrate entire systems and I think it was your phrase and I'm going to rob you here in public but taking the science out of doing data science, right? Just making it easier for customers to be able to leverage these tools for a unique business use case in a way that's not so painful for them, right? I think one, and I'm sure these stories have been told before, but one I was actually reading about recently was Netflix. When they first launched, they had what most people thought was sort of a blockbuster killer, right? It's a DVD delivery service and then they went to a streaming model and what they were actually doing was collecting information. It's a data model. Yeah, it's a data model. They built the data model, the killer data model. They really did. A step above that or simultaneous to that is the microservices that they're using to develop applications that are widely used as well. But back on the data side, if you look at viewership and viewer affinity for a specific show in a specific region, how often they're watching a show, how many episodes they'll watch in a row before they burn out, they had enough information to then go into the production business and know with a great deal of certainty if a television show was going to be successful before they spent a penny on writing a script or hiring a director or the rest. So it's really about collecting that information and being able to leverage it with the application and that's what we're going to focus on in the center of excellence. Yeah, and I think from the Federation side, from the EMC side, Craig and I were talking about this before. I banged the DevOps drum pretty hard within EMC and obviously Pivotal has a strong DevOps presence that are more on the Dev than the Ops side right now. But the whole concept of DevOps is, it is where the rubber meets the road with this analytics workflow. And EMC is trying very hard in that, for lack of a better term, that legacy environment, the storage loan administrator environment to talk about DevOps. And it just doesn't resonate with that type of audience until you bring in an actual reason for them to want to do it, right? And when you talk about the complexity and the scale and the diversity of analytics and analytics tools, you can't do that unless you are embracing a DevOps cultural mindset, period. You will find yourself right back in 1993 very, very quickly with a thousand physical servers that all look the same. And that's a horrible place to be. I came from that operations world. I never ever want to see anybody go back to it. So I'm trying really hard to make sure we don't. And it's a different message for EMC, right? And I'm sure you guys know that the Federation are bringing together Best of Breed. That's a new tack that we've taken and then we've embraced and I think is huge, our customers love it. It resonates very strongly with everybody who hears the message and speaks about it. And when we tie all that together, with that cultural mind shift, that actual DevOps mindset of, I need to be able to make my physical environment work like my developers do with their software environment. It's absolutely unparalleled the capability we have within the Federation to bring that to fruition with our customers. I mean, the stuff that we're doing in the COE and joint development of platforms and things like that, we're embedding Pivotal Cloud Foundry into our Federation Business Data Lake. So it's very much the, was it the Gillette, not only am I a customer, I bought the company, right? We're very much, we're embracing that technology. We want that within our platform because of the power that it gives us to meet that DevOps demand. And we've got to bridge that gap and we, as Federation partners and Craig and I work very closely together, but I've been embracing that Pivotal message for a long time, personally and within my solutions team and it just, it makes so much sense. So it's interesting what you're saying about the DevOps culture and the 1,000 servers all the same, presumably because that's the only way we can deal with all this complexity. So we're going to force this standard on you and good luck. But you talked earlier about some of the successes and I know a couple of sort of Federation customers. I guess I won't name their name, but one is I'll say Describe Insurance Company and you talk about this DevOps culture, which Gartner has this thing about bimodal IT and it's kind of interesting, but, but what you're describing, I wouldn't say is bimodal. It's unimodal in a new way. So the bimodal reeks of, and Gartner's a smart guy so I don't want to be pejorative here. If they were here, I'm sure they could defend it better than what I read. But this idea of you have old and new is not what people want to do and the one customer I'm thinking of this insurance company, they really understood that problem. They drove the DevOps culture. It was a Federation situation. You probably know who I'm talking about. And they basically took their legacy infrastructure and they said, we have to make it speak new. And they worked really hard to create that layer. Maybe you call it an abstraction layer, but so that it could participate as a full first class citizen within that new DevOps environment. Is that sort of the model? You guys know who I'm talking about, so okay. So is that the model that most of your customers are taking or is that the sort of top one percent that went into each other? So I would say it's a majority and there are actually several examples of this being done successfully. The one I think you're talking about started as a lab's engagement based on the customer realizing that it was a big insurance company that Google is getting in the insurance business. And when Google gets in anyone's business the whole phones start ringing out. So they wanted to develop a better user experience, be closer with their customer and we came at it from an application perspective. As we start showing customers, paired programming, extreme paired programming, our DevOps sort of culture and we immerse folks in that we actually bring them to our facility to put them through that and then have them take that back to their headquarters and their devs. That's really where we introduced this idea of leveraging tools like Pivotal Cloud Foundry to build, push, scale and push apps out. Once that's sort of, and that's a process in many instances as you talk to a large enterprise company that needs to move very quickly, you kind of have to surround the different players within an organization to get them to make that big cultural shift, if you will. So our sales model, our go-to-market model is really based on surrounding folks like VP of IT. After you get the devs involved and they buy into this technology and process, then it's how do you get the enterprise architecture folks involved? How do you get the lines of business involved and get them to buy in? And once the devs are in, it seems like the rest of the ecosystem will come along in many cases, but after you start wrapping that together, then the question becomes, and even immediately in the process of trying to disclose what this technology brings in terms of capability, the question is asked, do you want us to spin up a proof of concept on-prem and some infrastructure that you have currently or do we need to do it online in some way? And we'll make recommendations that are friendly to our brothers and sisters in the Federation, but that's when the connectivity between the infrastructure and the process for the operations needs to map to the devs at the top. Okay, well what about that? So obviously, the guys we're talking about are big customers of all three parts of the Federation, and that's clearly where you want to start. I mean, that's sort of the obvious top of the pyramid, but what about other customers who don't want to necessarily, they want to mix and match. That's a great point, and actually, our solution philosophy, and Craig stole it, so it's fine, but data science without the data scientists. And the reason that we're approaching it that way is because an organization like Nameless Large Incarnation Company, or Partners Healthcare, who I will be speaking with on theCUBE later this week, talking about the FBDL and the Federation solution. They have a lot of resources in-house. They've got talent, they've got data scientists. There's a lot of things that they genuinely just don't need our help with, right? They have technical challenges that they have a specific need, and the Federation is there to support those things and help them build a bigger framework. But to your point, what about everybody else? What if I don't have 20 data scientists on staff? But I know data analytics are a big deal, and I know Google is getting into my business, right? And I'm a small insurance company somewhere. How do I start, build up this capability and this infrastructure, and meet that rising demand? Because if businesses don't evolve, they're going to go away, right? And we're really focused on that platform piece, that scale out capable platform, right? Because a lot of our customers don't want to go to the cloud. They have data sets that they don't feel comfortable leaving the building, right? It's much easier to adjust and manage on-prem than it is in the cloud. And that shadow IT concept that you brought up earlier, that's a very pervasive thing. And I know we see that in a lot of our customers and people are trying to stomp that out. So we give them that infrastructure piece, that on-prem side, right? And then we bring in this cultural side from Pivotal, which personally and professionally, I'm hugely thankful that EMC made that investment because we were sorely lacking that, right? We needed that cultural infusion of dev into our ops to actually get to where we needed to be. We're great at storing data. We're great at compute. We'll do that flawlessly all day long. And then we would get to a certain point in the conversation. And people are saying, well, okay, now what do I do with this thing? Stuff, right? Now we have that credible end-to-end detail discussion. We've got incredible momentum in the open-source community through Pivotal. And that's feeding back into us, our own engineering side, even just in core EMC and solutions is contributing back into open-source projects. That's a huge departure for EMC. Well, I think this is huge. Five years ago EMC couldn't spell open-source. Pivotal brings instant credibility and timing couldn't be better, right? Yeah, I would say, John, to your point, it's a process, not an event. There are business units within EMC that develop applications in the old sort of waterfall methodology that we're sort of learning how to crack the code with and nudging them over. So it's happening in fits and spurts. And in some organizations within the ecosystem the federation are in wholesale from day one. So I think it's a good blend. The success will drive others to adopt this strategy as well. Where's the COE? So I'll tell you where the infrastructure is going to ship to Hopkinson because we've found space, power, space and cooling. The intellectual property and capital will be, human capital will be based in Denver. So close to Chris. And our goal there really is to two-fold. One is to make sure that we have the right engineering resources together, whether they VPN into infrastructure somewhere else on the planet is of no concern. It's really about having these folks in the room, discussing new ideas, new differentiated technology and capability for our customers and being able to do it in an environment that promotes this sort of DevOps culture. And actually we're building it at Galvanize in Denver, which is a new sort of epicenter of new young talent that is really coming up through the DevOps sort of culture. That's a fantastic collective thing. It's really cool. It really is, yeah. And then a long-term goal is after we're well-established and build up some momentum, is to start to bring customers in and let them experience kind of firsthand what we're doing from a federation perspective in terms of developing these new technologies and they will get a taste of that cultural atmosphere when they attend. Well, it's an awesome story. I mean, maybe to Joe's point, maybe it took too long to get going, but like you say, you're a big company. But with the slow collapse in infrastructure software pricing, you know, the key is solutions. That's where you're going to add value to customers. That's what they'll pay for. There's certainly money to be made in this business. Gentlemen, thanks very much for coming on theCUBE. Sharing your story was really great to have you. Thank you, sir. Thanks for having us. Keep it right there, everybody. We'll be back with our next guest. This is theCUBE, we're live from Big Data NYC at Strata and Hadoop World. We'll be right back.