 from San Jose in the heart of Silicon Valley. It's theCUBE, covering Big Data SV 2016. Now your host, John Furrier and Peter Burris. Okay, welcome back everyone. We are here live in Silicon Valley. This is theCUBE, SiliconANGLES flagship program. We go out to the events and extract the signal from the noise. I'm John Furrier, my co-host Peter Burris. Our next guest is Ian Andrews, VP of Products at Pivotal. Welcome to theCUBE. Thanks guys, excited to be here. So we'd love to talk about products because into-the-day product market fit is a big topic in the community and also has people try to figure out how to build apps as big. You guys are a leader in that area. So quickly, what did you talk about this morning on the keynotes? What's some of the themes that you guys are talking about? Yeah, I really aim for fun this morning in the keynote. The five minute talk, it's hard to go deeply technical. So I aim for the laughter factor. And the topic was actually based around a conversation that happened a couple of weeks ago with a few colleagues and a couple of customers. Happened where all the best conversations happen, which is late at night after a few cocktails. And the net of the conversation was really about this idea of, have we reached peak BI? And this is derivative of peak oil, right? The concept that goes back to the 50s where Mirian Hubbard wrote about this idea that maybe we've reached the total maximum output possible of oil as a natural resource. And we were using that analogy with the BI market because we've seen some really crazy things happening lately, right? Click view is all of a sudden up for sale. Tableau's market cap cut in half in a day. And these are terrific companies with great products. People love them, right? So there's nothing that should explain that shift in the market's perception. And so the conversation that we had a few weeks ago was really about is it the case now that all the users who would ever want to consume BI now have it? And so each incremental, exciting product that's coming out, the next startup doing self-service BI or self-service discovery is actually claiming a customer from another company that already had them. And so we've reached this peak market of business intelligence, what does that mean, right? That's kind of a scary concept, particularly for companies like ours and all the ones you guys have talked to this week, who, BI is the face of the platforms they sell, right? What's the window into Hadoop? Is a BI tool or any database platform? And so that led to kind of a disturbing conversation, like, hey, is there actually not just peak BI, but is this peak platform for data? Meaning the market has actually reached the maximum consumption of a Hadoop system or databases, and we're really just trading customers between vendors. And that's a pretty scary thought. And the good news is the takeaway that we had from this conversation was that actually, we're just at the beginning from a platform's perspective, but that the consumption of that data is really about how do you deliver information in context? And this becomes an application's conversation, right? It's the example I gave this morning in the keynote. I'm a huge Uber user because I travel all the time. I haven't rented a car in two years. You open Uber, and it tells you you're going to have a car in front of your house or your hotel in three minutes or five minutes or seven minutes. There's no BI experience. There's no complex math that I as a user have to execute. But I get a tremendous amount of information delivered to me quickly. Now, behind the scenes, Uber is a fantastic company in terms of leveraging data. Complex algorithms, complex platforms, they've put all these pieces together to give me as a potentially very non-technical user tremendous amount of information in the context where it's most valuable to me. And I think that is the shift that is happening here, and that's the next wave of consumption of these big data platforms like Hadoop that starting with Doug cutting 10 years ago, we've been kind of building. Ian, you bring up a great point. And Peter and I talked about this all day yesterday pretty much throughout the entire day, certainly in our segment of the analyst segment and our close was this future new way of doing things. And I think if you look at peak BI concept, okay, is it peaked out and or is everyone a BI customer shifting vendors, if you will, it takes a whole other approach of new way versus old way. And that's kind of what we see happening, the old way of doing things in a new way. So there's going to be all the chips around the table and the money's going to be divvied around based on who came up with the best approach. You mentioned Uber. But I want to get your thoughts on a comment you made and certainly on the Twitter feed was information in context and data swamps. What does that mean? Is that the environment people are living in? Is that the old way that needs to be retrofitted to the new way and people who are essentially cloud native or data native, if you will, have these new environments? Because Uber does abstract away all the complexities. Exactly. Is that the context that we're trying to get to in the new way? And what is the data swamp? Is there just data that's not being used? Explain what that means. Yeah, well, just to clarify my comments on BI, it wasn't at all critical of BI tools. I think there's some terrific companies doing really exciting things. My friend, Sharmila Mulligan, who I've known for over a decade, what they're doing a clear story, terrifically exciting. The team at Zoom Data, who's a pivotal partner, doing some really neat things. So it's not at all critical of what's happening in BI. I think the point is that if you measure the world of potential BI users in the hundreds of thousands of people globally, out of the 7 billion people on the planet, there's a big, a large number of people who aren't BI users and probably never will be. But it doesn't mean those people should live without information. And so delivering information to them in context that doesn't require them to learn a new tool, but allows them to have a really compelling consumer experience or do their job better, that's what I meant by information and context starts to win. And I think we've seen this turn happen in the consumer world long before the enterprise has really picked up on it, right? And this is the interesting trend for me being in an enterprise software company is Uber, Netflix, they've used data to their advantage to build great experiences for a long time. Enterprises have traditionally struggled at being great software companies. And you're now seeing this trend where folks like Jamie Dimon at JPMC and many other companies, they're sort of demanding their organization become a technology company, regardless of the industry they're in. We've had a tremendous pleasure to work with the team at Ford recently. As Mark Fields, their CEO, has been in our office just talking about continuous integration and delivery, which just blows my mind. I mean, at its heart, Ford is a great manufacturing company. They build cars, but Mark actually understands this kind of core competency in the new way that software is being built, which is really, really exciting. And this user experience, that's just telematics when you've got stuff in cars is a great example. They have their own kind of industry agenda, telematics, et cetera. But now, what it's doing about Ford is we had a chance to interview the CIO, as well as the VP of research. Marcy's phenomenal. Yeah, absolutely. It was at one of your pivotal events. We ran down the city and did it on the ground. But her view is different. She's like, okay, yeah, we got some things to do in the car. But Mark's vision, Mark Fields vision is that when the user gets into the car, that the experience the user wants to bring in to the car is augmented by what the car can offer them. Exactly. This is a completely new way to think about it. Yeah. A digital progression that is completely different than what they thought they'd use digital for. Yeah. Well, the car becomes a peripheral. Yeah. Literally. I asked that question, Peter. Is the car a computer or a peripheral? Yeah. I mean, that's the question. I think if you look at the amount of time that we spend in cars versus out of cars, if I'm Ford, I want to capture your attention, whether you're sitting inside of the Ford Focus and driving it, or you're sitting at home on the couch, right? That connectivity and relationship with Ford as a vendor extends far beyond the car. And you start to look at the sharing economy that's evolved with companies like Uber and Lyft. Like I don't rent cars anymore. I travel all the time. I never rent a car. I rarely drive my car at home because I'm on airplanes somewhere else. That's completely different than the experience that most people had with their car 30, 40 years ago where it was prize possession, probably the second most expensive thing you ever bought. And you cared about it deeply, right? I now look at it much more utilitarian. And that's a tough spot for an industry of automakers to really reconcile. The next generation of people may care less about their core product. How do you still build a compelling relationship with them? And the answer is through software. Yeah, and the relationship that the user has with the device or peripheral or computer, in this case of cars, interesting. So one of the things I asked Marcy at Ford was, would I request an Uber that's a Ford? So what about the Uber drivers? What if Uber drivers bought a Ford and had a unique differentiator? So you start to see Ford thinking that way. It's not, the purchase of their customer journey is different. And people may not buy cars, but they may want to share a car or buy an Uber that's a Ford. Is there anything better bandwidth or whatever? Or if that car provided a set of software APIs that were crucial to bringing your experience wherever you went, that that might be the circumstances in which you asked for a Ford. So one question I have here is, you mentioned the notion of peak oil. And Harvard, when they did that original survey, they picked a number that turned out to be 30% what the number actually turned out to be. And in fact, it looks like it may still be going up as we've discovered new technology to find new sources of oil. Is the same thing also going to happen in BI? Is that really what we're talking about? That the technology is evolving, so it becomes, so the promise is a BI to get your question answered, regardless of where the data is, is becoming more available or is becoming available to more people, even though the precepts of how it happens are changing. Is that also happening as well? I think that's entirely possible. We've seen the evolution from, think about the earliest BI tools, 20, 25 years ago, built for data warehousing applications. They barely resemble the experience that you might have from some of the most compelling tools that are available on the market today. And certainly the user base has grown radically. But I think where you see exponential growth of data, consumption of all this stuff that we're storing into big Hadoop clusters today really becomes application driven, because it means that if I have to learn how to do something, there's only a small set of people that are ever going to go through that learning and training process. And no matter how good you make the BI tool, teaching the world how to use it is an unscalable problem. Right, so let's, let me dig in this a little bit. So for example, if I am a day trader, and Schwab provides to me a set of tools that allow me to do analysis from a BI standpoint, I'm doing things that look like BI. But I'm doing it through a specialized application by specialized services. I am still kind of entering into that context of doing BI, even though it's provided by a tool that provides the appropriate constraints, so I'm not overwhelmed. So the context becomes crucial to the set of services that need to be delivered so the customer gets the experience that they want. Yeah, yeah, yeah, exactly. I think that's a great example, and that one's not far away at all, right? We see Zoom data, a big part of their business, as embedding BI into the context of other applications. They've built, actually on top of the spring data framework that Pivotal builds, they've built a microservices architecture that allows decoupling of presentation layer from compute layer and this easy integration into other experiences. And I think that's the start of this. The thing that we're helping companies at Pivotal do, and we mentioned this off camera, is really get good at building applications. And this is where it starts to get exciting because I think the last 20, 25 years of enterprise IT has been operating with the mindset of build it once and then never touch it because when I touch things, it breaks. And if you look at the best companies today in terms of their ability to deliver great software compelling experiences, they change things all the time, right? We've all heard the rumors about Amazon updates the homepage every 11 seconds and Etsy deploys code every 100 times a day and the list is on and on and on. The reason they do that is not because the business model's changing every minute, it's because they've got an iterative development process. So they understand that the best way to build great software is to build something and put it in front of users like you and me and observe what those users are doing and use that to drive the next iteration cycle. And the people that win are the ones that have figured out how to make that iteration cycle operate at the highest possible velocity. And this is what we're helping companies do and we talk about transforming the way the world builds software. It's this shift from traditional waterfall methodology, requirements, spec development, developers go off into a distant land, they do some magic, they come back nine months later and you've got something that wasn't even close to what you thought you wanted or what you asked for to. Locked into that too, it's like that. The whole waterfall thing is just so slow, why would you go slow? Right, you've missed the market opportunity even if you hit the requirements spec 100%. And so the companies that we're working with, like Ford and many others have recognized there's huge business opportunities but they're not well defined. Like just in our conversation there, we came up with 10, 15 different things that could happen. It's unknown which one of those will prove to be the successful winning next business opportunity for a company like Ford. So the necessity there is really to figure out the right process that allows them to, with consistency and high fidelity, produce great software. And if you can do that repeatedly, you can actually risk, hey I'm gonna place my bets on 10 important things and see which ones succeed and fail and it's okay if some of them fail. I think that's the key. The agile and speed process is the winner. So the folks watching out there, if you're doing waterfall, get the agile as soon as possible, call Pivotal. That's right. They're great. We'll get the plug in. But let's talk about the builders. People out there, the doers, the builders, they're out there trying to build apps. So I wanna get your best practice on this because this is a philosophy or religion or architecture that can be generally applied to the broader market which is a data first, we call it the data first concept which is okay if you think about the data, what is the role of data in that development? So if you're doing agile, dev ops and all that great stuff that we love, okay what's the role of data? Is data gonna be the glue layer? We heard that yesterday in a couple of our interviews that the new glue layer, the new middleware is data. So this concept of decoupling is interesting because that is part of that decoupling. You want the data to be frictionless. At the same time you want it to be an asset in the process. What's your advice and what have you seen as a best practice for people who are using data at the center of it? Because if you're updating the website, I know Facebook does it too. I saw a new thing on my Facebook feed yesterday, profile featured pictures. They got the data behind it, so they need data to make these calls. So again it's instrumental that the data is part of that process. What's your best practice on this? Yeah, so I think there's two interesting things going on here. One is this growing need for telemetry about what's going on with the application. And I'm a big fan of following patterns. I like to look at top performing organizations and copy them. And if I can do 90% as good as the best performing organization, I'm probably doing pretty well. So in the software world, a lot of people like to follow Netflix. They've been amazing about sharing a tremendous amount of information about their internal architecture learnings and actually source code of things they've built. One of the more interesting projects they've come out with in the last couple years is something called Atlas, which is their internal telemetry system. So they're able to understand at a very granular level the behavior of everything on their platform. What it's doing, how it's working, how you're using the new features in that experience. And I think as companies look to get good at software, the thing that drives that iterative development process is data about how people are actually interacting and using the applications. And if you're missing that, you're missing a key ingredient to getting good at this. So that's one thing. The second thing is actually thinking about the data architecture pattern itself. So in enterprise computing over the last 20 years, there's kind of two predominant patterns. One around analytics, which basically says I'm gonna have a series of transactional systems that produce data. That used to be point of sale systems and inventory systems. Today, we've got sensors on everything. So the data source is a little more varied, higher volumes. But I'm gonna take all that data from the systems where it's created and I'm going to shuffle it along into some system where I can then report on it and do analytics and BI. And in the middle, we call that ETL, right? And there's a whole cadre of vendors who have built interesting products to help with that ETL process. But I would say that today, state-of-the-art there is still pretty painful. Eric Frankel from MemSQL talked about this this morning. They just open sourced something called Streamliner, specifically about tackling data ingestion and transformation ahead of the analysis system. So I think there's a huge opportunity to rethink the ETL pipeline in the same way that we've rethought how do we deliver software, right? This idea of continuous integration, continuous delivery, highly iterative, welcoming of change needs to be applied to ETL. Because today, it's generally a static process, team of specialists that build things that generally are brittle and hard to scale. Like, that doesn't work well. And it actually hampers your ability to go fast at the software layer. The other big pattern from a data perspective that I think is in the process of shifting is around what people historically called EAI, or Enterprise Application Integration, which eventually evolved into ESBs. And you see this pattern of how do we connect all these applications together. And again, you end up with this kind of monolithic approach, hidden dependencies. Libraries everywhere. Exactly, very hard to change. If I change things, I generally break it, so I never change things. Again, that slows down your velocity of iteration in such a way that it prevents you from being really good at software. And so I think the world has picked up this development side piece pretty well at this point. Agile is now a common term. People are very happy to move into an Agile software development requirements design process. Continuous delivery is right on the cusp. People are still scared of that. But assuming you can figure out the platform side in such a way that you can deploy without risk, that's great. The next wave is going to be in this data layer of how do we start to think about data in the same way we're now thinking about the application architecture. And you guys are promoting that heavily, I'm assuming. Absolutely. No, I mean, this is a core part of what we're building. So that's a self-serving view of the world, certainly. But it's what we're hearing from our customers. You guys certainly at Pivotal have done a lot of great stuff, certainly on the development side, really pioneering the whole Agile and moving stuff out fast. Great user experience now. You know, get the whole assets underneath that as well from infrastructure standpoint. And you got the spring stuff. And I know we've got to wrap up the segment, but I want to give you a quick last word on promoting the event you guys have coming up. You have a spring event coming up. What is that event? We do. So we have the spring one platform conference. So spring one is a conference that's been around for a number of years. Last year, it was in Washington, DC, had one of the best events we've ever had. We're expanding the focus of that this year. So it'll happen in August 1st of the 4th in Las Vegas at the ARIA. And it will be across all the Pivotal technologies. So Cloud Foundry, Hadoop, Green Plum Database, our Hawk product, Gemfire, the entire product portfolio, as well as all the process around our Agile development methodology, product design, and UX. Very customer focused. We just opened the call for papers a couple of weeks ago. We've already got 20 or so customers who are signed up to come and talk about what they're actually doing. We expect it to be really exciting for both practitioners and executives. Of course, we'll be there with theCUBE. I think that's in progress. So I think theCUBE will be covering. So we're getting on the ground. Again, we want to follow up. Let's talk again. Again, there's a huge thirst for how do I build stuff? I mean, beyond Agile, really going to the next level is about the data. It's about abstracting the way the complexity. And I think that is the focus that we hear from folks is the DevOps is a horizontally deployable environment. And the app is one area, but there's a lot of other stuff on the other. So it's super exciting, a lot of in-depth conversations. Again, thanks for coming on theCUBE. Really appreciate it. Thanks for the insights. Getting the product scoop here from Pivotal. We'll listen to theCUBE extracting the signal from the noise for you. We'll be right back more from live coverage in Silicon Valley for Big Data Week, Big Data SV, and Strata had to do back to the short break.