 Hi everyone, this is Paul Gillan. Welcome to this CUBE Conversation. Topic today is big data and the big data executive survey, the new one out from New Vantage Partners. Summary right here. It is a survey that's been run for several years now and really documents in the Fortune 1000 the progress of big data as moving from trial into the operations of these big companies. The survey is fielded by New Vantage Partners and joining me today is Randy Bean who's the founder, CEO and managing partner of New Vantage or a management consulting firm. They work with Fortune 1000 companies and they specialize or they exclusively work on big data and analytics. So Randy, welcome, thanks for joining us. Paul, thank you for having me. Notice that in the introduction to your study, a study you say that's the final edition of the survey. I believe this is your fourth year running this survey. That's correct. Why the final edition? Yeah, we began the survey in 2012 and as a firm, we host a series of executive thought leadership breakfasts where we bring together CIOs, chief data officers and other C-level executives from Fortune 1000 companies. About four years ago, a CIO from a large financial services firm, actually I guess I can mention it was J.P. Morgan stood up and said, big data's a new phenomenon and we'd really like to understand whether this is something that's taking hold within peer institutions, whether it's being adopted, what institutions are thinking about doing with big data and really what it means for them. So we decided, you know, we're not in the survey business but we decided that we had access to about a couple hundred C-level executives, particularly in financial services and we could reach out to those executives and hear what they were doing. Why it's going to be the last survey is that after four years, we've come to realize that big data has really been widely adopted within most of these organizations. It's not a new phenomenon and so we may come back next year and ask some questions but it will really be a different set of questions. Not so much about what do you make of this phenomenon but what are the measurable results that you're deriving? Well, you say big data has achieved critical mass, your survey would support that. One of your findings is that more than 62% of the firms you surveyed say they now have at least one instance of big data actually in production. Raise a question to me of definitions. How are they defining big data exactly? Yeah, we supplied a definition for all of the survey participants and when we use the term big data, we use it very globally. We really mean big data as referring to all of the data. It can be large data or small data, old data or new data, legacy data, or behavioral data, social media data, can be structured data or unstructured data. When we're using the term big data, we're really referring to the new approaches that have been brought to bear such as Hadoop that have made it much more possible for organizations to be much more agile to democratize their data processes to put data in the hands of business users much more rapidly. So it really changes the whole equation. Traditionally, firms had to go through rigorous data engineering that took a lot of time. Often you'd hear from organizations, it's 15 months and three or $4 million just to get an answer to a simple business question. Big data has really changed that considerably. So when we use the term big data, we're really referencing the agility that's been brought about through these new technologies. Now when a market goes mainstream, the priorities change up until now for the last five years, big data has really been struggling to get a foothold and there's been a very active growth ecosystem, a lot of startups in this area. Now it's going mainstream. How do you see that affecting the market? Well, in the early years, a lot of organizations were focused on doing proof of concepts, pilots, setting up an analytical sandbox, setting up a big data laboratory, a center of excellence. In other words, they were trying to prove out what would be the most relevant use cases for big data. So now we're at a stage where almost, well, the 62.5% of organizations now have big data operationalized or enterprise within the organization. But what we're learning is that organizations aren't really able at this point to point to the business value metrics, in other words, the measurement, what they're getting back in terms of the return on their investment. So many organizations can point to having created a big data environment but very few organizations can point to the return that they're getting from their big data. And in future years, we see that as being the major challenge, the major opportunity. And if we did the survey in the future, what would be looking to measure? Your survey also found there's been a dramatic increase in the number of people with the title of Chief Data Officer. Now this has been a somewhat controversial position because it is different things to different people. In some cases, it may be in the marketing organization. Sometimes it's IT. So a superset of IT, a subset of IT. Were there any threads or trends that you saw in how this role is being defined? Well, I could speak for a very long time on this particular topic. As a matter of fact, yesterday I was a keynote speaker at the first International Society of Chief Data Officers sponsored by MIT, Massachusetts Institute of Technology. You know, when we first did the survey in 2012, I believe it was 12% of organizations had Chief Data Officers and now it's well over half of organizations. But there's no clear definition of what the role means. Last week I was on the West Coast with Charles Schwab, their Chief Data Officer reports to the Chief Marketing Officer. Yesterday at MIT was speaking to a number of organizations, many organizations, the Chief Data Officer reports to the CIO and instill other organizations, for example, JPMorgan reports into the Chief Financial Officer. So it really varies from organization to organization. There's also this notion of the offensive versus the defensive role. Many Chief Data Officers were hired due to particularly in financial services because of the demands for regulatory requirements. For some organizations, it was a box that needed to be checked off to satisfy the regulators. But at the same time, many Chief Data Officers bring strong analytical capabilities and they are looking at how can organizations take the data that they have, leverage it as a corporate asset, create new products and services, create new capabilities. So that's something that a lot of Chief Data Officers aspire to and would like that to be the focus of their work. But to this point in time, so much of their time is responding and firefighting and responding to regulatory issues. So there's what the Chief Data Officer role has been and there's what the Chief Data Officer role could be going forward. Does it appear to you that the Chief Data Officer role could actually become a more important position, more important role in the organization than the CIO because this is the person who is the steward of that corporate asset data? Well, that's funny that you asked that because I wrote a column about three years ago for CIO Magazine entitled Why the Chief Data Officer Will Subplant the Chief Information Officer. This was really meant to be more of a provocative point for discussion and thought as opposed to a firm position I was taking. It's really unclear what the role of the Chief Data Officer will be and because it's a new role and because it's raised some prominence within a number of organizations, it's also a little bit of a target too. Some organizations now were a few years into having a Chief Data Officer and they're starting to say, well, what exactly are we getting in terms of a return for this role? So it's both an opportunity and a challenge. It's a new role and the path that will be taken is to be determined. Sort of in that vein, of course, the CIO typically responsible for the technology and the platform's Chief Data Officer for the data itself. Your survey found, surprised me at least, that technology platforms were actually a minor consideration in implementing big data. Was that surprising to you and does it have implications for the IT organization's role in these initiatives? It wasn't entirely surprising for two reasons. First of all, in terms of new vantage partners, when we started 15 years ago, we assumed that we were gonna be focused 95% on technology and 5% on other issues. In reality, 15 years later, we found that we've been focused 5% on technology and 95% on issues of organizational alignment, business process, business adoption. So it's really the soft issues, people issues. There's a variety of technologies you can use to solve almost any big data or analytics problem. It's more what is an organization comfortable with? Where do they have the traditional relationships? What type of environment are they setting up? But regardless of what most organizations are looking to do, there's really three or four or five different technologies that they can use to address the challenge that they're facing. So it's not so much a technology issue from our perspective. It's organizational and business process and communication and coordination issues that are the real key and ingredients to success. One thing that I'll add that's a major highlight of the survey is that the partnership between business and technology organizations was a consensus in terms of the most, the most, the key factor in the success of big data initiatives. And that's something that was not the case several years ago. And sort of related to that, another finding that surprised me a little bit was that big data initiatives are currently being driven more by operational efficiency rather than business transformation, customer experience. We hear so much about disruption and about customer centric experiences, but companies are still really using this to cut costs and get leaner and meaner, aren't they? Well, for many people, that's a major disappointment because there's been the hope and the aspiration that big data would really be used to some exciting things. A number of folks say to me, a number of C executives say, we're all of the sexy applications. You can think of the Uber and some things of that kind, but in terms of large financial services firms, in terms of life sciences companies, right now it's primarily a migration of processes from mainframes to Hadoop type of environments. It's not the really sexy stuff. I know some of the large credit card companies have written about how in the past they might look at a subset of data to make credit decisions. It might be a year's worth of history and big data creates the ability so that they can look, for example, at 10 years worth of history and actually do it within a fraction of the time. So there's some benefits of those kinds, but these are largely behind the scenes and not sexy new capabilities that consumers see and there's a real desire and interest and a lot of executives and consumers are hoping to see the sexy applications that result from big data. Do you think those will come? Do you think that IT companies are sort of looking at the low hanging fruit right now and those transformational applications are more difficult, more ambitious? Absolutely. They have to look at the low hanging fruit to provide the initial justification from a return on investment perspective and there's some processes that are very obvious to move from a mainframe to a Hadoop environment, but big data really should be looked at from the point of perspective. It enables discovery and it enables it in a rapid fashion. A big data environment creates a much more iterative process where organizations rather than coming up with a hypothesis in advance can learn as they go. They can look and see where the correlations are, load more data, continue to iterate in that process. Sometimes they use the metaphor about failing faster and failing more frequently. That's what big data enables, but in terms of some of the more innovative applications, those have been slower to come than I think many people hoped for. So Randy, Gartner last year made some waves with a study that said that most big data initiatives were just still on the starting blocks, not paying any dividends yet. Your survey indicates something quite different. Was Gartner looking at something else or has the market really changed that fast? Well, I think it's part true. It depends upon whether you look at the glasses half full or half empty. If you look at where we were four years ago and the degree to which organizations have implemented, have stood up big data environments, whether it be big data labs or analytical sandboxes that support discovery type of activities, there's been substantial progress. So that's in the majority. In terms of creating new products and services, new capabilities, firms are still in the infancy. It's much like the adoption of the internet. There was the early stage when people were saying, hey, I'm using the World Wide Web and then 10 years down the road, nobody ever said, hey, I'm using the World Wide Web, they just did it. Excellent analogy. Another surprise in your research was relatively low interest in the use of unstructured data. And of course, there's been so much talk about this, so much excitement about the ability to finally throw a rope around all these text documents and social media conversations that make some sense out of them. But it appears from your research that that's not really happening yet. Well, it's not the most important area thus far. To my surprise, a lot of firms are focusing on bringing in legacy sources of information that they hadn't been able to capture before. You know, you look at the typical data warehouse that may have 10, 20, 30 data sources, but there are many others that have not been captured. I also had the opportunity to spend some time and write an article based upon a conversation with Michael Stonebreaker last summer. And the article was about the long tail of big data and think, for example, in terms of scientists where there's tens of thousands of scientists within an organization and each has a separate database. And traditionally, these haven't been brought together into the data repositories, into the data warehouses. So there's a long tail of data that organizations are beginning to think about and focus on. There is an interest in unstructured data, but it's really been secondary to these other two primary drivers. You have, going back to your finding about the relatively low interest in platform issues. Of course, the vendors talk about technology issues a lot. We hear a lot of speeds and feeds out of the vendor community. Are they maybe talking past their customers? Should they be focusing more on strategic and business bottom line benefits? Well, I think that big data initiatives are driven by the business. There's a lot of technology solutions out there. It's hard for business folks that just focused on they want faster answers to their business questions. There's a lot of different terminology that's being used, accelerating time to answer, time to decision, time to analytics, speed to market. That was one of the indications of the survey. These were the primary drivers from a business perspective that organizations were looking at. There's a lot of technology solutions. That's not the critical gatepost at this point in time. You're very active in getting your opinions out there. Your columnist for the Wall Street Journal. You do a lot of writing, a lot of speaking. Where can people find out more about what you're thinking and follow you? Well, I'm out there on Twitter. wwwnewvantage.com is our website. And we have all of our content, the results of the survey, the Wall Street Journal articles, recent MIT Sloan review articles. And we run a series of thought leadership breakfast around the country. Next one's in New York on February 3rd with C executives from leading Wall Street firms and large pharmaceutical companies. That's it for this episode of Q Conversations. Be sure to follow us on SiliconANGLE TV as well as on our SiliconANGLE channel on YouTube, SiliconANGLE on Twitter. And stay tuned for further episodes in our Cube Conversation series. This is Paul Gillan.