 From Orlando, Florida, extracting a signal from the noise. It's the Cube, covering Pentaho World 2015. Now your host, Dave Vellante and George Gilbert. Welcome back to Pentaho World 2015, everybody. This is the Cube. We go out to the events, we extract the signal from the noise, Silicon Angle, Wikibon. Pedro Martins is here. He runs visualization at Pentaho. In Cape to Pentaho, Pedro from a company based in Portugal, Portugal, and Soccer Central, called Web Details. And we're going to talk about the Viz, a very important part of the Pentaho platform. So Pedro, welcome to the Cube. Thank you so much for having me. So tell us about the history of how you guys came about and how you ended up getting tucked into Pentaho. Sure, so it's a story that goes back even beyond my times at Web Details and Pentaho. I started in 2010, but the Web Details team actually started in 2008. And the whole idea was to leverage Pentaho as a platform using the open source technology and using the capability that Pentaho has to develop new plugins that could be used to improve its visualization capabilities. Back in those times, Pentaho was known for the pure strength of PDI. Data Blending was already coming over there as a term. People were really happy on the ETL piece, but visualizations needed some refinement. And what happened was that we created a set of tools which today are known as C-Tools that created an abstraction layer. So it allowed people without technical skills to start developing visual content that could aggregate the business information that they wanted to extract from their data. This continued for a few years. We ended up having customers from several places in the world. Actually, Mozilla was one of our big customers that's known. And Pentaho started being one of our customers as well. So we started providing some solutions and visualization, some upgrades in different aspects of the platform and the plugins as well. And then there was the point where we were so well aligned that two years ago, we got acquired and made part of this. That's an amazing story. 2008, imagine betting your company in 2008 on Pentaho. Nobody had heard of Pentaho. Nobody had heard of Hadoop, right? That's vision. And so the integration into Pentaho was essentially already done. Yes, indeed it was. We had already units that were very well aligned with the Pentaho philosophy, right? For instance, I run the services team on visualizations. So what we did was basically to integrate a team that operates globally in custom visualizations for every customer that needs to have a pixel perfect dashboard and wants to go beyond just the traditional offer. The development team has now several branches. So it started initially doing pure development just for the C tools plugins on visualizations. But then they start scattering the people across different teams. So these days we'll find people in the US or in Portugal building elements either for Pentaho Analyzer, for Mondrian or for DTL like PDI or other elements over there. So we have people everywhere and the teams are always rotating. And their team names are quite fun. Talk about open source culture within your team and how that impacted the, I guess the acquisition, but more importantly the integration and go to market post acquisition. Oh yes, so there was always a Latin fear that with the acquisition our open source philosophy will be affected by maybe trying to close the code. But that was one of the things in which Pentaho was really aligned with us in trying to keep all the development still open and actually use the community like we were using with the C tools, C stands for community. Must use the community as the lab, as the ramping platform that we have to kick start something that could later mature and become an offer that we will allow to exist in the community. But then also let's say mature as a product internally to become an enterprise plugin. It's all keeping the open source philosophy. So all this code is still open and it's a way of keeping our customers comfortable with the knowledge that they can still look at what has been done, what are the inner works of the plugins and the solutions and the platform over there and come to their own conclusions when we're doing some sort of development. So let's talk segue into the visualization piece. When you think about the legacy data warehouse and BI business, it is slow. You have to go through this narrow pipeline process. You have to find the right people who know how to build the cube. It just takes a long time. It's not in the hands of the users. You are changing that world. I mean, that's not, I mean, that's what you do, but Pentado as a company is doing a lot more. But talk about what's changed in that whole notion. I don't even want to call it business intelligence anymore because to me it's a pejorative. Just in terms of being able to take action from insight, systems of intelligence, which is sort of the goal that we always talk about. Talk about what's changed in that whole, making decisions business. Yeah, so it's a fun roller coaster, right? So on the way that we are working on making sure that access to data is fast and snappy and make data available to the end users so he can make informed decisions as soon as possible, it's a challenge because the market is changing very quickly as well. And you get existing customers who are really, really happy playing now with new big data technology. They are migrating to Hadoop, but they are still using older technology. And one challenge that you find on the business side when you're doing visualizations, it's trying to keep the business description faithful to its main purpose, right? So we like to have this abstraction between the business and the back end, independently on how you're blending the data. If you're using your old style structured data, if you're running to unstructured data or you're running from structured data, if you're just doing this kind of transition, you should never change your visualization or changing your approach to the business just because you're changing the back end. And Pandaho provides that because on this last piece, so we have been speaking today and yesterday about all the streamlined data refinery. So how can we get all the information into a data lake and then just extract a model or basically a result set that can be digested, so to speak, by our visualizations? So this result set is actually agnostic to what is behind it. So by using this concept on the visualization layer, on the Seatles layer, what we can do is to keep the same visualization consuming the data whether it's coming from place A and place B. So that allows you to keep a focus on what really matters, which is the business data, and then act on it by providing, for instance, predictive analytics on top of it. They run on top of basically this result set without caring so much where it's coming from. And then you can, again, go back to the other plugins. You can have, for instance, the PDI layer to send alerts when something is not going well. You can actually have a visual system that can alert users that an action has to be taken and make a prescriptive action on some elements that are requiring your attention. And that's a very powerful capability that is now available because of this modular approach that was done when putting a pent-up solution together. So I'm curious, it's interesting and compelling to hear that the visualization can hide differences in where you're getting the data and what you're doing to it, perhaps. Yeah. How much of a role does the visualization play in selling, let's say, the packaged app that it might be embedded in? Or in selling to the end user community the custom app that a company might be building it into? It really depends from customer to customer, right? So you will find that smaller customers will rather build the visualization themselves. They will rather build the solution themselves. Some other customers will rather have the capability of creating a custom dashboard that is predefined in the control environment, so their end users don't get exposed to elements that might not be the best for the business goals. Our offer aims a lot at integrating the visualization approach that makes them successful. Our main goal is to make the customer successful. And by using this, for instance, my team is involved very often in discussing before even the ETL process starts, how would you like to draw your solution? So it's very common that we sit down with a piece of paper or even an app can and start just sketching how would you like your solution to work, right? How do you want to navigate? What are your KPIs, the breakdowns? And then we bring the technical people who will look into those breakdowns, those KPIs and understand where they are on the data. I'm not going to speak about the blending process. This is, I mean, what you're describing is really critical, but it's also classic data warehouse, BI sort of roles. I want to display this type of information, these measurements, and so we have to go get it here and pump it through the pipeline. Do you see that, has that been changing now that Pentaho has a pipeline that can pull data in from the data lake where you don't have to source everything precisely? In other words, all the data's there, you know, and you have faster and faster ways of projecting it onto the glass. Does that make a difference in the solutions? It does make a difference in the solution because now you're getting to a point where you have immediate access to your data and your data is mutating, right? You're adding properties, you're adding things, so all those things can become exposed. And you can basically, so your traditional dashboard is evolving into a situation where you actually mutate it to adapt to your current business situation. So there is a new property that comes in, you can select it, you can look into it, and then drill into it to see how that looks like and act on it. So that's actually really significant because that goes to sort of, it's a step in the agile direction where it's like, okay, we want to measure something new, what's the time to get that on the end user screen? How does that change? How much of the lead time has changed from when you had to rewrite the ETL pipeline versus now, you know, you source it from the data lake through Pentaho? I'd say it's a huge leap, right? Before you will need some back-end expert to go through the data, run an ETL, check the data model, and then from checking the data model, there will be probably some queries that need to be upgraded or at least an exploratory tool that will go through the new model to extract it. At this point, with a refinery in place, you will actually get all the blending is done and this additional property or additional dimension or additional KPI is immediately available at the end. So you cut the time on the back-end development, maybe there will be some inline configuration that needs to be done, that's natural, but you immediately focus on the business application of that additional property. Meaning the measurement that the user needs to see. Exactly. So would that be a 90% reduction in time or 50% or 95? Well, traditionally we work with two people, right? Usually have one guy working on the front-end and business one guy working on the back-end. If we say that we are removing, let's say almost 50% of the work which is the guy working on the back-end, I'll say that you have at least a 50% improvement in the development time to make this kind of agile approach to business. Okay, is there, you had mentioned about the predictive intelligence, how does that figure into the visual elements of the dashboard? So, predictive, there are many ways of doing predictive. I'm not going to speak here about what data analysts do because that's usually a group that has their own tools very well defined and they are very happy with those. But you can still use the visualization layer for the end users and provide them with some predictive capability at a very quick level. It's very, very simple to integrate in a visualization layer because we use standard JavaScript technology, a linear regression or any kind of polynomial regression just to apply to the data that comes over there. Can you give us an example, like something that would recommend a user to do such and such instead of what he was used to doing? Yes, for instance, we have, there's a demo on the Pentaho website for the Big Wireless. It's a fictional company called Big Wireless where you can actually see how this company is doing at selling phones and other elements. You can actually go to an order manager which is an application developed using the same dashboard technology of the C-Tools where you can see how your inventory is going and then there is over there already a linear regression that will tell you when do you need to buy more stock or basically how much money do you want to spend? That will give you stock for the next month, the next two months or three months. It's a very quick application that immediately allows you to see what do you need to do to act on this and has already a way of pushing this down into your ERP or any other system that is processed your order. This is going towards you, all right. This is really important. We have this theme systems of intelligence where you take the operational systems, you put an intelligent layer on them that helps advise either a human operator or automatically makes the best decision. So elaborate on this part where you're taking a decision about what's the best next course of action. Elaborate on that example if you can. But I should say, what was the, beyond the visualization tools, what underneath was making that happen? So underneath the visualization, that's really, it's no rocket science there to be honest. It's what it makes it possible to people like me and the guys on the consulting team to make it available. So the decisions are always made regarding the business. So within Pentaho, we are not aiming at being the best at your business. We still believe that you are the one who has to advise us how would you like to take a course of action depending on the data that we are helping you to achieve. What we put in place on the back end is just really a structure to be able to process the data very quickly on the back end because all our visualizations are processed on the client side, but people are using phones these days to access data. I don't want to run regression analysis on my phone. That will not work. So we can run either WECA or we can run simpler analysis, like I said before, with linear regressions to make sure that this makes a much faster approach. And then you can create a decision system using, you can use basically PDI if you want to visualize what kind of actions are to be done or you can use just JavaScript if you are in the coding aspect of things to see where you want to go. And then from that point you can either do one of two things. Either you do the predictive approach where you display to the user what is going on or you can do the prescriptive approach where this information is immediately pushed down into a system that can trigger an alert and at the same time take action on top of it. One aspect, sorry for that, but one aspect that is important is to quantify how safe the decision that you made is done. So basically that's done using statistics that you can apply to understand whether you actually doing this in the best manner or not. So this either prediction or prescript, prescription. Yeah. Is that an engine that, or is that a model that's being deployed into this workflow, part of which is the visualization, part of which has the engine in it or the model in it where the engine prepared the model and that was done offline by data scientists with their own tools and stuff like that. So Pentau's approach for predictive analytics and for WEC actually, which is our offer is really to make you enable to use it. So we are not building a predictive analytic solution. In this case, what I'm describing is what we do typically with services where we use the PDI as a solution to process the data and then on the visualization layer we can integrate usually just a custom set of code. It's usually a small module that is usually maintained differently because it's business is different honestly. Every time we speak about analytics, they change that. And what is happening is that when we end up using this kind of approach very, very often then product management will productify this and make part of the current offer. So it sounds like there's a framework that says, okay, there's room for the model to predict or prescribe and then you have this visualization layer which presents the information and you might or might not need the human interaction. What we provide today is basically a plugin that allows you to create other Pentau plugins without having to resort to Java coding. You use PDI to describe the logic I use C tools to describe the visualization and this can push down information and pop up information. So, yes. If I may, Pedro, when you get called into an engagement what's it like? Why are they bringing you in? What kinds of things are customers asking you to do? So when we get called to an engagement usually it's two people from our team at Pentau that speak. So I'm the head of the technical team on services and then Nuno is our head of UX. We think it's very important to have the first conversation with the customer speaking about business. People get lost on technical details and engineers tend to be lazy so they want to do a solution that it's easier to implement and it's easy to maintain and sometimes you may lose focus on what kind of things you could achieve if you were a bit less lazy. So that's interesting because I keep hearing they want to solve the hard problems even if they're not the relevant problems. Exactly. So that's what we do in the beginning. We get to a customer and we ask them what are our KPIs? What drives your business? So how would you break this down to understand where can you improve your business to make you more successful? And it's very common that afterwards we ask them to draw things. We are very visual. I mean, this is a visualization team actually. So we asked them to get something described and our designers usually put that together. We try to respect all the branding and make pixel perfect mockups. This is really done to the pixel level and from that point onwards then we do the actual engineering, right? So we do a functional breakdown describing which components are over there. What kind of queries or what kind of data is driving each one of those components? Performance is assessed. And then, of course, there's the point where we decide with the customer who is doing what, right? So very consultative. Yes. How do you look at the market? There's been a couple of, you know, last few years, the market has broken out in visualization. You've seen Tableau and Click. You've got multi-billion dollar valuations. How do you guys look at the market? So the first thing we do on the market is not trying to compare ourselves with others feature by feature because we do a different approach to things. So whenever we try to do a feature by feature comparison, we end up comparing the stuff that we know and at least on my side, my knowledge on other BI tools that is actually quite limited. So I'd rather tell you how I'll do a solution for you using my technology and then you make an informed decision whether that makes sense or not compared to your other vendors. I think that our offer in visualizations and in BI as a complete solution is great in the sense that it's complete. So if you speak to us, we'll be able to take you from your raw data all the way to the visualizations. You already know that you can use ad hoc plugins to create at your own will your own reports. On the visualizations thing, we can actually help you building custom visualizations that may not be available today on the plugins and make that part of your specific solution. Or you can just get the designers to actually design a full-fledged dashboard, pixel-perfect with navigation, several layers of drill downs that will be exactly reflecting your company, your company brand, your ideals. And it's something to which you'll be basically identifying. You'll be basically completely blended and merged with what was already being offered in your previous BI solution. All right, Pedro, we got to leave it there, but I'll give you the last word. Pentaho World 2015, you know, what's the bumper sticker? When people pull away from 2015, what should they be thinking about? What's, from your perspective, what's the catchphrase? I think that's a tricky one. I think the catchphrase for Pentaho 2015, it's going to be that we are here to make enterprise successful. We are going to drive big companies into big data and make sure that they are not afraid of using that data. They will stop being afraid of that data. They will start looking at what they have and try to understand the hidden areas of the data that they didn't want to try because they were going to be too slow to access or too complex to distinguish from the rest. It's really a moment in which they will gain confidence and courage to go, attack it, and then decide how to expose it because there'll be several options provided by us to make it happen. Excellent, business value message, and you're there to help. Cool, thanks very much for coming to theCUBE, Pedro. Thank you for having me there. All right, keep it right there, buddy. We'll be back with our next guest right after this.