 From the SiliconANGLE Media office in Boston, Massachusetts, it's theCUBE. Now, here's your host, Stu Miniman. I'm Stu Miniman and this is a CUBE conversation from our Boston area studio. We spent a lot of time talking about digital transformation and of course, at the center of that digital transformation is data. In this segment, we're going to be talking about the data integration platform. Joining me for that segment is Idemar Ancorion, who's the Senior Vice President of Enterprise, data integration with CLIC. Thanks so much for joining me. Thanks Stu, glad to be here. All right, so as I just said, customers, digital transformation, when you talk to any user, there's some that might say, oh, there's a little bit of hype or I don't understand it, but really leveraging that data, there are very few places that that is not core to what they need to do and if they're not doing it, their competition will do it. So can you bring us inside a little bit that customers you're talking to where that fits into their business needs and how the data integration platform helps them solve that issue? Absolutely. So as you mentioned, digital transformation is driving a lot of innovation and a lot of efforts by corporations and virtually any organization that we are talking to sees data as a core component of enabling the digital transformation. The data creates new analytics and helps to power the digital transformation whether it's in making better decisions or whether it's embedding the analytics and the intelligence into business processes and customer applications to able to enrich the experience and make it better. So data becomes key and the more data you can make available to the process, the faster you can make it available in the process, the faster you can adapt your process to accommodate the changes, the better it will be. So we're seeing organizations, virtually all of them, looking to modernize their data strategy and their data platforms in order to accommodate these needs. Yeah, it's such a complex issue. We've been at chief data officer events. We talk about data initiatives. We worry a little bit that the C-suite sometimes here, it's like, oh, they heard data is the new oil and they came and they said, according to the magazine I read, we need to have a data strategy and give me the value of data. But where's the rubber hitting the road? What are some of those steps that they're taking? How do I help get my arms around the data and then help make sure it can move along that spectrum from kind of the raw ore to real value? Yeah, I think you hit a great point, talking about the ore to value or as we refer to it as a road to ready. And part of the whole innovation that we're seeing is the modernization of the platform where organizations are looking to tap into the tremendous amount of data that is available today. So a couple of things have happened. First of all, in the last decade, first of all, we have significantly more data that is available than ever before because of digitization of data and new sources that become available. But beyond that, we have the technologies, the platforms that can both store and process large amounts of data. So we have foundations, but in the end to make it happen, we need to get all the data to where we want to analyze it and find the ways to put it together and turn it from more raw material into ready, material, ready products that can be consumed. And that's really where the challenge is and we're seeing a lot of organizations, especially the CDO, CIO, the Enterprise Architecture and Enterprise Data Architecture teams on a journey to understand how to put together these kind of architectures and data systems. And that's where with our data integration platform, we focused on accommodating the new challenges that they've encountered in trying to make that happen. Yeah, help us unpack a little bit. You know, here today, you know, it's the API economy, everything should work together. When I rolled out, you know, in our company, you know, the industry's leading CRM, it's like, oh, I've got hundreds of data sources and hundreds of tools I could put together and it should be really easy for me to just, you know, allow my data to flow and get to the right place. But I always find a lot of times that easy button, I've been having a hard time finding that, so. So that's a good point. And if you kind of take a step back and understand what are some of the core challenges or the new needs that we're seeing because we talk about digital transformation and modern analytics fueled by data being part of it. Modern analytics created new type of challenges that didn't exist before and therefore kind of traditional data integration tools didn't do the job, they didn't meet those modern needs. And let me touch on a few of those. So first of all, when customers are implementing modern analytics, many times what they refer to is scale. What they're trying to do is to do AI, machine learning. We all use those terms when we talk about them but machine learning and AI get smarter the more data you give them. So it's all about the scale of data. And what we're seeing with customers is where if in the past data warehouse system would have typically had five, 10, 20 data sources going into it, we're now seeing 100x times that number of sources. So we have customers that work with 500, 600, some over 2,000 sources of data feeding the data analytics system. So scale becomes a critical need. And when you talk about scale you need the ability to bring data from hundreds or thousands of sources, source systems efficiently with very low impact. And ideally to do it also with less resources because again you need to scale. The second challenge we ran into has to do with the fact that modern analytics for many organizations means real time analytics or streaming analytics. So they want to be able to process data in real time and respond for that. To do that you need a way to move data, capture it in real time and be able to make it available and do that in a very economic fashion. Then the third one is in order to deal with the scale in order to deal with the agility that the customers want the question is where are they doing the analytics and many of them are adopting the cloud. And we're even seeing multi-cloud adoption. So in order to get data to the cloud now you're dealing with a challenge of efficiency. I have a limited network bandwidth. I have a lot of data that I need to move around. How can I move all of that and do that more efficiently? And the only thing that would add to that is that beyond the mechanics of how you move the data with scale, with efficiency, but in real time there's also how you approach the process where the whole solution needs to be way more agile in terms of the iterations you can implement and accommodate any type of architecture any type of platform that you may choose and we're saying customers change those over time. So you need the ability to be agile and flexible. Yeah, well, a lot to unpack there because, you know, I just make the comment, you know, if you talk about us humans the more data we give them doesn't mean I'm actually going to get better. It's, we need to be able to have those toolings in there to be able to have that data and help give me the insights which then I can do on. Otherwise, you know, we understand most people it's like if I have to make decisions or choices and I get more thrown at me there's less and less likelihood that I can do on that. And boy, the data lakes. Yeah, I remember the first time I heard data lakes it was, you know, we talked about what infrastructure we're building and now for the last couple of years the end of the cloud, public cloud tends to be a big piece of it even though we know data is going to live everywhere. You know, everything not just public and private but Edge gets into a piece of it. So, you know, the data integration platform, you know, how easy customers get started on that. We can talk about that diversity of everything else. You know, where do they start? Give me a little bit of kind of that customer journey if you would, and maybe even if you have a customer example that would be a great way to illustrate it. Absolutely. So first of all, it's a journey. And I think the journey started quite a few years ago. I mean, Hadoop is now over 10 years old and today we're actually seeing a big change in shift in the market from what was initially the Hadoop ecosystem into a much broader set of technologies especially with the cloud in order to store and process large scales of data. So the journey customers are going through we had a few years which were very experimental. Customers were trying to get on for size. They were trying to understand how to be the process around it. The solutions were very batch oriented and would map reduce back in the early days of Hadoop. But when you look at it today, it's a very, it's already evolved significantly. And you're seeing these big data systems needing to support different and diverse type of workloads. Some of them are machine learning and science. Some of them are streaming analytics. Some of them are serving data for microservices to power digital applications. So there's a lot of need for the data in the journey. And what we're seeing is that customers as they move through this journey, they sometimes need to pivot and they need to find new technologies that come out and they need the ability to be able to accommodate to adapt and adopt new technologies as they go through it. So that's kind of the journey we have worked with our customers through. And as they evolved, once they figured it out, the scale came along. So it's very common to see a customer start with a smaller project and then scale it up. So for many of the customers we've worked with, that's how it worked out. And you asked for an example. So one of our customers is one of the words largest automotive companies. And they decided to have a strategy to turn what they believe is a huge asset they have, which is data, but the data is in a lot of silos across manufacturing facility, supply facilities and others inventory and bring it all together into one place, combine the data with data they bring from the car itself. And by having all the data in one place, be able to derive new insights and new information that they can use, as well as potentially sale or monetize in other ways. So as they got started, they initially started by rolling it out to a certain number of their data centers and their source of information, manufacturing facilities. So they started small, but then very quickly, once they figured out they can do it fast and figure out the process to scale it, today they're at over 500 systems. They have a mouth is over 200 billion changes in data being fed daily, okay, into their data lake. So it's a very, very large scale system. And for all we can talk a little bit about what it takes to put together something so big. Yeah, now please take the next step, that would be perfect. Okay, so I think one of the key things customers have to understand, and we're seeing that with enterprise architecture teams, is that when you need to scale, you need to change the way you think about things. And in the end of the day, there are two fundamental differences in the approach and the underlying technology that enable that. So we talked earlier about a lot of things hard for the mind to understand. Now I'm gonna focus on and highlight only two. That should be easy to take away. First is the move from batch to real time or from batch to the delta, to the changes. Traditionally, data integration was done in the batch process. You reload the data. Today, if you want to scale, if you want to work in real time, you need to work based on the delta on the change. The fundamental technology behind it is called change data capture. And it's a technology and approach that allows you to find and identify only the changes in the enterprise data systems. And imagine all the innovation you can get by capturing and processing only the changes. First of all, you have a significantly less impact on the systems. So you can scale because you're moving less data. It's very efficient as you move the data around because it's only a fraction of the data. And it can be real time because, again, you're capturing the data as it changes. So the move from batch to real time or to streaming data based on change data capture is fundamental. Fundamentally creating a modern data integration environment. Yeah, I'm assuming there's an initial load that has to go in, something like that. Correct, but you do that once. And then for the rest of the time, you're really moving only the deltas. The second difference, I mentioned there are two. So one was, again, moving from batch to streaming based on change data capture. And the second is how you approach building it, which is moving from a development-led platform to automation. So through automation, you can take workloads that have traditionally been in the realm of the developer and allow people without development skills to be able to implement such solutions very quickly. So again, the move from a developer tool to configuration-based, automation-based product. So what we've done at the Trinity is, first, we have been one of the pioneers and the innovators in the change data capture technology. So the platform that now is part of the click data integration platform brings with it over 15 years of innovation and optimization in change data capture with a broader set of data sources that are supported with lots of optimization, ranging from data sources like SQL Server and Oracle, the mainstream to mainframes and to SAP systems. And then one of the key focus we've done ahead is, how do we take complex processes and automate them? So from a user perspective, you can click a few buttons, turn a few knobs, and you have the optimized solution available for making moving data across the diverse sets of systems. So through moving on to the delta and doing automation, you allow the scale. So a lot of the systems I'm familiar with, it's the metadata comes in the system. I don't have to, as an admin or somebody setting that up, I don't have to do all of this, or even if you think about the way I think of photos these days, it used to be I took photos and trying to sort them was ridiculous. Now my Apple or Google, not only facial recognition, but timestamp location, all those things, I can sort it and find it, it's built into the system. Absolutely, and metadata is critical to the whole process. First of all, because when you bring data from one system to another system, somebody needs to understand that data. And the process of getting data into a lake and into a data warehouse is becoming a multi-step data pipeline. And in order to trust the data and understand it, you need to understand all these steps the data went through. And we also see different teams taking part in this process. So for a team to be able to pick up data and work on it, it needs to understand it's metadata. By the way, this is also where the click data integration platform, bring together the Attunity software together with click data catalyst will provide unique value proposition through that. Because you have the ability to capture change data as it changes, deliver that data virtually anywhere. Any data lake, any cloud platform, any analytic platform, and then refine the data to generate analytic ready data sets. And together with the click data catalyst create derivative data sets and publish all of that through a catalog that makes it really easy to understand which data exists and how to use it. So you have an end-to-end solution for streaming data pipelines that generate analytic data sets for the end of the day, roll to ready in an accelerated fashion. Yeah, so Inamar, your customers have rolled that out. How do they measure this? So are there critical KPIs? Is there some journey map that they help go along? What are some commonalities that you find? So it's a great question. And then naturally for many organizations, it's about an ROI. It's about total cost of ownership. It's seeing results, as I mentioned earlier, agility and the time to value is really changing. Customers are looking to get results within a matter of very few months and even sometimes weeks versus what it used to be, which is many months and sometimes even years. So again, the whole point is to do it much, much faster. So from a metric for success, what we're seeing is customers that buy our solution to enable large-scale strategic initiatives where they have dozens to hundreds of data sources. And one of the key metrics is how many data sources have you on board? Have you made available? How many in the end of the day, data assets that are already analytic ready, have we published or made available to our users? And I'll give you another example from one of our customers. Again, a very large corporation here in the United States and they bought its unity after trying to move to the cloud and build a cloud data lake and analytic platform. And in two years, they were able to move two to three data sets to the cloud. After they tried the new data integration platform, they moved 30 data sets within three months. So completely different result. And the other thing that they pointed out when they actually talked about their solution is that unlike traditional data integration software, and they took an example of one of those traditional ETL platforms, and they pointed out it takes seven months to get a new person skilled on that platform. Okay, with our data integration platform, they could do that in a matter of hours to a few days. So again, the ability to get results much faster is completely different when you have that kind of software that goes back to what I mentioned about automation versus development-based platforms. It really seems like the industry is going through another step function just as we saw from traditional data warehouses to when Hadoop rolled out, that just the order of magnitude, how long it took and the business value return seems like where we're going through yet another step function there. So final thing, yeah, what's some of the first things that people usually get started with? Any final takeaways you want to share? Sure, first of all, what people are starting to work with is they're usually selecting a platform of choice where they're going to get started in respect to where they're running analytics. And the one takeaway I'll give customers is don't assume that the platform you chose is where you're going to end up because new technologies come to market and new options come, customers are having mergers, acquisitions, so things change all the time. And as you plan, make sure you have the right infrastructure to allow you to kind of pivot and support and make changes as you move through all these innovation. So it'll be maybe a key takeaway. And the other one is make sure that you're building the right infrastructure that can accommodate speed in terms of real time, accommodate scale, okay? In terms of both enabling data lakes, including cloud data stores, having the right efficiency to scale, and then enabling agility in respect to being able to deploy solution much, much faster. Yeah, well, Idomar, I think that's some real important things to say. We know that the only constant internet industry is change and therefore we need to have solutions that can help keep up with that and be able to manage those environments and the role of IT is to be able to respond to those needs of the business fast because if I don't choose the right things, the business will go elsewhere to try to fuzz it in. So thank you so much for sharing all the latest on the integration data platform. It's been a pleasure. All right, always watch more on thecube.net. I'm Stu Miniman, as always. Thanks for watching.