 From around the globe, it's theCUBE, presenting ActiveDQ, Intelligent Automation for Data Quality, brought to you by IOTAHO. Are you ready to see ActiveDQ on Snowflake in action? Let's get into the show and tell and do the demo. With me are T.G. Matthew, the data solutions engineer at IOTAHO. Also joining us is Patrick Zymet, data solutions engineer at IOTAHO, and Centil Nitin Carpaya, who's the head of production engineering at IOTAHO. Patrick, over to you. Let's see it. Hey, Dave, thanks so much. Yeah, we've seen a huge increase in the number of organizations interested in the Snowflake implementation, who are looking for an innovative, precise, and timely method to ingest their data into Snowflake. And where we are seeing a lot of success is a ground up method utilizing both IOTAHO and Snowflake. To start, you define your as is model by leveraging IOTAHO to profile your various data sources and push the metadata to Snowflake. Meaning we create a data catalog within Snowflake for a centralized location to document items, such as sources to owners, allowing you to have those key conversations and understand the data's lineage, potential blockers, and what data is readily available for ingestion. Once the data catalog is built, you have a much more dynamic strategy surrounding your Snowflake ingestion. And what's great is that while you're working through those key conversations, IOTAHO will maintain that metadata push and partnered with Snowflake's ability to version the data, you can easily incorporate potential schema changes along the way, making sure that the information that you're working on stays as current as the systems that you're hoping to integrate with Snowflake. Nice, Patrick, I wonder if you can address how the IOTAHO platform scales and maybe in what way it provides a competitive advantage for customers. Great question. Where IOTAHO shines is through its active DQ, or the ability to monitor your data's quality in real time, marking which rows need remediation according to the customized business rules you can set, ensuring that the data quality standards meet the requirements of your organizations. What's great is through our use of RPA, we can scale with an organization. So as you ingest more data sources, we can allocate more robotic workers, meaning the results will continue to be delivered in the same timely fashion you've grown used to. What's more, IOTAHO is doing the heavy lifting on monitoring data quality that frees up your data experts to focus on the more strategic tasks, such as remediations, data augmentations, and analytics developments. Okay, so maybe TG, you could address this. I mean, how does all this automation change the operating model? We were talking to AJ and Duncan before about that. I mean, if it involves less people and more automation, what else can I do in parallel? You see, Dave, I'm sure the participants today will also be asking the same question. Let me start with the strategic tasks Patrick mentioned. IOTAHO does the heavy lifting, freeing up data experts to act upon the data events generated by IOTAHO. Companies that have teams focused on manually building their inventory of the data landscape leads to longer turnaround times in producing actionable insights from their own data assets, thus diminishing the value realized by traditional methods. However, our operating model involves profiling and remediating at the same time, creating a catalog data estate that can be used by business or IT accordingly. With increased automation and fewer people, our machine learning algorithms augment the data pipeline to tag and capture the data elements into a comprehensive data catalog. As IOTAHO automatically catalogs the data estate in a centralized view, the data experts can partly focus on remediating the data events generated from validating against business rules. We envision that data events coupled with this drillable and searchable view will be a comprehensive one to assess the impact of bad quality data. Let's briefly look at the image on screen. For example, the view indicates that bad quality zip code data impacts the contact data, which in turn impacts other related entities and systems. Now contrast that with a manually maintained spreadsheet that drowns out the main focus of your analysis. TG, how do you tag and capture bad quality data and stop that from, you've mentioned these dependencies, how do you stop that from flowing downstream to the processes within the applications or reports? As IOTAHO builds the data catalog across source systems, we tag the elements that meet the business rule criteria while segregating the failed data examples associated with the elements that fall below a certain threshold. The elements that meet the business rule criteria are tagged to be searchable, thus providing an easy way to identify data elements that may flow through the system. The segregated data examples on the other hand are used by data experts to triage for the root cause. Based on the root cause potential outcomes could be, one, changes in the source system to prevent bad data from entering the system in the first place, two, add data pipeline logic to sanitize bad data from being consumed by downstream applications and reports, or just accept the risk of storing bad data and address it when it meets a certain threshold. However, Dave, as for your question about preventing bad quality data from flowing into the system, IOTAHO will not prevent it because the controls of data flowing between systems is managed outside of IOTAHO, although IOTAHO will alert and notify the data experts to events that indicate bad data has entered the monitored assets. Also, we have redesigned our product to be modular and extensible. This allows data events generated by IOTAHO to be consumed by any system that wants to control the targets for bad data. Thus, IOTAHO empowers the data experts to control the bad data from flowing into their system. Thank you for that. So I mean, one of the things that we've noticed, we've written about is that you've got these hyper-specialized roles within the data, the centralized data organization. And I wonder, how do the data folks get involved here, if at all, and how frequently do they get involved? Maybe you can take that. Well, based on whether the data element in question is in data cataloging or monitoring phase, different data folks gets involved. When it isn't the data cataloging stage, the data governance team, along with Enterprise Architecture or IOT, involved in setting up the data catalog, which includes identifying the critical data elements, business term identification, definition documentation, data quality rules and data event setup, data domain and business line mapping, lineage, PIA tagging, source of truth, so on and so forth. These are typically in one time setup, review, certify, then govern and monitor. But while when it isn't the monitoring phase, during any data incident or data issues, IOTAHO broadcast data signals to the relevant data folks to act and remediate as quick as possible and alerts the consumption team, it could be the data science, analytics, business ops, about the potential issue so that they are aware and take necessary preventive measure. Let me show you an example critical data element from data quality dashboard view to lineage view to data 360 degree view for a zip code for conformity check. So in this case, the zip code did not need the past threshold during the technical data quality check and was identified as non-compliant item and notification was sent to the ID folks. So clicking on the zip code will take to the lineage view to visualize the dependent system such that who produces and who all of the consumers. And further drilling down will take us to the detail view where a lot of other information are presented to facilitate for a rootcast analysis and to take it to a final closure. Thank you for that. So, TJ Patrick was talking about the as is to the 2B. So I'm interested in how it's done now, versus before, do you need a data governance operating model, for example? Typically a company that decides to make an inventory of the data assets would start out by manually building a spreadsheet managed by data experts of the company. What started as a draft now gets baked into the model of the company. This leads to loss of collaboration as each department makes a copy of their catalog for their specific needs. This decentralized approach leads to loss of uniformity which each department having different definitions which ironically needs a governance model for the data catalog itself. And as the spreadsheet grows in complexity, the skill level needed to maintain it also increases, thus leading to fewer and fewer people knowing how to maintain it. About all, the content that took so much time and effort to build is not searchable outside of that spreadsheet document. Yeah, I think you really hit the nail in the head, TJ. Now companies want to move away from the spreadsheet approach. Ayotaho addresses the shortcoming of the traditional approach enabling companies to achieve more with less. Yeah, I'm interested in what the customer reaction has been. We had Webster Bank on one of the early episodes, for example, I mean, could they have achieved what they did without something like active data quality and automation, maybe Centel, Nitin, you could address that. Sure, it is impossible to achieve full data quality monitoring and remediation without automation or digital workers in place. Reality that enterprise, they don't have the time to do the remediation manually because they have to do an analysis, confirm, fix, and any data quality issues as fast as possible before it gets bigger and no exception to Webster. That's why Webster implemented Ayotaho's active DQ to set up the business metadata management and data quality monitoring and remediation in the snowflake cloud data lake. We help in building the center of excellence in the data governance, which is managing the data catalog, scheduled, on demand, and in-flight data quality checks where snowflake, snow pipe, and stream are super beneficial to achieve in-flight quality checks. Then the data exception monitoring and reporting. Last but not the least, the time saver is persisting the non-compliant records for every data quality run within the snowflake cloud along with remediation script so that during any exceptions, the respective team members is not only alerted but also supplied with necessary scripts and tools to perform remediation right from the Ayotaho's active DQ. Very nice. Okay, guys, thanks for the demo. Great stuff. Now, if you want to learn more about the Ayotaho platform and how you can accelerate your adoption of snowflake, book some time with a data RPA expert. All you got to do is click on the demo icon on the right of your screen and set a meeting. We appreciate you attending this latest episode of the Ayotaho data automation series. Look, if you missed any of the content that's all available on demand, this is Dave Vellante for theCUBE. Thanks for watching.