 Hello and welcome. My name is Shannon Kemp and I'm the Chief Digital Manager of DataVercity. We'd like to thank you for joining the latest installment of the DataVercity Webinar Series, Data Insights and Analytics, brought to you in partnership with First-Hand Francisco partners. Today, Kelly O'Neill and John Lowly will discuss advanced analytics governance, effective model management, and data stewardship. Just a couple of points to get us started. Due to the large number of people that attend these sessions, you will be muted during the webinar. For questions, we'll be collecting them by the Q&A in the bottom right-hand corner of your screen or if you'd like to tweet, we encourage you to share highlights or questions by Twitter using hashtag DI-Analytics. As always, we will send a follow-up email within two business days containing links to the slides, the recording of this session, and additional information requested throughout the webinar. Now, let me introduce to you our speakers for today. John is a business technology thought leader and recognized enterprise information management authority. His 30 years of experience including planning, project management, implementation information systems, and improving IT functions. John writes and speaks on a variety of topics and enjoys sharing his expertise on strategic planning, data governance, and practical technology applications that solve business problems. Kelly O'Neill is the founder and CEO of First-Hand Francisco partners, an information management consulting firm. She is a veteran industry leader, speaker, and author, and trainer. Kelly is passionate about helping companies leverage the value of data, empowering them to derive insights that inform decision-making and improve results. And with that, I will turn it over to John and Kelly to get today's webinar started. Hello and welcome. Hi. Good morning. Good afternoon. Hello, hello. Well, thanks, everyone, for joining us today. We really appreciate it. John and I are going to share in the delivery of the slide deck today, and we're looking forward to getting all of your questions and being able to have some good conversation at the end. So our goal is always to leave the last 10 minutes for Q&A, and we're looking forward to that this time as well. So today we're here to talk about advanced analytics governance. So, you know, similar to data governance, there's some benefits and some challenges around governing those analytical models, output, resources, et cetera. So thank you, John. Today, what we'll cover, we will talk about some of the synergies and differences between analytics governance and data governance. I'm sure lots of you on the call today already have a data governance program. And so considering some of the things you're currently doing and applying it to analytics governance is a great way to get started. We'll talk about some best practices and ways to leverage those for different analytic use cases. And then we'll focus on ensuring that analytics governance is aligned with your critical business objectives. So this is something that you've probably heard John talk about quite a bit. This idea of a line of sight between your business objectives and your delivery model, whether that be analytics, data, or what have you. And then finally, we'll wrap up by talking about the balance that is necessary between risks and rewards. As more data is available to acquire and consume at very little effort and integrate into your environment, that comes with risk. And the opportunity to share data outside your environment can come with some reward yet also risk. So we've got some good stories to share, and I'm sure that you've seen a lot that's been in the media recently. So this is a great opportunity to talk a little bit about that. Okay, so let's go to the next slide. So hopefully this resonates with some of you on the call. So has it ever happened to you where you end up in a meeting with a bunch of other executives and you are sharing your numbers and another executive is sharing a similar perspective and for the same metric you have different results? That can be a challenge, embarrassing and costly. Have you ever experienced where you're looking for some information and the best place to find it is your colleague that you used to work with in another line of business, so you just call them up and they email you an Excel spreadsheet full of a bunch of data. Then what do you do with that? So that usage of the data is sometimes used appropriately, sometimes not because of course when it's shared cubicle to cubicle, it's not necessarily governed in the same way. And then of course you're trusting your colleague to be giving you the right data when in fact that data that you need may actually not be that data that was sent. So that leads to a revalidation process which impacts productivity, et cetera. So I'm sure you've experienced a bunch of these different challenges within your own workforce. And the concept of the Pareto principle of the 80-20 rule, there's a lot of different sources in the media that indicate that there's about 80% of the time spent finding, cleaning, reorganizing data and 20% of data scientists or an analyst time on in fact data analysis. So this is what we're going to talk about today is how do you reduce the Pareto principle to something that's a bit more manageable and more productive within your own environment. Analytics governance just to make sure we're all talking about the same thing. Again, very similar to data governance, it's an organizing framework for establishing and aligning the strategy policy and decision-making process around analytics with the goals and expectations that you want to accomplish from a business perspective. So very specifically, analytics governance is about that process that covers aggregated data, derived data, transformed data, including the algorithms and the models themselves and the, what's the word I'm looking for, and the spread of reports that can occur when there's not an understanding of reports that already exist or how to find and reuse reports. So that's quite a broad spectrum, but the idea is that the difference between data governance and analytics governance is the difference between data and analytics. So why don't we just keep going? And let's drill down into that a little bit more. The big difference between analytics governance and data governance is the content. So the content from a data governance perspective tends to be that raw data. A data element, a critical data element, data that is stored uniquely in different systems, whereas analytics governance is the aggregation or the combination of different data elements into something that is generally considered to be used for analysis. So that combination of different things could be the end result of that combination that is delivered to a user in the form of, say, a report, or it could be that process of derivation in the model. So because the content is different, the usage is also different. The usage is more at the end consumer, where it is the end result of a bunch of analytic processes versus the elements that are put together to create that end result. Many times the timing is also different because analytics is meant to be executed as a way to solve business problems and business questions can come up quite immediately and quite urgently. Whereas data governance, although there is a variability and a changeability around data governance, the data itself might not be changing quite as quickly as the demand arises for changing around analytics and your models and your algorithms might be invented quite quickly in order to address an immediate inquiry or demand from an executive. So the timing tends to be different and the requirement for agility tends to be different. Because the usage is different, the participants are also different. Now, not always, and we're going to talk a little bit about that in the next few slides, but recognize that there are going to be some differences in participants. So then if we're thinking about leveraging the constructs of data governance into analytics governance, there might be some folks that aren't so familiar with it. And then lastly, of course, the differences in technology. There are many capabilities from a technology perspective that are easily accessible and downloadable from an analytics perspective, which means that it would be important to consider the level of governance around that as well. John, is there anything that you wanted to add? Anything that I missed here, possibly? Oh, I think we're good. I'll have a few little things when we recap the section. I think we can move on. Okay, great. So let's just talk specifically about some of the roles that we see. And again, when we're thinking about an alignment process between data governance and analytics, that alignment fundamentally comes down to activities that are performed by people. So we want to actually be looking at the people. So this little kind of triangle here is meant to demonstrate some of the relationships and consistency of the types of roles that occur within data governance as a core capability, as well as other partner roles and then typical roles within analytics programs as well. So I'm actually going to start at the bottom to kind of discuss this slide. So core data governance roles, these are probably very typical and familiar to you, but the idea is that there's a concept of a business lead or sometimes called a data owner that provides guidance and is that key person for a data domain that is ensuring that the data is available and a level of quality to meet the business requirements for usage. They are supported by roles such as data stewards, data quality folks, technical data stewards, not just business data stewards, but technical data stewards. Other people that participate in IT like data architects. And then some organizations are also lucky to have business analysts and data analysts who are part of their core governance office, if you will. That core governance office, of course, works quite closely with other partners in the organization to ensure that they are getting the information they need from those subject matter experts. So they will have partners in the lines of business that are considered to be a data subject matter expert based on their usage of the data. Maybe they are doing analysis of that data so they're considered a data analyst or a data quality analyst. And again, we've got support from technology and IT and then those folks that maintain the data aspects and truly act as a librarian for the data. From an analytics perspective, you also have a business lead or an executive that guides the analytics program to ensure that it is addressing those business requirements and that it is aligned with the business expectations and isn't analytics for analytics sake. You also have someone that explores the data to make sure that you've got a person who is understanding that it's the appropriate data for appropriate use. They could be a data domain expert. They are also, many times, someone that will be creating reports. They will likely leverage data visualization tools. And then, of course, you have the data scientist who is the one developing some of those models and algorithms to do the statistical analysis. So they are the ones that are exploring the data that's managed by the analyst and what they're exploring is kind of the trends in the anomalies that you see within the data. And again, we've got support from the technology organization to make sure that this is automated as much as possible. If we think about this kind of triangle, it's highly possible that that business owner and that business manager, business SME or data owner could potentially be the same person within a line of business because they're most familiar with the data and they might be most familiar with what that line of business demands from the data. The analyst role could be that line of business data analyst as well, for they are both working within that data domain. They're working with the raw data to ensure that it is fit for purpose. That data scientist, even though they're doing certain modeling and they're working on the algorithms and that sort of thing, a data quality analyst might be doing a similar sort of modeling but much more basic from a quality perspective. So they're looking at how the models and thresholds for things like accuracy and consistency are created so that it can actually serve the data scientist. And then of course, IT has an opportunity to share some of this, I guess, economies of scale where there are understanding requirements from both the data perspective and IT perspective. So you can see how there's quite a bit of overlap or at least partnership in some of these roles where depending upon the size of the organization, they could in fact be the same individual or they could work very closely with each other so that there's an organizational synergy, not just a functional synergy. We also want to talk a little bit about the concept of these governance models and how you can leverage the models to align governance across either data or analytics. And fundamentally, there's centralized, decentralized, and then of course everything in between. And as you start to think about what sort of governance model works for you as an organization and how centralized versus how decentralized you want to be, these are just the two extremes and the conversation would be where do you want to land somewhere in between? So if you have a highly centralized model where you've got an analytics organization and a subsequent analytics governance organization that is very similar to an organizational chart, it's great for providing consistency of output because there's so much consistency in the actual individuals that are participating. You of course can leverage the same technology and you end up with a skill set economy of scale based on the representation of those individuals in more of a centralized way. Now because everybody is centralized into kind of a single sort of organizational unit, if you will, one of the things that happens over time is their experience and their subject matter expertise from a line of business perspective starts to wane. As a result, then their applicability to the enterprise starts to be reduced unless of course you very proactively recognize this and create alternatives for that. The other thing is sometimes with a centralized model depending upon the demand, that demand could exceed the resources available and so there's less agility available with a centralized model. Now of course a decentralized model is going to give you the exact opposite, right? So with a decentralized model where everybody stays within their line of business versus coming together in a kind of a centralized analytics team or analytics governance team and because they stay within their line of business, their business relevance is maintained and that means that there is a constant link between business demand and what that organization is doing. It may have a significant amount of agility and responsiveness as a result, but the challenge is if you're trying to get a highly networked organization to create some consistency of output, there's a lot of communication and negotiation that happens to make sure that everybody agrees on the same technology, the same look and feel in the output, the same reusability of resources is virtually non-existent. So again, most organizations don't end up at one extreme or the other, but where you're thinking about economies of scale versus agility, that's how you end up somewhere in between. And we've got a couple of examples of this coming up. So here's an example of a model in which the centralization between data and analytics occurred via a forum that was made up of different executives that came together on a regular basis and through their conversations, their planning and their execution, they were able to create that consistency while being a decentralized unit. Supporting that data and analytics leadership forum, there was a separate forum that was really focused on data stewardship and data governance, and then another separate forum that was focused on the actual analytics process and output. The two forums, the data stewardship forum and the analytics forum, they primarily focused on their own spheres or their own community, if you will, although there was some activity that occurred to unify those forums as well. Primarily that unification happened at the leadership forum. So this is a highly decentralized model where there's a recognition of the requirement to create consistency, and this organization was able to do that via a very involved leadership forum to create the consistency of planning and the consistency of execution. Okay. This is a build slide, so I'm going to talk through it just a little bit. And so this is another example, but this has a slightly more centralized model where there was a head of business analytics that ran the analytics organization for this company. And that head of business analytics had a group of folks that actually ran analysis on behalf of certain functional units within an organization. So in this instance, you could see that they've got market analytics, credit analytics, client analytics, et cetera. And they had those data scientists and those data analysts who were exploring the data and creating the models and algorithms to support those groups that were doing the analytics in a specific category. Now, they did consider that they needed to reach out to those line of business units that were doing some more of the operational reporting to ensure that there was a level of consistency, and that's what they called kind of the analytics working group. This head of analytics reported up through the CFO, and there was the recognition that they need to have a little bit more cross-functional involvement from an executive perspective. So they leveraged the presence of the Enterprise Infrastructure Committee to get feedback to ensure that the analytics direction was serving the full enterprise versus just the finance organization. So this is what the analytics group looked like. Again, a little bit more centralized from an organizational perspective. So if we... Next slide, please. If we compare that to what the data governance organization looked like, it was a little bit similar where there was a centralized data governance office that had capabilities within that office to manage the data governance program itself. And then the data governance office had a bit of a decentralized model where it leveraged business steward leads that maintained their role within the line of business, had data stewards within the line of business as well, and an IT support group that worked all as a working group and a kind of decentralized body that was unified by the data governance office. In this way, data governance was a bit more decentralized than the analytics organization. The data governance ultimately reported up through the chief operating officer, but in another way, in kind of a similar way as what we saw on the analytics side, that leadership forum that provided the relevance and the guidance to the data governance office was the same leadership forum, the Enterprise Infrastructure Committee, that was being leveraged by the analytics group. And if we consider what was occurring at that kind of more execution-oriented level of this operating model, the reality was that some of those business steward leads were actually those same people that were on the analytics side because of their usage of the data in the analytics practice. It made them good representatives to be a business steward for that line of business. Some of those data stewards were also people that were acting as data analysts on the line of business side. And there was even some crossover in terms of the people that ran the operational reports because, of course, we want to have some data stewards and business stewards that have an operational perspective, not just an analytics perspective. So you can see that in an organization, this organization was able to leverage some of the capabilities across these different models with a varying level of centralization and decentralization because when it came down to it, the people and the individuals themselves were actually the same. John, did you want to add anything to these discussion points or this section? Just to recap a couple of things. One thing that leaps out to me is, and I think some other people will be hearing this, is that a lot of this sounds familiar to perhaps something that is just said about BI reporting versus analytics. And again, the core difference is content, but don't think that the content difference is going to take away some basic blocks being and tackling that you have in these operating frameworks. You can't base your operating framework or the way you operate this on a particular type of architecture. Classic corporate information factory type thing or modern dupe-centric type thing. You have to base it on what works for your organization. It's not an either or one of the different or that's old or that's new. I think that's the takeaway from all of this. Should you be decentralized? Should you be decentralized? I don't know. It depends on what you require. Remember, the expectations are very, very high for these types of analytic environments. And your operating framework has to work and communicate very, very well. So spend the time to do it. Just don't throw some boxes up on the board and say, yeah, that'll kind of work. Put some conscious effort on this type of, this aspect of your analytics environment. That's all I had. You ready to move on to the next section then, Kelly? Sure, absolutely. We'll talk about best practices. And I believe I'm talking about that for a little while. Sure, go ahead. Well, I think that was the plan. Feel free. Okay. No, that wasn't the plan. Actually, no, I had something else. Go ahead, Kelly. I got something else to go ahead and do that. Not a problem. And of course, to those of you that haven't, that this is the first time dialing in, John and I do go back and forth quite a bit to make sure that we have this shared experience that we can discuss with you guys because we each have different backgrounds and we bring different things to the table. So generally it covers all bases. So since we just talked about some of these models to kind of align data governance and analytics governance, there's also other aspects, not just organizational aspects, that you can leverage across both analytics governance and data governance. And so data governance can help facilitate analytics governance, but analytics governance can also help facilitate and provide direction for data governance as well to make sure that the way that data governance is operating is providing value to the analytics organization as well. And so whereas one provides a focus on data as a foundational asset so that it can be used across both operational and analytics aspects of the business, then the analytics governance helps to ensure that that voice is heard within the data governance forums as well. Now data governance traditionally defines data standards to ensure data consistency, whereas analytics governance would be defining analytics standards, model standards, algorithm standards, and in order to ensure consistency and appropriate use, they will think about the model management process, for example, and the model development lifecycle in a very similar way that a governance organization is going to think about the data lifecycle and how data may be created or acquired, brought into the organization, protected, managed, maintained, archived, et cetera. The same sort of process would come up around analytics as a process and the actual algorithms and models themselves to ensure that there is reusability of those models as well as maintenance of those models to ensure that as things change both in the business and from a regulatory perspective, that those models adapt and are changed appropriately as well. So there's a lot of ways in which governance can facilitate analytics and then analytics can help guide and ensure that governance maintains relevance within the organization. Next slide, please. Some of you have seen this next slide previously. So this is the First San Francisco data governance framework, but the idea isn't necessarily to talk about it from that perspective. It's more that when you're considering data governance, that you think about these primary categories and how they're relevant to your organization. So thinking about the strategy and how the strategy aligns to your organization, what that actual operating model and organization looks like. And the idea is if you break each of these components down, it's a similar sort of question or consideration when you're setting up your analytics practice. For example, how do you make sure that your analytics and analytics governance practice maintains relevance to the organization and adapts and changes in the same way that your regulatory environment may adapt and change or your requirements may adapt and change? The concept of directive, so policies and rules around how algorithms are created, shared, and maintained and how you create traceability and how those algorithms are created, maintained, what data it uses, who uses those, why do they use them, are they authorized to use them, etc. So it's very similar to data lineage as well. You want to make sure that you are measuring the output of an analytics program in the same way that you would have proactive measurement around governance. And then a couple of other things to point out. I mean, obviously there's a difference in the way the technology supports it, but activities associated with communication and change management are highly leverageable and reusable across an analytics governance practice. And the way that you go about doing a change readiness assessment, for example, would be virtually the same regardless of whether your content is data or your content is analytics or models or algorithms or what have you. So these sorts of best practices can be repurposed and reused and the success that you've had on a data side will help to accelerate your analytics program as well. So the next slide talks about a few design principles and in the interest of time we're not going to drain this slide. As usual, you will all receive a copy of this. But a few things that I just want to highlight here because they are really important and they do cover analytics and data governance from a best practices perspective. Ensuring that you have a clear goal and that clear goal is aligned with business objectives. If you do, chances are you will also be considering the enterprise even while you're thinking about lines of business requirements because, of course, there's going to be specific requirements around analytics that will be different for the finance group, for example, from the HR organization. You're going to be working with different models. You're going to be... There will be different data scientists, what have you. But they might also be using some of the same data and so there might be things that can be leveraged across the different lines of business. So you want to be thinking about the enterprise even if the implementation or the exercise is within a line of business. So there's some other kind of design principles here in terms of thinking about flexibility, simplicity, usability. And the idea is that the demands on the organization is going to change and so you need to be available... You need to be ready to change with the demands of the organization. A few slides ago we talked about one of the big differences between data governance and analytics governance is the timing and the requirement for agility because the questions that the executives will ask may change quite regularly. So that will be important to ensure that you're able to respond quickly from an analytics governance perspective as those requests and demands may change. And then finally, the concept of communication as being a huge validator and as an important component to ensure that the rest of the organization understands the good work that you're doing is incredibly important and should be considered as a proactive, funded, and resourced exercise. Communication is time-consuming, especially when it's done well and that does definitely need to be considered. I think now we're moving on to... Oh, well, did you have anything you wanted to add to this section, John? Again, by way of summary, there's kind of a theme with this section. And some folks might be saying, well, again, there's a lot here that isn't explicitly exclusive to analytics. And I think that's a good thing because it's not that there should be... It's not that we're applying even lessons learned from one or two generations before now this current generation of using data. It's more that there are essential principles and capabilities in running any effective organization. And don't overlook those. Don't overlook some solid principles to do that. Don't necessarily... Certainly, there will be specialization, but the whole theme here today is kind of a Pareto principle. There's a 1% of activities that are going to get you 80% of the way there. And we just highlighted a few of those there. And I do believe the next section... Now, I've finally got my notes in order. Kelly, I think I've got the next section here as lead. Sure, keep going. On that. And that's the analytics and BI strategy kind of blending them together and having an alignment. Again, this is... And the reason I take this section is because this is a message that we continue to hammer on everywhere we go or work or speak. And that is that if you're not doing something explicitly for the business, you can't measure... You don't know for first and foremost whether you're making any progress or not. And even if you are making progress, you can't prove it to anybody. And since a lot of organizations have a bit of change and cultural stress when they move into highly governed or highly exposed data-driven type capabilities, there's always pressure to go back to where you were. And if you want to be sustainable, you have to show some alignment. So we talk about a super high value exercise called line of sight. And that makes sure that what we're doing is... And what we're focusing on is what we're doing for the business. A great many folks will say, well, we want to build models that answer questions that we've never asked and all those wonderful things. And that becomes more or less the objective, but that's not the objective. The objective is, of course, to do something for the business. So the kind of the process we go here through is, first of all, capture what the business is about. What does business value mean? So all businesses have strategies. They have major initiatives. All organizations. We will talk about government or nonprofit. You've got major directions you're going. Capture that. Where are you going to improve value and fulfill the mission of your organization? Those have to be measurable, right? Once they're measurable, and I have an objective that has a measurement on it, I now have a data requirement. I am now in the realm of, now, how do I meet that data requirement? And if I meet it with analytics or big data or whatever, fine, or I'll put this simple BI reporting, fine. I need to take the entire population of those needs. And it could be an analytical model. It could be operational BI. It could be a traditional managerial monthly report. But you do need to build kind of a meta layer of this or a way to understand what your broad needs are. And then each one of those requirements, as it were, for using your data. Let's just say we have an analytical model that is going to predict medical outcomes for a certain population. And I'm just saying that because I've done a lot of healthcare lately. It's kind of in front of fine. So I'm going to build some sophisticated models. Well, those models have to be managed. They have to be tracked. I have to assure the consumer of the results that the results are accurate. Well, that is data governance and data management. So now I need to say, what am I going to do to assure quality, assure timeliness, assure trust in that information? And of course, there's also some ancillary things with that, which we learn about in analytics or anything else. You know what, data quality is important. There's some dimensions that I want to use, or I want to group the data by certain populations, and that reference data is deficient. So the reference data and data quality, all these things come to bear, but those are all data management components. So you want to decompose from what your business needs to what your data governance and data management needs. Here's a simple example. And again, since healthcare is front of mind to me here recently, we threw one in there. And that is the initiative of the strategy, the organization is to improve outcomes and wellness through a predictive model of some sort. But that's tied to a particular financial goal, which is reducing readmissions by a certain percentage. There's a bunch of metrics. We're not going to empty this slide out. I like the way Kelly refers to that in great detail. But you can read this if there's a bunch of stuff that we want to have our model support and measure. So if I have a model and it tells me to do something and readmission should go down, I've got to also measure the readmission. That's something else a lot of folks overlook is that that model is going to drive business activity and we need to measure that business activity. So then I have some use cases or some way, I'm going to leverage my data and we have a lovely big long list of things that are going to happen there. And I can look at all of those, I can say data management wise, I need to have some reporting, I need a registry, I have some domains that I need to pull data from. And from a data governance standpoint, I need to assure data quality is correct. I need to have my semantics of the data correct. I need to manage my models. And I'm going to have to require a certain amount of data lineage and or a certain amount of data provenance or provenance or however you say that. I keep referring to an area of France and it's not an area of France. Anyway, this is a simple decomposition. Now, here's where it's helped you. You get challenged by someone to say, oh, we don't need to get out of this. We just need results. By gosh, we need results. We tried this data management stuff 10 years ago and it was no good. But you can say, okay, call it what you want. Well, we won't call it data governance anymore. But we need to do these kinds of things. Do you think that's smart? Like, oh, yeah, well, of course, that makes sense. Just don't do data governance. Fine. Proceed as requested. All right. But you've done the alignment exercise regardless. This also, everyone wants to advance their data maturity to be competitive or efficient or stand out in your particular industry or segments of government or market. So, and we all know that some of us started some spots and we want to go to somewhere else. Any formal alignment exercise will help advance your maturity. So you don't need to be at a certain maturity to do this exercise if you have nothing. All right. And you do the alignment. You're going to find yourself leapfrogging to a level where you can see commonalities of bubbling out and economies of scale and things like that. If you're starting perhaps a little more advanced, we already have a level two on the second part where you've got some policies and some governance and along comes analytics. And now you're going to do these models. You'll discover that you're going to have a really good driver for enterprise level policy enterprise level of data. You'll have further support of probably something you've been striving for a while anyway. So there's a wonderful opportunity to advance your maturity. Kelly, anything on that section here? And then we'll let you continue on here. I think you covered it all. All righty. Onto risk and rewards and the next slide. It's you. It's me? Oh, my goodness. I thought I went back to you. Anyway. I've talked enough. All right. All right. I'm, but everyone here, the paper, John's going over to his next page. You know, my gosh, it is me. How about that? There's a lot of risks associated with this. Another reason to manage your models really, really tight. And it's really, we have a really cool example here. It's come out here in the last few weeks. And I think if unless you've been in a cave, you've heard about the Facebook problems. And this is a classic example of, of data modeling and data analytics run amok. And it's, I don't know the market cap of Facebook was down some 40 billion dollars or something like that. Some amazing amount of money. And if you don't think there's a cost associated with no governance, now you know differently. There are obviously ethical aspects to using your data. And the more powerful the model, the more you need to consider the ethics of data usage. You need to consider that there, that your constituents in this modeling process are have to be considered even if they are the subject of some type of data driven action. And of course, consider the tone of privacy. American privacy laws are looser than Europe. But after this, you might see some tightening. So consider the entire context of the privacy world. As you're, as you're going through your modeling exercise, a real significant capability within the management of analytical models is an ethical capability and a risk management capability. And if there's anything to add to that Kelly or I'll just going to move on here. I think in the interest of time, let's just keep going. All right. Okay. The other kind of a balancing force here is the fact that when you start to work with analytical models and with data so much, when projects become data centric versus becoming process centric, your methodologies are going to change. Even if you have an SDLC or your agile, whatever, you're going to find that a lot of those processes don't fit. You feel like I've had someone describe this to me as a round peg in the score hole trying to use corporate PMO and a corporate SDLC and corporate agile work and then doing these heavy data centric model type analytical projects. It just doesn't feel right. And that's because they don't fit. So about again dumping this whole slide out, this is something we've shared before and we're happy to share with folks again and again. There is a different mindset or a different process which we call data centric. But the takeaway here is think about that your go-to market process is going to be a little bit different with anything that's model driven. And then lastly, we couldn't do this any better than Gartner did it. And so we brought this in here. And that is depending on the different type of platform, you might have some different drivers, some different characteristics. You might have some different governance objectives with them. But at the end of the day, those capabilities we've talked about are going to be very, very consistent. All right. So if I take, for example, a standard information portal versus a data science lab, we'll take one side or the other. I'm looking at efficiency and integrity and quality while looking at that portal. The predictive model is looking at some type of viability of that model, the transparency of the data. And I have to worry about the privacy, the legitimacy of what I'm doing. But the capabilities underneath those visible objectives are very, very similar. So if we go back to our prior sections, you can see that we've got a bunch of core things that have to be done. A basic alignment has to be done. A basic balancing of risk and rewards with your using your data on that. So let's move on into our summary then. And we'll have time for questions. We do have a few questions here, which, and we'll take a minute here, that if you do have a question, we do allow time now to answer them as much as we can. And if we can't get to them all, we'll get to them some ways somehow, but we'll get your answer to you. So please don't be shy and enter a question here. It's not like YouTube, you don't have to click like below and or anything invasive like that because we've considered our data risks and rewards with this exercise. We just want to help you get your questions answered. So feel free to put one in. And while you're doing that and now we get prepared to answer your questions, let's just do some takeaways and Kelly, let's just kind of toss these back and forth here. Sure. All right. What business attempts to do an analysis directly influences what you do with your data? Form follows function, right? Absolutely. Yeah. The analytics requirement is a key driver for what you are doing from a governance and process around your data, most definitely. And then we talked about existing constructs, existing ways we've done things are perfectly appropriate for figuring out what you're going to do with environments too. Absolutely. There's this, you know, the don't reinvent the wheel idea. And yes, the content's different. Yes, your participants might be different. They might be the same, et cetera. But recognize that the ability to implement an analytics governance program will be significantly faster if it doesn't feel foreign to the organization. And so leveraging what you've done from a data governance perspective and what's worked will give you a sense of what you're doing and what's done in the way that you're going to do. And so that's why Gephardt will allow you to accelerate your implementation. We talked about risk and rewards and balancing by other factors other than regulatory compliance. And of course we use that example of Facebook but there's many others out there that, you know, of balancing. The more sophisticated we get with our data, the more we're going to have to do things about how we use our data that we've never thought of before. And these analytic models, you are now on the forefront in your organization of that type of thought process. That's absolutely right. And although a lot of the regulations focus on the data aspect, there are already regulations that are ensuring that the models that are created are not discriminatory. And so there will be a requirement to create, to have transparency, validity and be able to expose what those models are easily in order to demonstrate compliance. And that's just going to get more and more pervasive because the more that companies are basing decisions on analytical models and that sort of thing. And the risk of those models being discriminatory or something like that becomes higher and higher. So it isn't just data regulation. It is algorithm and model regulation that is coming. Absolutely. Absolutely. And we hammered again on our alignment and form following function and alignment makes things a whole lot easier in the long term. So let's just take time for the questions. I'll read through our questions here and we'll go from there. Wow, we have a bunch of questions. Let's just start near the top. What is your view on a bottom-up approach versus a top-down approach if you're decentralized? Sure. I'll take a first swag at that one. Yeah, so decentralized is fundamentally a bottom-up model. So it's not decentralized and being top-down. In fact, I don't think I've ever seen that exist. And many times, all sorts of programs are started bottom-up because it's the people that are on the front line, the people that are delivering the value either to the client or internally to an internal client. They're the ones that see the pain in the first place. And so I would say that most programs start as grassroots programs. And the trick is, is going from grassroots to more repeatable and sustainable models and kind of crossing that chasm, if you will, to ensure that you get some top-down mandate to authorize and empower you to spend the time actually doing that work. The challenge in maintaining a decentralized model is that most times in a decentralized model, you don't have that function in your job description. And guess what? As resources get tight, your management team is going to reference back to your job description. And so might your compensation. So that's where the top-down needs to come in place, is to give that mandate and that authorize it. Hello? Sorry. I don't know what happened there anyway. Okay. Hopefully you were hearing me. I don't know if you wanted to add anything to that answer. We have a bunch here. So we're just going to charge ahead with them. I'll take this one, because I have a kind of a quick answer for it. Can you please address how the activities of data scientists who after all are on a voyage of discovery with no a priori knowledge about what they're looking for should be governed? Well, again, let's kind of go back to the theme here. Maybe we could have called this thing, there's nothing new under the sun thing today too. The good answer is why wouldn't they be governed? Your organization, they're going to do something that could put your organization at risk or could do really, really wonderful things. There needs to be an expectation at the beginning of what the boundaries are. There needs to be a consideration of risks, whether they be compliance or ethical or whatever like that. And there needs to be a consideration of what the expectations are. This question was answered, it's time stamp, it was answered before that Gartner group thing. We will still reset. Gartner group kind of addresses that as well. Next question. Let's see here. When data stewardship is assigned at the source or a transactional data level, is it common for a data steward within the operational data warehouse and data march to mirror and always trace back to the source? I would think that that's basically lineage is kind of a required capability these days. And if it's not done manually by the steward, we want to automate that. Let's see here. Kelly, one for you. Does the Gartner slide recommend to apply different levels of governance based on how the data is used? For example, exploration versus a decision model or corporate support. And we'll go back to that. There's one other question to go back and put slide 24 up. So we'll kill two questions right there. So why don't you take that one, Kelly? I think the answer is kind of sort of yes, but why don't you explore that a little bit more? Yeah, I would say it's an absolute yes. And again, form follows function. So it depends on what the demands are. So if you are looking at and we'll take the two extremes again, if you're looking for as an information portal, then there's a lot more demand for quality of information that is pulled out of that portal because there is the requirement for to adhere to the corporate standards. So for example, and so there is a need to ensure that the data that is pulled out of that in order to deliver on analytics does comply with any sort of corporate standards. So it is much more governed, if you will, versus the data science lab. The focus here is I would actually like focus on that bottom bullet point, calling it legitimacy. And from a data science perspective, the idea is not to limit what happens from a data science perspective, but to ensure that the activity is legitimate. Is it legitimate based on the requirements of the company and the culture of the company? Is it legitimate based on regulatory requirements? Or are these people exploring in such a way that it's not relevant to the goals of the organization and therefore the value isn't relevant, right? So there needs to be some level of guardrails around what those people are doing from a data science perspective. And so although they are being, they are experimental, it does need to be within the bounds of what's considered acceptable use within the organization. So yes. And so there might not be a care around levels of quality from that perspective, right? So again, the levels of governance and rigor would differ depending upon that use case. Yep. You know, the key there though, it's different flavors or different levels or intensities of governance. But there's still the essential components and capabilities within data governance that are brought to bear with maybe some different visible results. But what we see often, right, Kelly, is that people say, well, we're different. We don't need governance. And that, that is not correct. It's just it might look differently to an outsider. But yeah, the question is, what is the governance that you need, right? Because there is risk associated with zero. Lots, lots of risk. I think we have time for another one here. Do you have any recommendations in terms of assigning accountability and in prior and aesthetically ownership, assuming you recommend assigning accountability for analytical data and reports? And that's one that's a topic Kelly and I both like. I'll give Kelly the first shot at that for a minute and then I'll take Yeah. And what I would want to highlight is that kind of clause that comes after the question is that it's about accountability. And the challenge with ownership is that people tend to think about ownership as being, you know, by one individual. So accountability, especially as you get to a more distributed, either distributed model, or you get to greater amounts of experimentation and the type of data that you're using is more variable and things like that. There needs to be inherent accountability by that consumer of data. And so what they are doing in their innovative experimental lab, they need to be accountable for the data, the algorithms and the models that they are creating. So therefore they need to understand those guardrails that are available. So I think that's what what I would focus on is that concept of accountability as opposed to the concept of ownership. Ownership may work from a data perspective in the sense that someone really needs to be drilling down into quality thresholds and that sort of thing from a from a ownership perspective that might not necessarily apply quite as stringently depending upon your analytics use case. Yeah. Yeah. My short version of that is there's two concepts that in my last 20 years now of working on governance have caused immense headaches. And that is the term ownership and the term data storage. There's a tendency to just grab those right away. And you immediately get if there's a hundred people in a room, there is a hundred different pictures in the front of their minds as to what that means. But the term accountability or responsibility or control or informing like a bold fashion racy chart, those are pretty discreet. And those are much more important words to use than a steward or an owner or something like that. Now, let's see here. We've got one minute left. I mean, just I think we've got the big ones here. There is a question here. What are the major activities to initiate data governance of an organization? Pretty broad questionnaire. But our little proun slide earlier in the presentation is a really good start to take a look and kind of take a look at that and go, wow, well, this is kind of all the little various shades and shapes and forms data governance can appear in and what needs to be done to get that up and running. And then, you know, send Shannon and voter something. We've got other papers and resources and presentations we've explained that in. At that point, I think I just saw the clock tick to the top of the hour. Anything to do for a wrap up here? No, I think that I think that you're absolutely right. If I were to coach someone in terms of starting data governance, I would ask yourself, why? Why are you doing this? And then I would ask yourself, how are we going to make decisions? And those two big questions are the best place to start. Yep, all righty. Back to you, Shannon. Thank you both for another fantastic presentation. And thanks to our attendees for being so engaged in everything we do. I just love all the questions. Just a reminder, I will send a follow-up email to everybody by end of day Monday with links to the slides and links to the recording of the session, so you'll get all those wonderful charts and such and that. And I hope everyone has a great day. Thanks so much. Thanks, everyone. Thanks, all. Bye.