 Hello and welcome. My name is Shannon Kemp and I'm the Executive Editor of Data Diversity. We would like to thank you for joining the current installment of the Monthly Data Diversity Webinar Series, Real World Data Governance with Bob Piner. Today, Bob will be discussing Agile data governance, how to apply Agile governance to, or excuse me, how to apply governance to Agile. Just a couple of points to get us started. Due to the large number of people that attend these sessions, you will be muted during the webinar. For questions, we'll be collecting them by the Q&A in the bottom right hand corner of your screen. Or, if you'd like to tweet, we encourage you to share highlights or questions by Twitter using hashtag RWDG, Real World Data Governance. As always, we will send a follow-up email within two business days, containing links to the slides, the recording of the sessions, and additional information requested throughout the webinar. Now, let me introduce to you our speaker for today, Bob Piner. Bob is the President and Principal of KIK Consulting and Educational Services and is the Publisher of the Data Administration newsletter, TDAM.com. Bob has been a recipient of the Dean of the Professional Award for significant and demonstrable contributions to the data management industry. And Bob specializes in non-invasive data governance, data stewardship, and metadata management solutions. With that, I will give the floor to Bob to get today's webinar started. Hello and welcome. Thank you very much. And thank you, everybody. I appreciate you're coming back to another Real World Data Governance webinar. As you may know, this is one of my favorite subjects to talk about is relating data governance and Agile methodologies. So this webinar is called Agile Data Governance. We are going to talk about how to apply governance to Agile. I think in a previous webinar, we spent a little bit of time talking about how to apply Agile to governance, but now we're going to talk about how can we apply governance to Agile. So good to have you with us this afternoon. I hope everything is going well where you are. Just as a quick reminder that the webinar, the Real World Data Governance webinar set for next month is going to be about data governance and the Internet of Things. So if you're interested in the Internet of Things, and I think a lot of people are talking about that subject these days, you want to talk about the data that's related to the Internet of Things and the levels of governance that might be required to really, truly govern the data that's going to be passed back and forth through devices. So kind of a D to D device to device type exchange of data. We're going to talk next month about data governance associated with that type of data. A couple of other quick reminders. If you haven't been out there yet, please take a few moments and visit the new version of the Data Administration newsletter. Newly refreshed, the November issue is available. Please go online and let us know what you think about the publication. And that is also, that's produced by DataVersity, and I hope you'll find a lot of interesting content there. Also a reminder, the KIK consulting website has been updated. Take a look at that if you're interested in learning more about non-invasive data governance. Real quickly, the book that I talk often about or pull things from is called Non-Invasive Data Governance. It was published back in September of last year. And a quick reminder about two upcoming DataVersity events as well. One will be taking place in a couple of weeks in Fort Lauderdale. That's the annual Data Governance Winter Conference. And then DataVersity is trying something new in January. You might want to go and check it out. It's their Enterprise Data Governance Online Conference. I believe it's their first attempt at doing a virtual event. So there are going to be some interesting speakers, interesting topics. I will be hosting a panel at that event. So I look forward to virtually seeing you there. That's going to be fun. I think that's going to be interesting for a lot of people. I want to talk a little bit about what we're going to talk about today. The abstract that I shared with you regarding this installment of the webinar. Talk about the relationship between agile and data governance. So agile development efforts in organizations and data governance efforts in organizations are oftentimes at odds with each other. I know that firsthand from working with a lot of organizations that have huge agile efforts underway and yet they're focusing on putting governance programs in place as well. Even though both of these different types of disciplines have the sponsorship that's required at the top levels of the organization, there seems to be some level of disconnect between can we do agile and do data governance at the same way? Are there ways to apply governance to an agile methodology? So that's what we're going to talk about today. Supporters of both disciplines say that this is really the most important thing. The data people are saying we need to focus on the data. Agile people are saying we need to deliver our projects quickly, effectively, in high quality. Organizations want to see things done fast, but they also want to pay attention to the data and make sure that the data is accurate. So we're going to talk today about applying governance to agile methodology in some ways that we might be able to do that. The specific things that we're going to focus on in this webinar are relating the disciplines together for senior leadership. Oftentimes there seems to be a disconnect at that level of the organization that they want things to be quick, but they also want us to protect the data. They want to make sure that business rules are followed by the data. So we need to help our leadership to understand the relationship between these two disciplines and why it's important to do both and whether there's some levels of agreement that need to take place in order for them to both work together effectively. We're going to talk about finding common ground between these two disciplines, applying governance to agile efforts. I'll share with you quickly some best practices around agile data governance, and then we're going to talk quickly about gaining support of the data people and gaining support of the agile people for implementing agile data governance. Quickly, the definitions that I want to share, and I share as I typically get started in these webinars, is these are my definitions of data governance and data stewardship. So my definition is not the only definition that's out there. There's a lot of definitions. I like to put some teeth behind the definition I use. That's why I say that data governance is the execution and enforcement of authority. A lot of organizations will say it's about orchestration and harmonization of people in process and data. I say that's true, but we also need to execute and enforce authority through governance. That is actually the primary purpose of putting a data governance program into place. Stewardship, as I've spoken about in recent webinars as part of this series, I look at data stewardship as being the formalization of accountability. So in my opinion, virtually everybody in the organization could be considered a data steward if they have a relationship to the data. If they define or they produce or they use data as part of their job, they have an accountability for how they define, produce and use that data. So again, everybody is a data steward. There are different levels of stewards. We're not going to talk about that today, but just to kind of make it clear that stewardship has different meanings to different people in different organizations as well. So we want to make sure that the way that we govern the data is really through the effect of people's behavior, and those people are the stewards of the organization. Noninvasive data governance is the practice of applying that formal accountability through a noninvasive set of roles and responsibilities, applying governance to existing processes, rather than calling everything governance processes and redefining your processes. And a noninvasive really describes how governance is applied to make sure that you really do non-threatening management of the most valuable of data assets that you have in the organization. The goal then being to be transparent, supportive, and collaborative wherever we can. And it's funny that the word collaborative is in there because oftentimes that's one of the things that people who are focused on delivering things in an agile way say, we need to be collaborative. We need to get people to interact. We need to deliver things quickly. So on this slide, I have a picture of kind of a typical agile development methodology where we're talking about developing this methodology leading to good design through iterative pieces of the project. So we're going to talk about engaging people, interacting with people, being very speedy in how we deliver things, making sure that we're delivering solid work products and such. That's kind of how the definition of agile is applied in organizations. Organizations want to be quick, but they also want to make certain that their data is accurate. So let's contrast the agile approach to the waterfall approach for developing software products where the waterfall approach is typically recognized by people who follow the agile approach as being too slow and cumbersome. Where the people that are focusing on a staged set of deliverables during a waterfall delivered project that we make sure that we analyze the heck out of the project, that we gain thorough business requirements and data requirements, and that we follow basically the steps that you see on the screen in front of you to deliver projects. And so a lot of people in the agile world think that the waterfall methodology is a thing of the past. Now the truth is that even though for years individuals and organizations have been following the waterfall approach, the truth is that we really haven't effectively followed the data management aspect of the waterfall approach. So by addressing the agile approach to delivering software products, we can also kind of learn some of the same rules from that and apply those to, if you do follow the waterfall methodology, make sure that certain data aspects of the projects are being governed well. So the question really becomes what does senior leadership really want to see from data governance? And what I've done is I've narrowed it down to really three primary things. Now senior leadership, at least the senior leadership that I've talked to, they want to make sure that their data is well understood across the organization. So one of the things that they want to do with governance is they want to govern the definition of the data. You know, another thing that they want to do is they want to govern the quality of the data. They want to make sure that as we're producing the data, that people understand the impact of how they're entering data into the systems and that we really govern the way that data is being produced across the organization. And last but not least in this series of these three things is that organizations want to make sure that they're protecting their data. And I found that the majority of the organizations that I've talked to recently, and that becomes a primary requirement for their governance efforts, is to make certain that people understand the rules associated with protecting data in their organization, whether it's PII, personally identifiable information, or PHI, personal health information, or whether it's even business intelligence information in the organization. Organizations are not being given the option as to whether or not they want to comply to the rules that are being set upon them. They want to make certain that they are following the rules and that people in the organization understand that the protection of the data is necessary. So those are the typical three reasons why you see organizations want to implement governance programs. So let's look at that from an agile perspective as well. So what is senior leadership looking for when we talk about delivering things in an agile way? You know, they're looking for basically three primary things. One is speedy delivery, and one is incremental delivery, and they want to make sure that their projects are delivered in a high-quality manner. I don't know how many of you know who that person is on the bottom right of the screen there, Mr. McFeely, and he's known for saying speedy delivery, and that's certainly what organizations are looking for when they are putting agile projects into mode as they're starting to deliver things in an agile way. So what does senior leadership think about data governance? You know, one of the reasons, we talked quickly about the reasons why their organizations are putting governance programs in place. They want to manage their data as an asset of the organization. They know that the data needs to be protected. They know that having high-quality data drives the analytics and the decision-making in the organization. You know, one of the things that I've heard recently is that auditors are telling organizations that they have to have formal governance in place, and organizations are looking at other organizations and saying that they're getting more value out of their data. We need to do the same thing. So there are specific reasons why senior leadership think that data governance is important and reasons why organizations are putting governance in place, but there are also reasons why these organizations are not putting data governance into place. You know, they don't understand that there's multiple approaches to how you put your governance program in place. There's a non-invasive approach. There's an invasive approach. There's a command-and-control approach. There's a software-driven approach. There are multiple approaches to how governance is put into place. Most organizations think that it has to be about command and control. So one of the things that we can do is help senior leadership to understand that there are multiple approaches available to implementing governance in their organizations. Senior leadership think that data governance is going to be very complex, and that depending on the way that your organization is set up, it may take a very long time to implement governance across the organization. Those are other reasons why organizations aren't implementing governance. They think it's going to be expensive. They think that it's going to require a lot of resources. Data practitioners is up to us to really share with our senior leadership what options they have available for implementing governance programs. So there are reasons why organizations put governance programs in place, and then from my experience, there are reasons why organizations are not doing it. People are fearful of what data governance is and what data governance will mean to their organization. So why are organizations using the agile methodology now instead of using the waterfall methodology for delivering projects? Well, that's because large projects tend to take too long, and they cost way too much money. So organizations want to be able to deliver projects quicker, and they want to deliver them more effectively. They want to make sure that they're delivered quickly and accurately. Projects need to be delivered continuously, and they require that they have face-to-face contact with people and make sure that things are being delivered in such a way that are going to be quick and effective in their organization. So it's a question as to why organizations aren't implementing governance, or aren't implementing agile. Well, the truth is that they are implementing agile. And typically, if you look at it, the projects that follow the agile delivery method are not small projects in your organization. Most often, their ERP implementations or package implementations, their delivery of new systems, of new BI environments. Organizations, when they decide to take the agile approach, do not typically take small projects and run them through the agile methodology. So it's very important that when we deliver these large projects quickly and effectively, that we make sure that we take a look at how we can apply governance to those initiatives to make certain that we're not delivering things that the organization doesn't need, that we're following business requirements, data requirements associated with those major pieces or major efforts within the organization. So one of the things that I suggest to organizations is that we have conversations with our senior leadership to help them to understand that there is a relationship between agile and between data governance so when I say what does senior leadership know about relating agile and data governance, well the truth is those conversations really haven't been held in a lot of organizations so they know very little about it. You know, how much effort has been made in those organizations to demonstrate to their senior leadership that there is a relationship between agile and data governance, very little has been done on that as well. Whatever has been made to align governance, the governance program with different agile efforts, well in a lot of organizations it's kind of funny, people joke with me about the fact that you don't see the agile development teams and the data people sitting around a table, a lunch table in the lunch room. They typically don't talk to each other very often. There typically hasn't been an effort in an organization to come to any level of agreement as to data governance requirements that need to be addressed by the agile efforts. And you'll notice here in a couple of minutes as I continue through the slides that's one of the things that I suggest to organizations is that they bring the agile teams and the data teams together and they come to some level of agreement on what types of data requirements are going to need to be obtained or gained through the agile efforts. So we'll talk about that more in a couple of minutes here but typically data governance programs don't act very agile. When they are documenting things, as somebody shared with me some time ago, they're looking for 18 pages of documentation on every data element, but they want to know the data flows defined to such a degree that it takes a lot of time and it slows down effort. So the last thing that the folks on the agile teams want is to slow down their effort. So what we need to do is we need to relate these two together. And one of the ways to do that is to look at the different principles that are associated with the agile development methodology. So here are some that I've gathered over time. Talk about completing features before moving on, testing things early and often, finding active user participation, timing things in such a way that we deliver things incrementally and quickly. All of these things that are listed on the screen are typical terms that are used when delivering agile projects. So we need to take a look at those principles of agile and look and see which of those principles are the ones that make the most sense for us to attempt to deliver data governance through. Before we do that, I also want to talk quickly about the data governance principles that I've shared in previous webinars where organizations want to do four primary things. They want to manage data as an asset. They want to formalize accountability for the management of data. They want to make certain that the rules that are being thrust upon them through the industry or through the government, make sure that the organization is following the rules and they want to be consistent in the way that they deliver data to the organization. So we've got to take a look at those principles and take a look at the principles that I shared just previously on the prior slide and look at how can we apply the governance principles to the different agile principles that are so common in agile projects. I'll share that with you in one quick second. Now, what we're going to do here is we're going to look at ways of bringing these approaches that are very different from each other closer together. We're going to be getting by looking at what they have in common and we're going to identify and pick off, you know, one or a few of those at a time and make sure that we can strengthen the way that these disciplines are working together. We want to build it on that. And as it says on the back of your shampoo bottle, we want to rinse and repeat. You know, in a lot of organizations right now, we have people that are focused on data governance. We have people that focus on the agile development. But these people don't talk to each other. So right now I would answer the question, is there a difference between data people and agile people? I would say yes. You know, we're looking at people that have two entirely different thought processes as to things that need to be developed and how they need to be developed and oftentimes they conflict. So let's talk about looking at these agile principles as I spoke about before and pick off a few of them that we can apply data governance to. And so the four that I've selected to talk about in this webinar are the active user participation. You know, looking at requirements at a very high level, which again is a principle of the agile development methodology. Talk about managing something that's called technical debt. If you're not familiar with what technical debt is, I'll talk about that in a few minutes as well. And we're going to talk about in agile projects where requirements evolve and the time scale tends to be fixed. So we want to make certain that we can apply governance where it makes the most sense to apply governance to your projects. So we're going to focus on these four today. So again, the agile principles that we're going to target to apply governance are the active user participation, the requirements at a high level, the technical debt that needs to be paid, and the fact that requirements evolve typically as a time scale stays fixed for these types of projects. So the first item I'd like to talk about is active user participation. Now, most places that I've read about agile development methodologies, they talk about actively engaging people through the processes associated with their agile projects. You know, it's kind of funny that data governance speaks the same language. If you recall, there was a paper that I spoke about years ago that I talked about the Data Governance Bill of Rights. And the idea of data governance typically is to get the right people involved at the right time to do the right thing using the right information to lead to the right decision. That is really, in a nutshell, what data governance is all about. Now, we want to make certain that we're getting the appropriate people, we're getting the users of the data, the users of the systems involved, we get the decision makers involved. You know, we want to make sure that we get the right people involved, so we want to have that active user participation in data governance as well. And we want to make sure that we get them involved at the right time. And in previous webinars, I've shared with you diagrams that I call a data governance matrix, data governance activity matrix, where we take the steps of a methodology and we define them down the left side of a page. And across the top, we have the different roles associated with the governance program, whether it's operational stewards, tactical stewards, the council, the data governance team. You know, one of the things that we try to do with these governance activity matrices is get the right people involved at the right time to do the right things and all the things that you see on the screen in front of you. Well, if it's true that the agile delivery methodology calls for very active user participation, you know, if we can involve data people in those conversations early on to make certain that the right people associated with the data requirements in the organization are involved at the appropriate times in the agile projects, then that's one way of being able to apply data governance into your agile efforts. Kind of do those governance activity matrices, but do them from an agile delivery perspective. And make sure that the steps for each of the scrums and each of the quick actions that are taking place during your agile project that we make certain that we have the appropriate data people or people with knowledge of the data involved in those steps of the methodology. So again, it's the data governance bill of rights. It was published back on TDAN in January of 2014. And it talks about, again, getting the right people involved at the right time to do the right thing. Kind of leading to the right decision. And I joke about it, kind of most of the time. But the idea is to get to the right decision as high percentage of the time as possible. So data governance is really all about that bill of rights. And on an earlier slide, I shared with you kind of those data governance principles, but in a lot of organizations, they design their governance policies around principles like that. And we want to make certain that we have people in the organization that understand that data is an asset, that we understand that they have clearly defined accountability, that they understand the rules that are associated with protecting the data or defining the data or producing the data. We need to have active user participation through all the steps of the agile delivery methodology. And again, that's one of the ways of applying governance to the agile methods. The second agile principle that we wanted to target to apply governance is that in the agile effort, it's often time the requirements are being defined at a very high level. And one of the things data practitioners love to do is they love to capture metadata. They love to capture documentation to the nth degree about the data, about the process, about the modeling of the data to make certain that the data is accurate for those projects. So data governance practitioners kind of pride themselves on capturing metadata as part of their work, whether it's business glossary information, vocabulary, dictionary, lineage, all of that information is very important to data governance practitioners. And so we spend a lot of time collecting that type of information. On the agile side, they pride themselves, the folks that are delivering agile projects, pride themselves on being structured, delivering things quickly. So one of the things that I've kind of added to this set of slides is something that I'm calling words of wisdom. And Shannon mentioned this earlier. We've got the hashtag RWDG for tweeting. These look like kind of tweetable things to say. But the truth of the matter is that these words of wisdom are truly words of wisdom when it comes to applying governance to agile efforts. So one of the first words of wisdom that I want to share with you is that we've got to get practitioners from both sides, from data governance, from data management, and from the agile delivery side to agree to a certain level of requirements. And if we can do that, again, now that's another way of applying governance to your agile delivery method. So what we want to do is we want to get them in the same room. We want to get them to agree what level of detail of requirements are we going to need to move this project forward. You know, it's nice if we could get those folks to talk to each other and get them to agree, but in some organizations it seems like that's just not going to happen. So it might take some level of senior level leadership education first on the relationship between these two disciplines. And if we can't get the practitioners of those two disciplines themselves to come to an agreement on the appropriate level of requirements that are being gained, that information might need to be kind of enforced from the top down rather than looking for an agreement from the data governance practitioners and the agile practitioners. But the fact is that both practitioners, both sets of practitioners, need to come to some level of agreement on the level of requirements that need to be gained for any given project. Can the data governance practitioners limit their requirements? Well, certainly I believe that there is a way that we can limit what we ask for when it comes to our involvement in an agile delivery process. There are five primary categories that I often talk about, and I talked about in last month's webinar, the first three, the idea of the vocabulary metadata, the business metadata, and the technical metadata. Well, we want to make certain that we get the business terminology well defined, that we define data standards, that we have entries for our data dictionary, for data that's going to be used by people within our organizations. On the technical side, we want to make sure that we can document their databases and files and things like that. We'll talk about models here in a couple of minutes, but we want to make sure that we're getting vocabulary metadata, business metadata, and technical metadata at a bare minimum from the agile projects. We may not ask for as many things as we've always asked for in the past, but again, from a bare minimum, we need to know how the data is going to be defined and what the rules are associated with the data. We also want to understand where the data is coming from. It doesn't matter how you deliver your projects, whether it's through the agile approach or through the waterfall approach. One of the first things that end users of the data ask is where did that data come from? So we know that we need to have a way to collect the data movement metadata, the ETL metadata, so to speak, the extraction rules, the transformation rules, and the loading rules. We also want to make sure that we, at least from a minimalist perspective, understand the linkage metadata as well, the standards for the data, how it relates to the data, the policies, how it relates to the data, the procedures, and also then the relationships between the business terminologies and the business metadata. So there's a lot of metadata that we can collect, but we as data practitioners need to look at that list and say, what do we need? Why do we need it? And why are we asking for all these things? Because that's one of the things that agile people don't like, is they think that we're asking them for too much documentation. We've got to be able to demonstrate to them that the information that we're looking to gain is going to be useful to somebody, useful to them at some point in time. So again, we want to look at the requirements at a high level as a way of applying governance to the agile delivery methods. So there's differences in the way that agile calls for their requirements versus data governance calling for their requirements. Agile has it built into their processes through user stories and through their scrums and through their iterations. In data governance, we typically look from a high level and we drill our way down. We look at the big picture. We're looking for upfront analysis of the application. We want to make certain that we're collecting the information that we're going to need to support those applications that are being delivered. So another set of words of wisdom is a main reason why agile people think that waterfall is the thing of the past is that the people that follow the waterfall methodology, again, are asking for a tremendous amount of documentation and a tremendous amount of requirements. So again, if we can limit what we are asking for, we can take the requirements at a high level and again, come to some level of agreement as to the level of requirements that will be gained during an agile delivery project. A lot of organizations do data modeling as a regular practice and that's a great thing. But the fact is, again, we have people that are trying to deliver projects quickly looking at our modeling efforts and saying that the models require the questions are asked about cardinality, about retention, about growth. The models, they promote a common understanding but they also take a long time to gather that information. So if we can find a way to deliver models quicker, then it's going to be more effective for the people that are delivering agile projects. So another word of wisdom is that without the detailed requirements, oftentimes projects create their own understanding and their own understanding for data, which can in turn result in ambiguity, replication, et cetera. And isn't that a problem that we've had all along with the waterfall approach? Is that if we don't have an enterprise or a consolidated view of the data, as projects are being delivered, they take on their own understanding of the data. And if we're trying to get away from that, but we're trying to deliver projects quickly, we want to make sure that we have at least a level of detailed requirements that we need in order to eliminate the ambiguity, eliminate the replication, et cetera, in the projects. So we want to make certain that we are obtaining the correct level of requirements within the organization. So for us data people, I've got a few questions for us. Are we asking for too much documentation? Are we not asking for enough documentation, or is it the way that we're asking for the documentation that's taking the time that's getting the folks that are delivering agile projects to look at us and say, we really don't need to involve you in the project? So we need to look at what it is that we're asking for and we want to ask ourselves the question, do we use everything that we collect? And how do we use it? Who's going to get value from it? Can we, going back to that first principle, can we work out an agreement with the agile people as to the level of requirements and data documentation that's going to be collected during an agile effort? Again, another way of applying governance to your agile efforts. Oftentimes, if we don't get together and we don't bring the agile and the data people together to have that conversation, that in itself is a decision. We're deciding that we're not going to come to an understanding as to the level of requirements that are being gained. And we need to make that decision. That is one of the conscious things in our organizations that will really help to bring these two disciplines together is that we get these folks into a room, we get them to discuss what level of data requirements are going to be required during the initiative, and then we stick to that as part of the agile approach. So another set of words of wisdom is consider the value that you get from the metadata that you require, and make sure that you truly require it and that you're truly going to use it before you go and ask for that information to be collected during the project methodology. So to summarize real quickly, this agile principle, there is a limited demand for requirements in an agile approach who want to achieve an agreed upon level of requirements. We want to provide the tools to the folks that are running and delivering the agile projects to effectively capture the essential requirements and capture them quickly. You know, we want to make sure that we're focusing on both the data governance aspect and the quality and the value of the data definition and production and usage in alignment with the agile methodologies and the approach that they're taking. We want to make certain that we incrementally apply that agreed level of requirements from the data governance folks to the agile projects. So the third principle that we want to focus on real quickly is the technical debt. And if those, some of you may not be familiar with technical debt, that's typically requirements they get put on a parking lot list. There are things that need to be talked about that can't be talked about right now because it's going to prevent the project from moving forward. So that's something that you can consider technical debt. One of the biggest problems that data people have with agile projects is that the data, that the technical debt is not being addressed. So in another slide coming up, I have some steps that we can take to govern the whole concept around technical debt in the organization. And so the truth is that oftentimes this is the biggest reason why data people are skeptical about agile is that this stuff called technical debt keeps building up and building up and that it becomes a problem when the project needs to be delivered. We want to make certain that the technical debt itself is being governed as part of the agile effort, another way of applying governance to an agile effort. So what are some of the reasons why technical debt accumulates? And I'm not going to spend too much time on these, but there's business pressures to move quickly. There's lack of process or understanding. There's a lack of a test suite, lack of documentation. That's often the reason why this technical debt builds up that needs to be governed and needs to be looked at at some time. So there's additional reasons why technical debt accumulates is there tends to be lack of collaboration where the user participation becomes very important to the agile effort. Oftentimes the collaboration ends up not being at the level that the agile teams want it to be or that there's parallel development efforts that are taking place and additional things get added to the technical debt list, to that parking lot list. There's delayed refactoring and there's lack of knowledge. And these are just reasons why technical debt gets built up. But the truth is it's not really a problem, but the fact that there is technical debt that is taking place is not the problem. The problem is that that technical debt itself is not being governed. So we need to apply governance to the fact that that technical debt does not accumulate in the ways that we talked about on the previous slides. So how do we do that? We apply governance to the process. We have formal accountability for making certain that the technical debt is being addressed. We have formal escalation issues associated with the technical debt where things can go from the operational to the tactical to the executive level of the organization, but there is an escalation path for resolving issues. There's formal reporting of the technical debt issues to the senior leadership and those people that are keeping a watch on the projects and keeping them moving. You know, if you look at those things, the formal accountability, the formal escalation, the formal reporting, those are all aspects of implementing an effective data governance project. So again, if we can apply what we've learned from running data governance projects to our agile delivery method, we will find that the data is going to be taken care of. The technical debt is going to be taken care of. The documentation is going to be taken care of. Again, another way of being able to apply agile or apply data governance to an agile effort. The last principle that I want to address is, and this is perhaps the biggest challenge when we're applying governance to agile, is that we've got requirements that are evolving and we've got a fixed time scale. And that's not a good duo of things. You know, we want to make certain that requirements are being defined and that they're being discussed and that they're being resolved. So if we have evolving requirements, then we need to have a constant method of being able to consider what those requirements are and the impact they're going to have on the data and on the process. So oftentimes when you're delivering an agile project, you're kind of shooting at a moving target. So oftentimes people talk about requirements evolving. It means that there's going to be a gradual process in which the requirements change into different, and oftentimes when they're different, a more complex form. So let's take a look at each part of that phrase. The gradual process, it's not all at once. We're going to make sure that if the requirements are going to be evolving, that we want to have constant management of that effort. As it changes into something different, we need to have constant management and governance of that as well. Usually the result is usually more complex. Oftentimes it's a better result, but we want to make sure, again, as those requirements change through a gradual process, that somebody has the responsibility for making certain that we're taking into consider not just the system requirements, but the data requirements as well. So as those requirements evolve, and there's a fixed time scale, we want to make sure that, again, from a minimalist perspective, a data governance perspective, we want to make sure that we're covering at least that level of requirements that are going to be necessary to deliver a successful project. So the management of requirements actually requires the governance of those requirements, and that starts, as I mentioned earlier, of bringing those teams and those disciplines together. In a lot of organizations, there are three things that are associated with a project. There's the time that's available to deliver the project. There's the deadlines and milestones and those things. There are the resources, the limited number of resources that are available to help to deliver that project, and then there's what's going to be delivered during the effort. And in most organizations, their philosophy is, you know, give me two of those three things, but don't give me all three. If you tell me the time that I have and the resources that I have, I will tell you the effort that's required in order to achieve those goals. If you tell me the resources and the effort, I'll tell you how long it will take me to do that and so on and so forth. And one of the things that I've found in agile delivery projects is to try to dictate all three. This is the time that you have, these are the resources you have, and this is what needs to be delivered. So in your organization, if you want to take a look at your requirements and say, you know, we cannot take, we cannot have all three of those force-fed to us, we need to be able to negotiate. You know, you give me two and I'll give you the third. That's another way of being able to apply agile, apply to governance, to agile efforts. It's actually a form of governance itself. It's a form of project governance. The only problem is that it's just not a data governance effort. It's a project governance effort, a project governance approach, and oftentimes it does not consider the necessary governance aspects of the project. So the bottom line is with governance, we want to execute it in a force authority, as I mentioned earlier in the slides, and how are we going to do that? We're going to do that through our agile efforts because the agile efforts are not going away, but at the same time, we want to make sure that the data aspect of these things are being addressed. So the challenge is that we want to make certain that we gather the most valuable requirements, the most valuable documentation, and we use them effectively, and we make them available, and we make certain that the essentials are covered, and we're able to do that in a short period of time. If we can do that, then we will successfully be able to apply governance to our agile efforts that are taking place in our organizations. So real quickly, a couple best practices associated with applying data governance to agile, as you saw through the analysis of the different principles for the two, agile and for data governance, that there are many synergies between those disciplines. We want to make certain that people at the senior leadership level, at the project management level, and that we show them to people and that we get them to agree that there's ways that we can apply governance to the agile efforts through those specific initiatives or through those specific principles. We want to negotiate the level of requirements and the documentation that's necessary. We want to get leadership to understand the relationship between the fact that agile is the way to go, but data governance is absolutely necessary as well. Oftentimes, software is considered to data the same way that pipes are considered to water. The water flows through the pipes, the data flows through the software. We've got to understand that agile is important and that software is really the engine for making data available as an asset. But we want to make certain that software that's being delivered has the best-valued data in it as possible, and that's the reason why we need to apply governance to our agile efforts. Typically, when I talk about best practices around data governance, I talk about these five things. We need to have senior leadership support, sponsorship, and understanding, but let's extend that now beyond data governance. Let's extend that to the relationship between agile and data governance and get senior leadership to understand that yes, these things are both necessary, but they can't operate in their own little utopian environments. They really need to come together. Somebody has to have the responsibility for making certain that governance is being applied to the agile projects. The roles and the responsibilities are basically the basis for an effective governance program. They're also the basis for an effective agile project as well. We know we need to govern the technical debt, and we need to make sure that we have an ongoing process for reporting to our leadership how these two disciplines that are different in the eyes of so many people, how they are truly aligned and why they need to be aligned and how they're being aligned in our organization. Real quickly, we talk about things called information technology. Every organization talks about information and technology. The funny thing is that very few of those organizations look at those as really two separate subjects. You've got the information subject and the technology subject. The fact is that they're really together in name only. Oftentimes the people that have responsibility for the information are very different from the people that have responsibility for the technology and the software and for what's being delivered. We need to bring those people together in order to have agile data governance in our organizations. We need to adjust the way that we do data governance for agile projects. We limit the scope of what we are requiring as data practitioners and control the volume of data requirements that we have for these teams. If we do that, we can get the folks who are delivering the agile projects to have more of a data focus in their daily routine. The thing is that we just don't want to get in their way. We want to make certain that they can deliver in their timeframes, but their timeframes have to be effective timeframes, not timeframes that are defined without having any concern for data requirements. We need support from the data people and we need support from the agile people. All you data people out there, we need to recognize that the agile development methodology is here to stay. It's a strong methodology that oftentimes it's not necessarily the methodology that is the problem. It is the way that the organization is following that methodology. If we can apply governance to that methodology, we have a much better chance of being successful. The goals of being agile align with what's expected for management, so we need to understand and find a way to work in conjunction with agile people. For the agile people, the truth is data governance is here to stay. The protection of data, the improvement of data quality, the use of data as an analytical tool is here to stay. It's not going anywhere. We need to make certain that if you're delivering your projects in a way that we have a way of collecting the appropriate data requirements, we know what data flows in and through the pipeline that you're developing that we help you to improve the data quality and really you need to find a way to work with the data people of your organization. Again, it's the bringing together of these two groups that tends to be the most effective best practice for applying data governance to your agile effort. I know I spoke a lot and quickly and there were a lot of slides, five main things that we focused on on the webinar here, and then I'm going to turn it over to Shannon to see if there are any questions and I will gladly provide the answers. But we talked first about relating the disciplines for senior leadership, getting them to understand that these things need to cooperate and work together. We talked about ways of finding common ground between agile and data governance. We talked about different ways to be able to apply data governance to individuals associated with the agile efforts. We talked about best practices and gaining agile support for data activities. And with that, I'd like to turn it back over to Shannon to see if we have any questions today. Thank you, Bob, for this fabulous presentation. As always, we do have a few questions coming up. And evidently, we have some sound issues going on. Thanks for everybody's patience with that. Evidently, we have a new step in the sound process to call into the Internet audio. So there's just an extra step in the communication and call computer. So we will get that sorted and do some more tests and make sure that that's included in the reminder emails moving forward. The second most popular question that we get, of course, or the most popular question, even still today, is a question about receiving a copy of the slides and a copy of the presentation. I will be sending a follow-up email within two business days. So for this particular webinar, I will be sending a follow-up email at the end of day Monday with links to the slides, the recording of the session, and anything else requested throughout the webinar for this particular webinar. If we don't get a chance to get to your question before the top of the hour, Bob will write up an answer and we'll get that out in the follow-up email as well. So let's dive into the specific questions here, Bob, for you. So we've got a comment here regarding the big picture. So you can start big picture, but then boil down to stories. And this is really where you're talking about the levels of agile planning. Okay. Yeah, the interesting thing is that since there seems to be so little communication between the agile and the data folks, is that we don't have the data people associated with those user stories in asking the questions that data people like to ask so often about the requirements for data. We do like to take a big we, I should say, the people that are data practitioners and truly I am a data practitioner. You know, I like to think that we want to make certain that we're crossing all of our T's and we're dotting all of our I's when it comes to understanding the data and having that relationship to the folks that are doing project methodology, following a project methodology, whether it's agile, waterfall, whatever approach it is that you're following. You know, over the years, we've looked to try to attempt to instill data discipline into the system development lifecycle in the waterfall approach. And we've had varying levels of success. A lot of people have said that even during the project methodology that we've taken kind of too much of a big picture approach and that we need to be more specific as to what we need. If we can take what we've learned from the management principles to the waterfall approach and we can kind of apply that same thing to the approach that's being used to deliver agile projects and if that's through user stories or through whatever it is, we need to make certain that we're getting the right people involved at the right time asking the right questions if things need to be parking lotted or technical deaded, as I spoke about earlier, somebody needs to govern that process as well. I'm not going to make a big picture approach, but if the project is being delivered incrementally, then again, we just need to associate data people to each of those user story steps to make sure that the appropriate level of data requirements are being addressed. I hope I answered your question right. And we can certainly get more comments in case a more in-depth answer is needed. But why not share a central data model with Agile Team as a starter for 10 to save them inventing their own understanding then adapt it to reflect any new requirements not yet included? You know what? That's a darn good idea to share the enterprise data model. First of all, you've got to have a data enterprise data model and some organizations do and some organizations don't. But if you do, the problem becomes getting the Agile people to want to look at that model. So you can have the model, but we really need them to take the time to learn the model, to learn the understanding of the data so they don't come up with their own. The problem with coming up with their own is obvious. It is that we're delivering another project with data that is now different from the other projects or the other data stores that we have in the organization. And so we really, really don't want that to happen. So what we want to do is we want to make certain that we're doing the right things from the Agile perspective and that we're also doing the same thing, the right things from the data governance perspective. And to do that requires a level of agreement. And how is Agile data governance different from extreme programming data governance? You know what? I really haven't heard the term extreme programming, extreme programming data governance, especially in some time. I mean, extreme programming, the understanding of extreme programming is that it was kind of a predecessor to the Agile approach and that it was very much like the Agile approach where things were done incrementally and being done quickly. And so it was extreme programming mostly because of the method that the requirements were being gained and that the project was being delivered. I'm not sure that I've heard of anything called extreme programming data governance, but to apply data governance to extreme programming would be very similar to applying governance to an Agile effort. So I would say, at least my thought is that one is the predecessor of the other and that we could take the same diligence to both. Yeah, and the comment continues that says it's much more extreme programming, it's much more Agile continuous development, no iterations, ongoing development, constant refining. Right. And so that's it. So if you're constantly refining and there's no control or there's no governance around that constantly refining, it becomes problematic. So I'm not saying that you can't do the constantly refining, it's just that that process needs to be governed better than it's being governed in order for projects to be successful. All right. So the next question comes in. I've seen Agile developers build that mimic report and input screen mockups, complicating ETL data transformations between the two. How do you avoid this pitfall? A bit of logical data modeling? If you can answer, that's like the $10 million question. If we can answer that then we can really answer what brings the two disciplines together. You know, it's got to be, we've got to get to, and I know I've said this multiple times during this webinar, but it really kind of sits at the core of what's required. It's bringing the Agile team and bringing the data team or the people that have the data business requirements responsibility for the project together. And help them to stay Agile and help them to follow the method that they're getting better at, at least for a lot of organizations. And help them to understand that there are data aspects that need to be done. So the screen mockups, the report mockups and those things are often really nice to have because they're kind of going to be the end result that the end user is going to see, but it doesn't necessarily cover the whole story. So we talk about user stories, we should talk about the whole story from beginning to end. Would a logical model help? Yes, certainly a logical model will help if you can use that to relate to people, you know, why the data is the way that it is and why the data becomes so important. So what we don't want to do is we don't want to interfere with how they're delivering it, but we want to make sure the data requirements are covered, and that's really going to be, from my experience, the only way to get the teams to work together as one or at least work together as partners aligned. It looks like we have time for maybe one more question, but like I said, these are some great questions coming in, so just do keep them coming in. We will get answers written to any questions we don't have time to get to today, and we'll get that in the follow-up email. The next question, Bob, is how would you suggest implementing and applying to many stories? Do you capture that as an epic or as a non-functional requirement? You know what, I'm really probably not the best person to ask, so I know this is going to sound like a cop-out, but I really, from a user story, from an epic perspective, I'm sure that there are sets of data requirements that can be gained from each of those. It's just that I am not an agile development team, and you bring together what they need and what you need and discuss it together. I can't really talk to the epics and to the scrums specifically, but talk to your local agile team, and they'll tell you what they need out of those aspects of the project, and then you look at it and you tell them what, from a data perspective, you need, you know, to solve that problem. Sounds good. Well, we are right at the top of the hour, Bob. Thanks for a great presentation. As always, much appreciated, and thanks to all of our attendees for being so interactive in everything that we do and to helping each other solve the little sound issues that you guys were having. I appreciate that, and I look forward to meeting up next