 Hello and welcome. My name is Shannon Kemp and I'm the Chief Digital Manager for Data Diversity. We'd like to thank you for joining the latest in the monthly webinar series, Lessons in Data Modeling with Donna Burbank. Today Donna will discuss Agile and Data Modeling. How can they work together? Sponsored today by Clover ETL. Just a couple of points to get us started. Due to the large number of people that attend these sessions, you will be muted during the webinar. And we very much encourage you to chat with us and with each other throughout the webinar to do so. Just click the chat icon in the top right-hand corner of the screen for that feature. For questions, we will be collecting them by the Q&A in the bottom right-hand corner of your screen. Or if you like to tweet, we encourage you to share our questions via Twitter using hashtag LessonsDM. As always, we will send a follow-up email within two business days containing links to the recording of this session and additional information requested throughout the webinar. Now, I'd like to send it over to our sponsor today, Clover ETL, for a quick message. Pavel, hello and welcome. Hello, everyone, and thank you for the introductions. I hope everyone is good. And also, I hope you didn't get scared off by Shannon's selection of spooky tunes before the webinar started. I guess we have a Halloween theme here. So, I'm not sure if my short bit will be a proper like customer, sorry, a sponsor or presentation. It's more an idea or something that came to my mind when I saw Dona's theme, which is agile and data modeling and how they go together. And there's a project that we're working on which is very, very close to it, like how to bring the world of data modeling and the benefits of data models into like the operational or production environment where traditionally ETL and data integration tools play their part. So, before I start and jump into the project, let me introduce who I am and what we do. I'm from Clover ETL, we're a data integration software. We're a platform that focuses on the end-to-end data integration process from tools for rapid design, developing, debugging, sharing with developers to focus on transformation and the power of combination of visual tools and code. We're heavily focused on orchestration and automation and at the end, obviously, we focus how we can get the data out of that process and make it usable to as wide selection of applications and systems that we can. So, we play in the, historically in the ETL space and we moved on to broader data integration. Now the project. We were a couple months ago, we had a client who came to us and said, look, we have a lot and a lot of data models and SQL scripts and I think it's in an IDA data architect and they started thinking how to transition to an ETL and the benefits of a data integration tool and we started a project where at first we realized that developing these data architectures and then operating them is very disconnected. The teams are different, the technologies are different. The modeling tools are great at modeling obviously. You can define the data relationships, you can discover data and so on but they quite weak in the production environment and that's where you get ETL tools or data integration tools like ours. Where you have that connectivity, you have that automation, monitoring and everything that keeps the thing running. So we started thinking about a bridge, how we can quickly move from one world to the other. First we had to deal with transitioning or converting the metadata that was kept in the modeling tool and then obviously has a similar concept so that wasn't the hard part. The biggest challenge and we've been working on this for a couple of months and I think it's very interesting that we actually pulled it off is converting the models from the data architect and the SQL scripts that describe those into clover ETL transformations which is what you see on the right. It's a visual representation of that SQL script and it has all the properties of that model and all the domain requirements and restrictions and constraints in it and once you have that transformation, that visual representation you can move to the world of data integration which offers you those production benefits like you can connect to data outside of your traditional databases you can work with message queues, APIs, modern no SQL databases so you get a lot of connectivity. You get that robust execution platform which allows you to repeat the processes as often as you want. You have the automation which allows it to connect to other systems and orchestrate that process and you have the monitoring on top of that. And finally you have that ability to publish the data to whatever you want. Again not just databases but also files, those message queues or whatever can be on the input. So this is my short message. It's just a cool project that we've been working on for a couple months and it's putting the worlds of data modeling tools and the outputs that these produce and making it very quickly usable in the robust data transformation or ETL world and being able to move quickly between those two without having a separate development team that takes the models and implements that into ETL. So I think that's all from my side. If you have any questions I'll be here throughout the whole session so feel free to ask questions and Shannon will be so kind to help us with the Q&A's and over to you Shannon. So thank you so much and again thanks to Clevver ETL for sponsoring today's webinar and helping us make all this happen. I'm going to switch it over to Donna now to get the presentation started. Donna she is a recognized industry expert in information management with over 20 years experience in helping organizations enrich their business opportunities through data and information. She is currently the managing director of Global Data Strategy where she assists organizations around the globe in driving value from their data. She has worked with dozens of Fortune 500 companies worldwide in the Americas, Europe, Asia and Africa and speaks regularly at conferences. In fact you can meet her soon and you can take a meet Clevver ETL in person at our upcoming Data Architecture Summit in November. So in Chicago which will be so much fun I can't wait to see everybody there. We've got a lot of webinar folks coming to that. So Donna hello and welcome. Hello. Always a pleasure to join these webinars. And yes it will be nice and warm in Chicago in November. So hope to see you all. I'm looking forward to that. So as Shannon mentioned this is me. I'm also on Twitter at Donna Burbank and if folks are Twitter folks we have a hashtag less than DM if you want to continue the conversation online as well. So jumping right in as Shannon also mentioned this is part of a larger series in data modeling and the good news is this is your first joining. They are all recorded on demand and they are all out on the Data Diversity website. So you'll see that there's sort of a wide range of topics. Everything from enterprise architecture to BI to metadata. Everyone's family favorite topic because as you know if you've joined these before you've heard me say data modeling is only interesting when you're using it for a particular purpose. So I love data modeling in and of itself but really the exciting part is when you can actually use it for business value and that's going to be a big topic of the conversation today where we talk about Agile or Agile. Potato, potato depending on how you say it. I'll probably switch between the two and how they work together because I think a lot of people in it think of Agile think that there is no data modeling involved and hopefully I will try to spell that notion that I think data modeling is actually very helpful in the Agile lifecycle and I've had several customers I've worked with that have done that very well. So hopefully we will delve into that today. So starting with that we'll just talk about some key definitions. What do we mean by data modeling? What do we mean by Agile? You may be familiar with one or both or neither of those so that's fine as well. And then part of the most important part of the topic is really that business value of data modeling in an Agile way and we'll talk about my definition of capital A Agile in terms of the Agile manifesto and kind of what I call the lowercase Agile of just being more efficient which I think we've done for a long time with data modeling. And then what that actually means is kind of a nitty-gritty of what it means to be Agile when we come to data modeling and then Q&A for either for myself or for the Columbia notebook. So jumping right in I just wanted to point this out that again data modeling is harder than ever so a little bit of a plug but this is a free white paper that is available on the DataVersity website that we just published last month actually so it's only a few weeks old. Over 96% of the folks that were interviewed in this paper were engaged in data modeling. I'm going to have to stop and pause there because I think as an industry we're sort of the unsung heroes that one of the problems they must always be behind seeing this little engine that could that's making everything else working. So we're not always necessarily in the front. And you might have heard that data science is the sexiest job of the 21st century. I beg to differ. I think it's data modeling. But I do think we have a bit of an image problem and the fact that everyone's doing it. Everyone's using it. People like it. It's effective but we don't always sort of shout our name from the rooftops. And so I'm going to. Over 96% of the folks are doing it. Clearly it's working because most companies are doing it. You know that said this survey was aimed at architects so slightly self-selected but it is showing that some of the in the survey you'll see the breadth and the size and the different scope across industries. It isn't just one. It isn't just health care. It's really across the board. So I wanted to throw that out there. If you're not familiar with data models this is sort of a quick summary of what they are. What's good about them is that they are sort of by nature agile. What I like about a data model is that it's very visual. And where it gets nice in one place both your technical metadata. So what do we mean by customer first name is it can it be nullable? What are the constraints that are actually going to live on a daily kind of the panel's point. Making it very actionable. Creating database structures in the operational environment. But at the same time you can just switch the display and a lot of tools and really make it a business tool. What do I mean by customer and anyone who's met in the business knows how difficult that can even be. Do we need a customer? An organization? Is it someone that's bought a product in the past year? Or what about people that might have bought from 20 years ago? Are they still considered a customer? And all the stuff that data architects like ourselves can wax poetic about can be captured and published in a data model and the beauty of that is this directly linked to your operational environment. It isn't one, you know, Pavel mentioned it as well. Sometimes the issue is it's not necessarily the technology of both the teams. Teams might be living in different parts of the building and living in different organizations. But a data model can kind of bridge those gaps of where the business rules from a business user. They may be in the glossary but why don't we just put them in the data model so that's what we're going to use in the technical mandate as well. The other thing that can be very agile, often I will do this first thing and a client is what do we even have in this database? You just quote reverse engineer it. Most data modeling tools can create a nice pretty picture like this that very visually, very intuitively in a very agile way, really see what even those data structures exist. So there's the top down aspect of I'm going to build my systems with a data model. But there's also that bottom up of what do I even have? And if you work for any large organization or even a small one, sometimes that's the biggest challenge of just making sense of what you have already. So that's the data model. And what is agile? So probably the easiest way to explain that is the agile manifesto, which is the founding principles of agile. And again, depending on what camp you're from, this may be what you do or may be what you breathe every day. But it has some sort of core tenets, which makes no sense. The idea of individuals and interactions over processes and tools. Not processes and tools are bad. But at the end of the day, we're trying to get why we're using tools to get people working to get, and building stuff, right? That's probably a good idea. But let's not forget that there's human beings at the end of each one of these processes and tools. The idea of work software over comprehensive documentation. And I would just say here that doesn't mean no document. And I think that spends on the fallacy of some folks interpretation of agile. Oh, we don't need to document anything. We're just going to run off the code. Well, that's not over time very productive because the next person who looks at that, what the heck does that mean? How do I document a computer story? What it means is just enough documentation. And I think a data model and hopefully by the end of this we'll show can do just not in a very comprehensive way to suss out sort of a business problem. What do we mean by customer? Can a customer have more than one account? Not a huge problem just to suss out in a small data model. It's much more effective. So again, not no documentation, just the right amount of documentation to get things done. The idea of customer collaboration over long contract negotiation or sort of fighting over specs. And it really should be, let's just try to get stuff done. I mean, if you've heard me speak before, you know that I'm a big fan of a business center data model. And yes, I have gotten extract. That's what I was doing this morning in a client. We really get business people in a room over a high level data model to suss out what do we mean by customer? What do we mean by contract? And it's just a nice, simple way for both technical people and business people to kind of speak the same language. There's a business rule and there's the way it's implemented in the database. And sometimes the best way to explain that is through a data model. So great for customer collaboration and responding to change over following a plan. And again, sometimes data models can have the, hopefully by now, misguided assumption that this is the old waterfall where it takes just a year to build an enterprise model and then you do something. If you can do modeling in an agile way, it's a great way to respond to change. Let's reverse engineer the database and compare that to the business rule, what's wrong with it and move on. So again, it's more how it's done, not the what. So data modeling can be very agile and it fits a lot of these actually very well, especially when it comes to things like collaboration and interactions. To me, that's a big part of what a data model does. But I am a big fan also that we've been doing agile for a while and one of my past lives was an English major way back and I don't like when people steal words for their own use. So we've been doing agile for a while. You get to take the word agile in it, right? So there's also the English word called agile, which is in the dictionary before the agile manifesto, which has a similar concept but slightly different. It could be, here's some definitions I took from dictionary online, quick and well coordinated in movement. And when you think of it in terms of software development, sometimes those are the same thing and they shouldn't be mutually exclusive. But often I've heard in development projects, I just need to get this done. I don't have time to talk to people. And sometimes that's true, but I think over time, and in fact one of the customers I'm working with now is trying to, that work when they were in the startup mode and now they want to be an enterprise company and that doesn't always scale, right? What I'm doing in my basement, you know, building code alone for a small project doesn't work when I'm an enterprise international company trying to coordinate 16 departments. So that idea of a data model helping coordinate some of the different definitions of what do we mean by fiscal month? What do we mean by account? That may be different across these groups. And hopefully a data model can be both quick and well coordinated because being quick doesn't mean going, what did they say? If you're going to go fast, go alone, if you want to go far to go together. That's sort of where data model can help, that you can be quick and well coordinated because over time we're doing both together much better. You know, active, that's sort of the most often used word, but active in the right way, right? So we've all been on those projects where everyone's running frantically around the company, what are we getting stuff done? So doing it in a planned way through a data model and getting your rules up front or post-fact, at least understanding what the databases look like can be active but active in the right direction. This last one I really like, marked by an ability to think quickly. I mean, I know I'm biased in the fact that I'm a big fan of data modeling, but I've done this so often with the client, it's sort of the second nature of there's a problem. You can a customer have more than one email. Yes, but the system won't do that. Well, here's why the system won't do that. You're storing it as an attribute. It really should be a second class. It's an entity, so you can have one to many emails. That's a really quick way to think quickly around a very focused problem because you see it visually and we are such visual creatures that's often where a data model can help you think quickly, just draw it out, just suss it out. What's the difference in the database and the actual business role? Hopefully you'll see by the end of this, if you haven't already, data modeling can be both agile in terms of the agile manifesto, which I like to call kind of capital A, agile. That's the agile in terms of the named term and then kind of lower case agile, which is agile, the adjective, agile, meaning quick, fast, well-coordinated. The utility is not a new concept and I know this might be a very strange picture, but heck, I had it and it seemed to fit. The picture on the left is my mother. She's the one doing the back bend in 1947 on the beach near Boston. The one on the right is me in the mountains of Switzerland. I went through this phase. I'm a big mountain fan, if anyone knows me. I went through this phase and gave you the top of the mountain doing a back bend that sort of matched the mountain, which is very odd, I know. The folks that have children, you might say, okay, suddenly my daughter loves to go to the Science Museum and all she's talking about is science. She wants dinosaurs for Christmas and she actually wants, I don't know, a chemistry set for her birthday. And then you look back in your family genealogy and your great-great-grandfather was a Nobel scientist for chemistry or something, right? And you say, how is this stuff genetically and some of these odd things? So I couldn't help but notice there's a bizarre family resemblance, so even the ponytail of doing agility. So that was a sort of a clever way, hopefully, of saying that agility is not new. We've been in that for a while in terms of the agile being an agile. And the sages agree, right? I'm sure you've heard the journey of 1,000 miles begins with a single step, right? That was the agile manifesto right there, way back in 500 BC of don't try to do everything at once, a big bang approach. Start small, do it in my little steps and you'll get there eventually, but don't try to bite off more than you can chew. And that's the same with data modeling. You might not have heard this, but it's out there. So if you bite into everything, it's the same. It's yard by yard. Everything is hard. Same idea. Just bite off small pieces. Do not try to do everything at once. Other ones, pitch in time, saves nine. Do it right the first time. You don't have to do it again, which is literally the next one. If you don't have time to do it right, you have to do it again. And that's sort of one of my, I don't know, I don't know if you've heard my webinars in the past of, yeah, it's faster if we just skip the design phase. Well, that's really not faster in the long run, right? So you're probably going to have to rebuild it. If you didn't get it matching the business rule, and it's going to be a whole lot harder to fix down the road. So spend the extra, you know, few hours to flesh out a data model. And I'll talk about that later. It doesn't have to take weeks. It doesn't have to take months. I've done a data model in five minutes to flesh out the problem. You know, kind of doesn't have more than one email. And it's just a way to do it. So can you integrate these data models in small chunks? Just fit your agile story and move on. You don't have to do an enterprise model every time you do modeling. You know, those of us who love it and they want to, you can go ahead and do that in your own time. But for the business, just pick those small chunks. And just do it, which is sort of Nike. Just do it. Often we, a lot of folks who do modeling tend to be academic and we can tend to overthink things. And I have to remind myself that. I know the model is not right. Just put it up. And that's the idea of agile. Iterate. I'm going to put up 80% that I know is right. And someone's going to tell me it's wrong. And don't take that personally. I'm just trying to get there, right? Or the last one. Implement your data modeling spelled wrong. Sorry. Project in small integral steps, creating quick wins that build to a longer term, sustainable architects. All right. Maybe that won't fit on a sticker. That's one of my points that you can do ads. You can do data modeling in an agile way in small steps. Build quick wins. And you'll get to that enterprise model over time, but in the sprint life cycle. Don't try to do it all at once. The journey of 1,000 miles begins with a single step. And that's true with data modeling as well. So I showed this slide before in previous webinars. I took my enterprise architecture for one. I showed this, but it's true with data architecture. And I've gotten good feedback on it. So don't give me good feedback or you'll keep seeing the same slide again. That's your punishment. But I think it sums it up very nicely. It's finding that right balance with data architecture. Sometimes the thinker on the left with the laptop is sort of the, whether it's valid or invalid, the reputation, our data architecture content, and have all those guys that sit in a room or gal sit in a room and think great thoughts and never get done. I don't have time for that. I just don't. Well, it may be slightly true with some teams. But at the same time, if you don't do entity design and modeling, you're that Wild West chaos and still nothing gets done because you're all running around agilely, but you're not running around in a coordinated agile, right? It's the bad part of agile. It's not the efficient part of agile. So there's sort of a lever in the middle. And what is that right balance? To me, that's the business value. Do just enough modeling that's going to give you value and then move on. As soon as you start, and I've done this in teams and I've done it to myself, as soon as I start to get too academic and just think for thinking sake, stop it. Am I discussing this point because it's going to change the business point or am I discussing a point because it's interesting and filter yourself. And that's sort of the right balance of either extreme is bad. So try to get that right balance point. This next slide, we don't have a survey going, but just if you want to put it in the QA or you want to think yourself, I think this is kind of a helpful kind of current state maturity assessment if we're going to use consulting terms. But kind of where do you sit? And maybe give that some thought in your organization. Are you too academic and nothing gets done? Are you not doing anything already? You have that right balance in your organization. You have that ultimate as a C, at least in the American educational system. But I'd say a C is just fine here. A little funny anecdote. I gave a similar presentation actually last week at the Denver Data Management Association, Dana. One woman raised her hand and she was like, what do you mean by your team? Would it be my team? Would it be the entire like, oh, you're in A. She wouldn't even answer it and she laughed. I gave her a hard time and she took it. But she couldn't even ask the question without really analyzing it, which a lot of us do. She was probably right in getting that clarification, but you know, we're just doing a raising hand level of effort. So that was a great example of overthinking a simple question and often we do that because we're architects and we love to architect everything, but don't just pick the right decision at the right time and use business value as kind of your guide. So how do you make sense of it all? Because there are a lot of data sources out there and it seems easy to just say, okay, just pick the right balance, but that isn't magic. So you do want to be agile and provide those results, but it is complex. We're all employed in data management. It's not like you have one little tiny database that does everything. You probably have hundreds across the organization, but you want to be fast but also sustainable both architecturally and organizationally. Hopefully this fits in with a larger governance process, not just you modeling for modeling's sake. Again, I've shown this one before as well, but I think that helps set the stage where modeling really is in the scope of a larger enterprise landscape. So you should be looking at your business strategy and how that aligns with your data strategy. Why are we doing this anyway? Why am I wearing it for a customer kind of more than one email? Well, because all of our marketing campaigns are now going to go digital. We won't send any physical mail and everything is keyed off email. So yes, it is worth us spending the time in this meeting for an hour to flesh out that decision and how it works in the database. It could be, you know, we have the customer's favorite hobby. Do we really need to worry about that? Probably not. But you might have marketing in your room and saying, actually, yes, because we've focused marketing campaigns based on their hobby. So we better get that right because you model it. They could have more than one hobby. Please model that correctly. So again, there's no right or wrong answer. That's a great reason to be working with the business folks and understanding what the business value is of each one of these areas. It needs to be governed by governance. And let's do a bottom-up inventory of data sources and that's often where a data model can really help, especially when we're in the relational world, not only relational, especially when you're at the business level. But that is really why we're doing all this at the end of the day, because you have to manage that complexity. What better way to make it more agile is a nice, clear picture, which is what I love about a data model. It just gives you that literal roadmap of where you're not literal. I actually figured it out of where your databases are. I can move my own slides in a someday. So speaking of the business need, I thought there's another blatant plug from this architecture paper that hopefully you'll find helpful. Actually, it talks a lot about just not just data modeling, but a lot of trends in architecture. But I thought it was interesting the amount of business-centric data models were being corrected. So over 70% are using logical and almost 70% are using conceptual, which I thought was a positive trend, which means that when people are with a business requirement or want to get that business view of data models, which is a huge part of that all-agile development lifecycle, understanding your user stories, understanding user need, folks are building them. It was a very high percent. It's also a physical data model. But maybe incorrectly, I sort of assumed that was going to be high. I was pleasantly surprised at how high the business side was, which is good. And just to curse you're not sure what I was talking about when I mentioned conceptual logical physical. And I've shown this before as well. But I think it's a nice way to kind of suss out the differences where conceptual you really are up at literally the business concept layer. This is super important and agile because you'll see kind of the pyramid, one of the things that's trying to show is the number of entities. So at the conceptual level, that can literally fit on one page even in terms of a very large organization. Super agile. What are we just talking about? We're talking about customer products, accounts, and I don't know, subscriptions. Well, is the account the same as the subscription? And could you say that's the same as the customer? You know, some of those basic terms can just be flushed out and the conceptual model can get those. And that can be hugely valuable for the rest of it. If you build out into the logical physical, you'll see the number of entities that's what that pyramid is trying to show. That gets a lot better. A huge, huge? Wow, I can speak today. Sorry. What larger? And so whereas your conceptual model might be a dozen entities, your physical could have hundreds or thousands of tables and it might implement those concepts. But even though those are a large number of entities for your enterprise-wide model or a large number of tables, you can still build those in those chunks. Again, that journey of 1,000 miles begins with a simple step, a single step. That's, you know, that as-all way of dividing it up into small sprints or small chunks of your model and don't try to do it all at once. So the other thing to remember is to speak with a wide variety of stakeholders. So I do that a lot. Sometimes I feel like a data therapist sitting there, so tell me about your data. And I'm probably one of the few people in the planet that actually wants to hear your data stories, but talk it up to being strange. But that actually fits very well with the agile manifesto. When you think of it, it's individuals and interactions over processes and tools. So data model is a great tool, but it's of no value unless you're using it to talk to people and it's of no value unless you talk to people to build that model. The whole point of it, and I am a fan of this, is customer collaboration over contact-to-contract negotiation. All right? Well, this is what the model says. The model, especially the conceptual and logical level, should model the business. So if you're not talking to business people, the model can't be right. If your business person says the model is incorrect, they may very well be right because they understand the business, right? I wouldn't necessarily think the business person's going to, you know, see how you normalize that database and the physical implementation. But if you're at that high level, great way to collaborate and talk to individuals with your model. Or at least the rules around the model, whether they want to physically see the model or not, I think a lot of people do. But as a very minimum, some of those rules can be discussed. And I actually am a fan of when I speak to those folks, I was writing down, even show a slide with bubbles like this of what people said. So I know I have sort of data on the brain and data models on the brain, but often when I see these problems, the user might not think of it, but it can be solved with a data model or data management in general. So the application isn't working right. It doesn't allow customers to have more than one email. I've mentioned that already and I'll keep going with that one because it's sort of a very easy to understand example. And I'm annoying. I know I'm annoying for a lot of reasons, but this is one of my annoying traits. I'm one of these consumers that everything I see in any business, I relate it back to the data. So for example, my insurance company, I'm trying to change my address and say that I have a mailing address, that I have a work mailing address, which is different than my home mailing address, and they can't get that right. And I actually finally wrote the support and said, you have a data management issue. I had a friend that did the same thing. He just put a car and then got an ad for the same company. He's been listening to me long enough. He said, I told them they had a data management issue. A lot of these issues of their business problems actually stem from how the database was designed. And often the business wouldn't necessarily see that, why should they? But showing them through that with a model or fixing it with a model is even better. But sometimes it very much is a data model related. What are the data structures using this application? Sometimes you're amazed what you see in organizations. I'm trying to query the warehouse, but no one will show me what the tables look like. How can I query? So what a great way, what usage for a physical data model to show. Very agile. Just let people know, communicate. So once you have those, tell a story. So agile development loves the idea of a user story. And this is a cartoon from one of my books that I published out. It's actually came from Steve Hoferman. Steve is probably stranger than me when it comes to data modeling. I think he probably does read a data modeling story for his children at night. But that whole idea of agile development is around the user story. So broken down, what is a user story? It's a very often, it's a couple of sentences. What's the small chunk of what the user is trying to do? Is clear and concise? It shouldn't be more than a few sentences. And data models are great from that. They're clear, concise, and they actually have the added benefit that they're visual, right? So you can actually see it very quickly and suss out the problems. If you've heard me before, you've probably heard me say many times that you can read a data model as a sentence. You can sort of diagram a sentence and turn it into a data model. So perfect for those clear and concise sentences. And the extra benefit of a data model for some of these agile stories is that it, by nature, is very data-centric or even database-centric. So if you're trying to solve a data problem, what better way to write a story than to use a data model for it because it has an extra benefit of being very data-focused. So here's the example I keep talking about just because it's so easy to understand, is that you can tell that user story with a data model. So it's a small snippet, but it can speak volume. So the example of the user issue, the application isn't working right. It doesn't allow my customers to have more than one email. And the developer might just sort of very quickly flesh out this diagram to say, okay, well, the current database design stores emails and attribute. Sort of by definition, that's a single-valued attribute, and that's why. It's going to overwrite the old email. Maybe perhaps a better way to do this is to sort of, it's own entity and say, not only what email the person has, maybe the type of email is at home, is at work. You can start to add other attributes there and say, you know, is it a required email, or could we have preferred contact information? Would they prefer to be contacted by email or by text? And again, a lot of these things can be sort of sussed out very quickly through a data model. And this could be it. This could be your snippet. This could be your chunk. This could be your user story. Again, it didn't take, I probably drew this in three minutes, right? If you're doing it on a whiteboard, same thing. So very helpful way to solve some of these user stories and design issues just quickly by drawing up a data model very quickly. And please avoid death by data modeling. And I've actually worked at several companies where they've gone through this in the past and just have verbatim said, I don't want to do data modeling anymore. I just don't know why. Well, because we did this, and they locked this in a room for three days. And I've seen these rooms, right? Where you have the enterprise data model sort of plastered across three walls and there's thousands of entities. And some person actually thought this was a good idea of, okay, let's just sit down for a few days and let's start by what the data type or account code is. And the poor business person just shoot me now. I don't want to do that again. Or even a technical person. I'm a fan of data modeling, but if anyone's seen me present, know that I'm rather hyper and active and do not like to sit still. So don't lock me in a room for anything. Even eating ice cream for three days, I would be bored. So the idea of having me do a data model for an hour and especially if it's focused around a problem that I care about, yeah, I'm not going to sort of die here. So think about that. And we're all human. So everyone loves their own data model. And I do this myself, maybe not always successfully. If you've worked with me and you're rolling your eyes of what's interesting to you isn't always interesting to somebody else. So what I might find interesting for our I have to realize that the other folks in the room might not. So I might, you might be thinking that right now. Can I keep it 10 minutes and just pick one, what chunk of this model is going to matter to the person that I'm talking with and is it going to solve their business problem? Is it the story that they care about? Quick case study and this literally was the words out of the sponsors mouth and was ironic about this. It was one of our best case studies of using complete integrated enterprise architecture where they had full data models, process models, capability models, data models were used to stage gates in every stage of development, right? The sponsor that started with it, he had done sort of death by modeling in the past. And he got the need for modeling, but he basically said, I don't want to do data modeling, that kind of data modeling. And other people in the company aren't as wise as me. They can't get past the painful experience to the fact that they need it. He said, I know we need it, but we're not going to call it data modeling and we're not going to do it in that way. So he literally called them blueprints and we call it data blueprint and your process blueprint. And we did it in a business-focused, agile way. We didn't use officially the agile methodology for this one, but it was the same idea of small chunks. So this was actually the R&D area. So these are literally people with PhDs and chemistry and developing new drugs to save the world. And they were doing data modeling. But how we started it was, here's your drug development processes and here's where data is touched along the way. And this problem you're having with data, like, yeah, yeah, that problem like that. And then we sort of showed how you could fix it with data through the model and they're smart enough people and they got that. And by the end, that is a Photoshop picture in the back, but this was representing a true case that these scientists actually had the data models printed on their walls and the best part of it, they had sticky notes where it needed to be added. They had pen marks to say, oh, we need to add this attribute. They got it. And it was agile in the fact that it was interactive. People were building it themselves and they were getting that feedback. And it was a great success story, but it didn't start out that way. It started out with, don't you dare make me do that again? It was painful. And again, it wasn't that data models were not helpful. It was the process in which it was done. It was not an agile way until we tried to fix that. And I'm a big fan of whiteboarding partly because I am hyper and I like to get around and put stick things on the wall, but I'm not completely alone there. I think sitting and just talking can be boring to anybody. The other nice thing is that it's not threatening. So sometimes putting it in a data modeling tool seems like, oh, I'm wrong. But throwing something on a sticky note, we're putting up ideas. And this is something that business people can understand. Technical people can understand. What are the main things we need to pick? Products and policies and agents and reasons and employees and accounts. Oh, is there a different customer and account? Or really they're the same thing for us. Get some really quick data modeling rules very quickly. It can be sort of fun. People can have real color sticky notes. And again, very low tech, but also very agile to start this way. That barrier to entry just start and iterate. And it doesn't mean that you'll never have an enterprise data model. You might get there. They're very valuable once they're done. But again, can you break them into smaller chunks or how data modeling is part of each sprint? And even a small data model like this can again show some good business rules. What's the difference between a customer and a client? Now a customer, a client has a broker. Would that just be your high net worth client? A customer has an account. Would that be the same thing really? Do we have maybe the sales person? Is that the same as a broker? All that kind of stuff. Just by showing it on the same model this way can kind of be sussed out. So again, just what is the user story? We're trying to better manage customers and their interaction with our staff. And just put a few entities on that model. You don't even show attributes at this time. And get your answers and move on. Very agile. And you may build that into a larger model, but it doesn't have to start there. So this is one way I've sort of built into an organization that was using Agile. And it's just kind of that idea of rapid development, rapid feedback, fail fast or succeed fast or you want to say that and focus on the communication and iteration. And again, I tend to be type A. I have to remind myself that all the time. And if you work in a project that may not even say that, we put it up there knowing it's wrong. And that's the beauty of it. The purpose of the model is to suss out the issues. So don't be embarrassed that there's an error. That's what we're trying to show. So first of all, what are we trying to do? That's kind of your agile story. And just pick a subject area working group. The key thing there may be different people every time. There isn't this modeling team that only works on it. Of course, there's technical people that understand data modeling, but your business stakeholders may change with each subject area. Who knows the stuff best? So I'm a fan of starting with the top-down design. It isn't always already done. But what's the scope? What are the core entities? Relationships, draft them out, whiteboard them, whatever. But then don't forget the bottom-up technical review. And often, that is where you find some of these issues. Business rules, X. But actually, in reality, the system is integrated this way. What comes up all the time is we bought a package application. That means we don't need a data model. And I slapped the person. No, I usually don't report the physical violence. But the problem with that is, yes, you didn't need to build the system and you didn't need a data model for that. But that system has inherent business rules in it. It could be just back for simplicity. That system stores a single email for every customer. And your business rule is that you want people to have more than one email and work at home and you want to email the bill. And your system isn't working because it's the way the system was designed. Can you customize that system? Could you even better look at that system before you buy it to make sure? So it doesn't mean you don't need to understand what that system does. And so often, doing that reverse engineer bottom-up review, it doesn't mean a package. That could be something you built in-house. But it'll flesh out what those business rules are, how the system's designed. You can iterate and define that. But do that with those small working groups, small sprint cycles, fix that one problem, and then move on. But then publish and communicate. And so that's the idea of sharing information across the enterprise. Because you may find that you don't want the whole enterprise, obviously, working at your data model. That would be a little unwieldy. But you do want to communicate it out with me too often. You don't hide your impression, right? And sometimes you might find something that one department said, but I think we actually have to have just one email. Because that's our primary key. We key off email. That's how we know that a customer is unique. Oh, well, let's iterate and refine that. And who needs to change? And how do we accommodate and all that? So by publishing it, you may find hidden business roles that you didn't know. So people saw it. And these are small little chunks. And rinse and repeat, as they say. These cycle keeps getting in. They might build off each other. The few entities from the first will be included in the second. And that's how you start building towards the whole. So I sort of hit this again, but I'll stress it again. A little data modeling up front prevents headaches down the road. That classic, if you have time to write, if you have time to do it again. And this is a big, you might have run it yourself, right? It's a bit of a joke, but it's not really. So we're almost done with acceptance testing and everything. Great. We're going to launch this new marketing application. Just one small question. What do you mean by customer? Which until you're in the biz, this isn't really a funny joke. And maybe it's still not a funny joke. But that is very risky. We meaning the existing customers, prospective customers, people that have bought the product, the people that have not. Right. And you might often want to skip that modeling documentation stuff because it's faster. That's not faster in the long run when you then have to rebuild it because you have the wrong difference in the board of customer was and all the rules are wrong. So I may be preaching to the converted here, but I just see this too often. We didn't have time to do it right. Well, now we're just, we're doing a year long effort to do it. So, you know, again, don't spend any year doing the planning, but at the same time, don't skip the planning. It doesn't help anybody. So this, I stole a slide from actually an implementation I had done with a very agile, very Silicon Valley pure development lifecycle shop that was completely about under the agile lifecycle development. And we're trying to add data governance on top of that. And there was actually their flavor of agile was safe, which I'm actually a big fan of. That sort of was a nice mix of engineering and agile, because for larger companies, that's sort of a nice agile approach in my mind. What they were trying to do, again, you have the data governance team working with the agile coach and the agile team. And you wouldn't think necessarily those folks up front would have a whole lot in common. They almost seemed like two extremes of the opposite coins, but it actually worked out really well, because they all both had the same common goal. And if you've heard me speaking, I'm a big fan of that, getting the motivations right. They both wanted to launch a really new, cool product that would not only, that would customize you to your experience. They also wanted to be GDPR compliant, and they wanted to do it fast because they were using agile. So one of the things they did was help whitelist privacy issues. Can we just say what people can do? What they can't do is let people move on and then integrate data into each of the agile sprints within the release cycle to do it in a minimalistic way. So literally, they just had a list of questions. If you're a product manager, you have a new product, you're going to launch a new piece of a product or a concept, are there any new data requirements? That's the only thing, that literally was the question they asked the product manager. If no, fine. They just used what's there. If yes, then maybe we need to think that out. And then this particular agile flavor, they would go through a release planning, a massive international organization. They would get everyone together and plan the release. But so we just added a couple of questions there. We have court. Maybe there is a new data requirement. How are we going to implement that? Can we all just spend a few minutes together, make sure we're doing it the same way? And we all agree that an active account is the same thing. Again, that's a very simple question. In some cases, this answered very simply, but it saves a lot of headaches down the road. This particular group would do the release planning, and then teams would prepare their own plans, and they get together to see if there was any conflict. And again, all they did was add data into that. Are there any conflicts in how you're using data and how I'm using data? If not, they move on. If so, let's just spend a few minutes. And none of these lasted a day at most. And this was a massive organization. So a day was actually fairly quick for anything. Just ask the question to move on. Are anyone else using the term differently? Just ask the question. And again, if there's an issue, we can escalate it and iterate it, but often it isn't. But at least you know before you go. And then during the agile development sprint, hopefully all of the main questions were answered ahead of time so that you don't have to be in the middle of a sprint cycle and say, what do I mean by customer? So that slows things down. But don't reinvent the wheel. If there's a standard, use it. What's more agile than that? Someone's already done my work for me. And I think most people, if they know that there's a standard that I can use or a template, I'll use it. But on the same term, in terms of being collaborative, how do I publish what I just did with others? So a step in your agile development cycle, and don't tell me you don't have enough time because you just published the schema of the database you just created. Could you spend the extra five minutes saying, this is what I mean by this column? It does not take that long. I know we're all busy. But if we're going to be in a team, please document it. Because, and people did because they started to see the collective value. Oh, right. When I needed a standard, I just used Joe's. And Joe was nice enough to document what he had. I think he'll do the same. But they're courtesy and contagious. So this actually worked fairly well. And similarly to what I just said, please avoid that. I just know what it means so everybody else doesn't have to go ahead. It just doesn't work that way. Even simple things. I've used this example in the past before. But part number. I really have to define a part number. But often there's subtleties. That used to be a component number before the acquisition. Oh, really? We use component number to mean exactly all that kind of stuff. Just document it and flush it out. Because what might be obvious to you might not be obvious to something else. Something as simple as address. Is it mailing address? Is it IP address? Is it presidential address? I don't see anything. So just take that extra two minutes to document it. When it comes to metadata management, you may know also a huge fan of metadata. I've worked with a lot of big metadata and management repositories in the market. But you don't have to go big bang to get value from metadata management. And the beauty of a data modeling tool, it has embedded metadata in it. So at one customer I work with, they say, oh, we need to build a data dictionary. And I say, no, you don't. You're already using data modeling tool X. Just publish that. Right. Light bulb goes off. I shouldn't be writing a dictionary. You've got it. If you're using it for the business metadata, you can do that. But it isn't perfect at everything. So if you do use one of the big data repositories, of course, you do get really advanced lineage. If you're using an ETL type tool like Clover or some of the bigger ones, then yes, they have lineage better than any data modeling tool. But that's almost a nice sort of handoff. You can do the high level lineage in the data model tool and use something else. And data modeling tools have the extra benefits that you're doing data modeling with them. So I'm not saying that that's always the right answer, but it's something to think about. It's sort of just enough, which is sort of the agile approach. If I've got 90% of the functionality that I can use, let's just use that and at least get the word out and move on. So something to think about there. In a summary, I had to put one more here to picture the next one. But data modeling is more important than ever. It could be agile and agile as long as you do it in the right way, a focus on your business wins. Do them in small, iterative chunks. And it is over 9% of us are using it. It's great. And now you're probably one of those pictures. As I said, I went through this odd phase of climbing at mountains and doing back bends, and I don't know why. It's clearly in my genes. But it also sums up agile. So I'm a big ski mountaineer, or at least do when I had a life and did work so hard. But this is a picture outside of Denver, Mount Toll, if there's any dendrites. And we met at the trail. We biked the trailhead and met at 4 AM. And we scaled peak. And we sort of got to the top. See if I can use my, wait a minute, where'd it go? Use my pencil. We sort of got up to the top here around 6 AM and skied down this descent. And it was awesome. And we had a lot of fun in the process. And we had enough time in the day to sit around and do funny back bends in our ski boots and do pictures. But the only way we could be that agile and have the great result is that we had planning. We sat together. It didn't take us hours. We sat at the trailhead and said, is there avalanche risk? We sat at the trailhead and said, what's our route? And we drew them out. And we sat at the trailhead and said, how's everyone feeling? Because when you do stuff like this, you've got to do it right. So I sort of think that, of course, there's alignment with ski mountaineering and enterprise data management. And then if you do plan ahead and you do the hard work, it doesn't take all day. Again, you can do it at the trailhead. You can do the fun stuff. And you see the value. And you can have a great day or a great enterprise management system. And I just had this cool picture. And I want this on my tombstone because I think it's really fun. About global data strategy, we do it for a living. If you need help, let us know. The next data modeling series coming up is on data quality, data governance, and data modeling. And if you wanted the white paper, I mentioned it is available on Dativersity.net. So at this point soon, we can open it up for questions, thoughts, ideas. Donna, thank you so much for another fabulous presentation. We've got lots of questions coming in, of course, and to answer the most commonly asked questions, just a reminder, I will be sending a follow-up email by end of day Monday to all registrants with links to the slides, links to the recording of the session, and anything else requested throughout the webinar. So diving right in, Donna, you know, what's the advantage of storing your business metadata within the physical database schema versus a separate enterprise business glossary? I think the benefit is sort of, to me, summed up in the question is that when it's integrated right with the data model, it's sort of living where the developers live, so you don't have to go to a separate place. So having the definition of customer in the same data modeling environment with the person who's building the table around customer is probably a good idea. That said, there is difference between glossary. I would almost see there is an overlap between the information and a glossary, which might be some of your core entities and attributes, but you might have other things in the glossary that exist in the data model. Some of your metrics, some of your acronyms that you might use in an organization. So they're slightly separate things, but the value of having it in the data modeling is that it's very close to the development and operational environment. Sure. And Pa, well, feel free to jump right on in anytime you have any questions. And everyone's been quiet today. I think maybe I did scare everyone with the music in the beginning. We've got another question coming in. Should diagrams of all levels of conceptual, logical, physical models show feedback loops, so we incorporate new facts discovered in the later stages into the earlier models? I would say yes. My only caveat there is just showing the right model with the right person. So I've had success showing a conceptual model on a SharePoint site or, you know, or a published business user with some definition shown. I wouldn't show a physical model to the business people. I would show the physical model to the development team. And the very reason for that is exactly what was just said, is that iterative is a huge part of agile. Get the feedback, getting, you know, fail fast if you want to call it. And that's a part of... You might think your model perfectly showed somebody else and they have a different interpretation or a different business role. So yeah, show away. And do you find in your experience and encounters with stakeholders, et cetera, that there is more resistance in incorporating data modeling and agile from the top business owners or the bottom end users' customers? I find, and Pavel might have a different opinion, but I find the biggest negative feedback is from the developers. I think often the business people that are most bought in are like, yes, how would you not do this without any design? And I would give feedback. They, of course, probably don't want to build the data model. I think developers often see it as an extra step. I think that's becoming less and less, because I think once people see the value of data design, they say we're going to be able to use somebody else's data model, they see my experience. It tends to be more from the bottom. It seems to be the pushback rather than from the top end. All righty. And everyone is really quiet today. Sanon? Yes, please. I wanted to add something to what Donna said. Like the resistance from developers. I do agree with that. And also, it might come from them actually not understanding the breadth of the problem. Like you talked about the definition of terms which sound very simple, like a part number or an address, and they might not see the business implications of not getting it right. And that's why they think they have it sort of sorted. But then when you start opening these questions, they say, maybe I haven't thought through the whole problem. So yeah, it's the lack of knowledge. Yeah, I would agree. Does conceptual and logical model evolve in sprint development or should both of these, at least conceptual model, be completed before sprint starts? I would say it depends on the use. Sometimes it can actually be a little bit of a public question. It depends what you're trying to solve. Sometimes just conceptual is not the mean. I'm trying to show you there's a different of what we mean by part number and component number, and that could be the definitions. Sometimes that logical model is needed to really show the relationships and the attributes. But I would say if you're keeping it at a small enough scope, going down to logical and even physical is a good idea. I would propose doing a smaller, smaller, I can speak, smaller conceptual and logical, physical, rather than just starting at a high level and doing too wide of a scope, if that makes sense. It really depends on the use case, I would say. Sure. And what do you think of creating conceptual logical models for the benefit of visual diagrams, but keep the metadata in an external dictionary? Either, I mean, there's kind of two flavors of the same coin. So I think, yeah, sometimes using the conceptual model is a way to suss out the difference, that there's a difference between a part number or a customer and a prospect or et cetera, et cetera. You can always use that and then publish that glossary. That's what everyone is looking at. And it's not necessarily exclusive. You could have them both. Again, they often fit data, different audiences. You could put it in code where you could put in a word document as part of a legal document. It's just one tool. So yeah, I'm not saying glossary is a bad thing or a metadata repository. I'm a huge fan. And there's often a lot of tools that help literally automate that integration. So you've done it in one place, and it'll automatically publish that for you. So there's a lot of options. There's only one tool on a toolbox. And are SQL scripts easier to transition than other scripts? I would say if you're talking, I'm not exactly sure of the context, but when you're talking data models, sort of that idea of DDL and SQL in relation instead of where it grew up. So perfect and excellent for that. You can both reverse engineer and forward and reverse engineer. That doesn't mean, especially at the conceptual level, it could be anything. I'm just talking general business data rules. And at the physical level, a lot of the data modeling tools are a lot wiser now. They can get things like JSON and XML and most document databases. It's Hadoop-hive structures and things like that. So it can be a lot more broad than just SQL. Yeah. And it's specific and she clarifies with metadata in regards to metadata. First to metadata, then if you're talking technical metadata, then I would think it's a very nice integration that you can reverse engineer from the SQL for the DDL to get to a physical data model. So yeah, it's sort of a nicer fit because it's grown up together, but it wouldn't be the only platform. I think we have time for one more question here. How do you resolve the mapping between a business-oriented concept logical model and the physical database model that can be quite radically different? They can be. So a lot of the tools you can start by, say you went top-down, but conceptual logical physical. Sometimes the logical, you can forward engineer and create the physical model directly from there. Often, as was mentioned, you don't want to do that. You might want to normalize. You might want to denormalize. There's different use cases for it. And that would be, I think, something that would, well, that would be for the developer today. I would not have a business person in that discussion unless it's kind of changed the business logic. But right, there are two different things, but they are in a lent, and you can kind of kickstart your effort by starting from the logical model. Well, thank you, Jonah, and thank you, Pavel, for joining us. Thanks to Clover ETL for sponsoring today's webinar. Both great presentations from both of you, and thanks to our attendees for being so engaged in everything we do. We just love the questions coming in and the engagement on that level, and that is all we have time for today. Again, just a reminder, I will be sending a follow-up email by end of day Monday for this presentation with links to the slides and links to the recording of this session. I will get you a link as well to the research paper, so in case you haven't seen that yet. And anything else requested, just shoot it my way, and I'll get that out as well. I hope everyone has a great day. Again, Pavel, thank you so much, and Donna, thanks as always. Thank you. You're a pleasure.