 Hello, and welcome. My name is Shannon Kemp and I'm the Chief Digital Officer of Data Diversity. We would like to thank you for joining the latest installment of the monthly Data Diversity Webinar Series, Advanced Analytics with William McKnight. Today, William will be discussing showing ROI for your analytic project. Just a couple of points to get us started. Due to the large number of people that attend these sessions, you will be muted during the webinars. For questions, we'll be collecting them by the Q&A section. If you'd like to chat with us or with each other, we certainly encourage you to do so. And just to note, the Zoom chat defaults to send to just the panelists. You may absolutely change that to network with everyone. To find and open both the Q&A and the chat sections, you can find those icons in the bottom middle of your screen for those features. And we encourage you to share highlights by your favorite social media platform using hashtag ADV analytics. And as always, we will send a follow-up email within two business days containing links to the slides, the recording of the session, and any additional information requested throughout the webinar. Now, let me introduce to our speaker for the series, William McKnight. William has advised many of the world's best known organizations his strategies form at the information management plan for leading companies in numerous industries. He is a prolific author and a popular keynote speaker and trainer. He has performed dozens of benchmarks on leading database, data lake, streaming, and data integration products. William is a leading global influencer in data warehousing and master data management. And he leads McKnight Consulting Group, which has twice placed on the Incorporated 5000 list. And with that, I will give the floor to William to get today's webinar started. Hello and welcome. Hello, Shannon and hello, everybody. I'm so glad you're here. This is a very important topic and I'm going to bring my slide on up here. There we go. And I'm excited to present it as always. I have been, this is an evergreen topic, meaning it just has been important for a long time. And it's been a topic of mine for decades now. And it is my most requested topic. Despite all the architecture things I talk about, this is the most requested topic because it's hard to find good information on this topic yet, no matter what else we do, we have to do it for the benefit of the organization. And so ROI is a great way of cutting through all the stuff and getting to the point where we're on the same page with different people that have to weigh in on whether we should do a project or not, whether we should continue a project or not, etc. Most of you have probably been in some permutation of that conversation. And I want to empower you today to do better at that conversation next time you have it, to actually bring that conversation to the table because you know it's going to have to happen. And to know how to navigate it once it's there for you. And it's not just a conversation obviously, but I want you to understand the bigger picture and that's what I'll give you here today. It's actually a pretty elongated process in most organizations. And it involves a lot of people skills and getting people to communicate and agree and things like this. It's not usually the most exciting topic that a technical professional such as myself I might add really wants to get engaged in. But nonetheless, it's important for the success of the project. And for that reason, I get involved and do what I can and built some methodology around it. Over the course of decades, once again, not only from speaking, but more importantly from practice. And so my practice, my field work has really influenced this program that I'm sharing with you. At the end of this hour, I want you to be able to navigate an analytic project justification. And we're going to hammer home the difference between project and program. And this will cut through 50% or more of the misunderstandings out there about this topic. Distinguish between tangible and intangible benefits. What's tangible to you may be intangible to somebody else. And what's intangible to you may be tangible to somebody else. It's all in where you decide to draw that line. So I'll help you with that. Presenting an itemized ROI is pretty important. None of us here can predict the future. I'll speak for myself. I can't predict the future, but I can put some stakes in the ground and say it might be this, it might be that, or it might be this other thing, you know, based upon some good information. And that's what we call an itemized ROI. I want you to be able to articulate the value of an analytics program of doing analytics for your organization. I'll have some examples to share with you. And I am sure you probably wouldn't be here today if you didn't agree that analytics were important. That analytics is important to the bottom line success of your organization and is definitely a competitive advantage. Gone are the days when we can just run our business, transact the transactions and get our products in the door and not even really put a lot of thought into that or a lot of data into that. We have to put data into just about everything these days. And that's analytics. So we'll go into that. And adapt the methodology in your program that includes ROI attainment and measurement because it's not a one off thing. It's not a one and done. It's something that it needs to be exercised on an ongoing basis because it does keep coming back around. I have encountered many situations where the information professionals have accomplished the ROI and it was such a painful process that they shut it all down hoping to never have to do that again. But it does come back around and you get more comfortable with it. These are some of the projects that I've justified over the course, help justify over the course of time. And of course, we don't really see budgets at this level one at a time. It's doled out over the course of time. And that's what I'm talking about here. These are multi-year projects, most of them anyway. Although it's definitely possible that this much can be spent in one year on one project for one company. It does happen. So these are some of the higher end projects that I've been a part of justifying. It is something that I get drawn into quite a bit just because people find out I can do it. And it's not for everybody. So it is something that I have been a part of quite a bit over the course of time. And I always am acutely aware that I'm wearing two hats when I do this. I'm wearing the architecture hat as an architect. I want it done right. I want it done scalably. I want it done with modern technology. And I want it done in an integrated fashion. And on the business side, I want it to deliver to the bottom line of the business early and often over the course of every year that it exists. What's the average lifespan of projects? Depending upon where you look for that information, it's anywhere from 10 to 20 years. And even more, if you're talking about airline reservation systems and healthcare claims processing systems and so on, there are definitely a lot of systems out there that go on well beyond 20 years, those systems that people do not want to touch and probably for a good reason. But anyway, we're looking at justifying something over the course of that kind of timeframe. Before we jump into the technical aspects of ROI and analytic programs, I want to acknowledge the strategic. And this is where people get confused, I must say. And they come back at me with, well, that's nice that ROI stuff, but I'm doing these things. I get that. I get that. There definitely are projects out there that do this. But I would say that not everything is strictly ROI or strategic. I think some things are both. I think most things are both as a matter of fact. But there's definitely projects that lean, more projects that lean heavily towards ROI. There are these strategic projects. So these are the primary mission of your project. You may or may not have to do ROI. I would encourage you to do it anyway. You never know when it might be needed. And it's a good mobilizing force for the build team and the business community at large. If you want to gain a competitive advantage, great. If you want to increase market share, develop new products or services, enter new markets, et cetera, the things that you see on here, although I would argue efficiency is more of an ROI TCO factor. But if you just kind of generally want to do these things, if these things are what's coming down from the boardroom that you need to work on, great. This may not be the end of the justification. However, there may be this plus the ROI aspects. As a matter of fact, largely it depends on who you're talking to in terms of why you're doing a given project. And there's usually a strong contingent in a company, even for projects that are doing these things primarily that want to see the ROI for it. So let's learn about that. Let's talk about workloads. First of all, and I like to use this graphic, we have to put ring fences around things. We can't say the entire organization. What's the ROI of the entire organization? Well, I suppose somebody is some roll up levels doing that. But on a project basis, it's one of many projects that the organization is doing. And how you carve up a project may or may not be how the company down the road carves up their projects. So implementing SAP, for example, that might be a project to you, but to some other party, it may be just too overwhelming to think about. And so it may be component by component or the real projects. And they'll get to the bigger project of implementing SAP globally in a longer term fashion by the accumulation of projects. The reason this is important to ROI is projects are the basis for the returns and the investments. And that's what we're going to measure. You can't measure everything. And you don't want to measure just something really teeny tiny either. On the other hand, you got to find that balance. And so a lot of us know what a project is when we see it in our organization. You know what I mean? But just keep in mind that when you are talking to some people, they may see that word as meaning something entirely different. I hope my SAP example was enough for this. But we have to put our arms around data. What data is part of it? What user groups are part of it? What kind of timelines are part of it? What kind of scope is part of it? And that will comprise an application. I'm not talking about a scrum. I'm talking about the accumulation of many of them that end up with something of meaningful value to the organization. So you might think of it as something that takes a good six months to a year to deliver, but not five years and not one week, okay? So prioritizing efforts within the organization. I'm not spending a long time on that here today, since we are going to get into efforts that we have already prioritized and decided we want to do, or at least we want to explore some more about. But these are going to be my parameters for you in terms of prioritizing efforts within your organization. Because you always have 100 things that you could be doing. And you've got to boil it down somehow. And hopefully you have a sensible way through governance. And so on the whole budgetary process that you go through, I'm sure every year, hopefully that process is great. And it considers things like this. I'm sure it does either overtly or not. But it's things like ease to do. How easy is it to do it? Can we use our current technology set? Because some organizations are really are really gun shy about adding technology, especially when it comes to things like, oh, let's add master data management, let's add streaming data, let's add a graph database, let's add a data lake, let's add data science. And a big one for today is let's add artificial intelligence. So broaching that is something that a lot of executives are wary about. So can we do it with our current technology set? Pre-requisites done first. So as you're ordering things, is there anything on that list that maybe builds to something bigger for the organization? And you want to do those first. You want to put things in the right order. So sequencing your projects is as important as determining what projects you're going to do over the course of time. And finally, the ROI, and I don't mean last, last, last but least, it's last but not least, the ROI of doing it. What does it do for the bottom line of the organization that definitely is a factor in prioritizing your efforts? It may not be the only one, but it's a factor. Okay, let's understand a few more things. Ordered benefits. This is what makes ROI difficult. And this is where I have to part ways with a lot of people out there who say it's impossible. It's not impossible. It's hard, but it's not impossible because you have these ordered benefits. First order benefits have a direct relationship to the bottom line. And that's where I think we could all agree. Yes, if I were providing first order benefits, we could all do it. And it's easy and it can be done. Right? If you were just, let's say you're building a data warehouse and you're selling subscriptions to the data warehouse externally. All right. I know that really doesn't happen that way, but let's say you were doing that. That's easy. How many subscriptions are we going to get? What are we charging for them and how much does it cost to build the data warehouse? And you have the basis for some ROI there, but none of us are doing that. We're doing more second order, third order, God forbid, fourth order benefits. We're into that. Where the second order benefits have an indirect relationship and a third order benefit has what we call a transitive relationship. It does something, it enables something that enables something else that finally produces some sales for the organization or reduction in expenses. And those are the two things that if you're going to use ROI, you have to get to one or more of those two things. Same again, more revenue or decreased expenses. Now, we're good at tasks and activities. We're good at breaking things down into scrums and building backlogs for our analytics programs. Good at that. Not so good at coming up with realistic returns. As a matter of fact, just coming up with returns of any kind is something that can be kind of difficult for most organizations. We have an overconfidence about what things are going to do and sometimes we have an underconfidence. It all depends. Sometimes both are present in the same organization. I do a lot of this, I'll call it surveying the organization in anticipation of a project to determine what the ROI is going to be to help prioritize the projects for that organization. And I find out that it returns, just returns period is hard. Realistic returns is really difficult as well. I said at the outset, I can't tell the future. I'm pretty sure you can't tell the future. So yeah, I get that. It can't be this or you're fired, right? But let's get in the ballpark of reality. And that's where maybe multiple people can help and outside influence and so on. We're not so good at the risk of things. We build our lists. We think we'll execute them. We don't consider a lot of the risks that are involved. And maybe this is good because we are optimists. And if we didn't have that optimism, it may be hard to get any project off the ground. We're sometimes not so good at costs. I'd say we're better at costs than returns by far, but we're still not so good at costs. We generally do underestimate the costs of these projects. And timelines, same story, right? How many timelines have been followed to the T that we've come up with over the years? I would say that's difficult. So let's talk about types of justification, program and project. Very important part of an analytics program. What are we doing? Are we justifying an analytics project? So somebody comes to you and says, what's the ROI of this? This thing here, this nebulous thing here that I call a project, but maybe it's not. Well, is it a standalone project? Are you building the analytics for one application? Is it the analytics for SAP? Is it the analytics for Salesforce, for Google, for our website, for our supply chain? Is it something like that that can be seen as a standalone project? And if that's the case, and it's probably a 50-50 thing where 50% of the time it's a project, 50% of the time it's a program that I'm justifying, at least I'll say. The analytics project's going to be a little bit easier because we're just looking at, I say just, we're looking at what the project is going to do for the bottom line of the organization. But a program is where we are building analytics for multiple projects. And we are storing those analytics. We're making them available. Data is a service internally. Their inclusion is hopefully based upon some sort of business governance. The goal of the program is to enable the component of the applications it supports. There's no ROI inherent in building application, excuse me, in building analytics. Well, applications too, but we're talking about analytics. There's no ROI there. There's only investment. Okay. The R comes from the use of the analytics. I think we know this, but I'm not sure that we're always on point with the counterparty when we're talking about these things. So that's why I bring it out. So what are we justifying here? An information management that's going to collect analytics, that kind of a program which will provide analytics for several projects. So you might think of this as a data lake, some sort of data science hub, a data warehouse, something where you're actually holding the analytics for multiple applications. And maybe you're doing more than just build it and they will come. Maybe you're building it and you're integrating that into multiple applications. So not talking so much here about who will do it, we're looking at the project itself and saying, what is this going to do? And is it a program that's providing analytics? So you're kind of removed from the return aspect when you are building a program or is it a project which will use analytics? And by the way, analytics is one of those terms that we haven't defined very well as an industry. And you could definitely be on different pages as you have these conversations about analytics out there. So please get on the same page as you go forward. Is it the inclusion of new projects into an existing analytics program? So you already have a, I don't know, a data warehouse up and running. It's doing a lot of analytics and you want to do more. And you want to, you're adding a new project. Let's say you're moving the supply chain to artificial intelligence. And so in doing that, you want to consider analytics inside of that project. So are you adding that to your existing data warehouse or wherever it is that you're collecting the data for that program? So these are just three bullets to help you break out. Is it a program or a project? Because it's different justification for each. So think about what it is that brought you here today or what it is that you're working on today for your organization. Is that a project or is it a program? Do you know where the returns are coming from? Or does your work and your mindset stop when you build the analytics? I built customer lifetime value. Great. I'm sure they're going to use it to benefit. And I have some vague ideas about how that will work and how it could work, but I don't go there. I'm on to the next analytic here. I'm building this for the company. Again, about a 50-50 proposition. And if you're in analytics for a long time in your career, you'll do both. Okay. I'm not going to beat us all over the head with math today. This is simple math, but you have to have some in order to understand how to do the ROI. ROI is simply return minus investment over investment. It's a concept we're all familiar with. Think about the interest or lack thereof that we get on our money in the bank. If it's what would it be today? 2% interest, and you put $100 in a year from now, it's going to be $102. So we kind of get it. So we're just trying to apply that concept on a grander scale here to these probably million-dollar-plus projects that we're working with. ROI should always be supported with a time period. So if you're saying you're giving me a 132% return, I'm going to say great if you're talking about a week or a month or even a year. But if you're talking about 10 years, not so great. Not so great. I don't want to put my money in that. ROI should be presented with assumptions and risks and be itemized by all the various components, however you want to break down your analytics. And these are just some ideas by source system, subject area, users, history data, the technology used, and whether this is the one that's flipping you over to the use of AIML. Add up the possibilities, but don't oversell it. And this is where a lot of professionals waste a lot of time because they think that when you're doing an ROI, what you want to do is you want to list 100 things that this thing is going to do for the organization. Not really develop any of them, but just have a big old list. And in my experience, that doesn't work. I'd rather have a quality list of one or two or three things that the project's going to do for the company. And by quality, I mean having some ROI associated with it than to have a list of 100. I'm not saying don't list the 100 things. I've never listed 100 things. Let me say 20 things. I'm not saying never list the 20 things, but don't waste your time developing ROI for 20 things. That's just too much. It should sell itself based upon one or two things. This is an ROI, by the way, it's used for predicting and measuring. Some of you came to this webinar because you are looking to present a project to the company that you want them to do, or you've been chartered with, hey, let's do this. What's the ROI? Something like that. But some of you already have something in motion, and either you're worried about ROI is going to be asked for, or you've already been asked for ROI, or it's been kind of implied, well, that's the ROI of that project that you're working on there, and you want a framework for doing that. So ROI is good for both. Frankly, in my experience, it's way more used for predicting than it is for measuring. Most of us don't go backwards and say, was that project successful or not? We just kind of know once it's been up and running for a year or more inside an organization, whether it's a keeper or not. But sometimes we do measure the ROI on it. And keep in mind, at that point, you're talking about there's some sunk cost involved. ROI is only about moving forward. You can't undo the past. You can't go back and say, well, we shouldn't have invested $3 million in that data science platform a year ago, because we don't have any returns from it. That is not the question. The question is, what do we do now? And we already have it up and running. It's barely working, but it's there. It's salvageable. And can where we are today, can we go forward and get some ROI out? That's really more the question. Now, from a program perspective, there's different ways you can justify analysis of the impact of operational systems on business processes, a single approach to achieving desired outcomes. Are these the things that you're looking for in your analytics program? Demonstrated proficiency in the use of relevant tools and technologies. In other words, in other words, and this is important, it's an architecture question. Our analytics program is about architecture. Are you using a single architecture? Perhaps the architecture you already have in place, and are you adding on to that, or are you going to do a one-off? If you're going to do a one-off, that's more a project. If you're going to add to what you have, that's part of the program. You're adding to the program. You're adding to, hopefully, demonstrated proficiency in the use of those tools. There's obviously some benefit to that. You're adding to the consolidation of expense streams in order to maximize efficiency, so you're getting good at this architecture. A lot of you are sitting on the question right now of, I'm doing something analytically. Should I double down on it in the next year? Should I continue to do it the way we've been doing it? Keep in mind that we're in a very dynamic era of analytics, and it's changing rapidly. Architectures I've put in place five years ago, they don't need to be blown up, but they need to be added on to with the new AI and ML capabilities in particular, but other things too. AI and ML need to be added to the architecture. Do you even have an architecture that you can, well, I'll stop there. Do you even have an architecture? Okay, going on. Do you even have an architecture that's ready for any AI or ML? Your architecture may need to be blown up in order to get to that goal. Okay, I kind of sidebarred that a little bit there, but back to the math just quickly here. Variations on the ROI theme, you might hear about payback period, which is how long it takes to get your investment back, your initial investment, and there's some art form to what's initial and what's ongoing, and I'll come back to it. Return on investment just like we're looking at R minus I over I, net present value and internal rate of return. This is all just math. This is all just easy math in Excel, but what do you think you need in order to do any of this? The hard part, that is, it's the cash flow. What's the cash flow from the project? Assuming we're doing a project now back with that hat on, we're doing a project. What is the cash flow from the project? We're okay at the investment part of it. The returns is hard. Investment can be hard too for all the reasons I've previously mentioned, but you got to have both. How much is it going to cost? What are we going to get out of it? And you don't get out of it things like, oh, we're going to improve the data quality of the organization. Oh, we're going to do this. We're going to do that. Now it has to come back to numbers, and this is where a lot of people have a hard time, but we'll hopefully empower you a little bit more as we go along here. And for each, excuse me, for each suggested business project, present multiple possible scenarios along with the odds of that scenario happening, forming a probability distribution. I like to do it this way. It's going to be the plan case. Again, we can't predict the future 100% accurately. This is going to be the worst case. Most everything goes wrong, and this is going to be the best case where little goes wrong, and everything goes right. And the project goes gangbusters, and we sell not 10% more than what we're selling today. We sell 25%, 30% more. Wouldn't that be great? It could happen. If it could happen, maybe that's your best case scenario. So sometimes I'm very much directed by the CFO office of an enterprise in terms of how to do this, how they like to see it, and that is fine. We'll roll with it. But when they don't have this, which is more than half the time, this is what I'm going to suggest. Three cases, and the possibilities of all of them will add up to you got it 100. So we can only put tangible things in here. Tangible or returns you decide to measure. And these are all key words here. Returns, you decide to measure. You can measure a lot of things. You can get your ruler out and measure quite a bit. But is it worth your time? And I tried to give you some guidelines on this earlier. And that's the second bullet here. Usually, one to two returns are reasonable to measure for each phase or each project. Intangible returns, returns you decide not to measure. And please don't tell me it's all intangible. It's not all intangible. Let's give you some examples here. These are tangible returns. And these are returns that I've worked with. You've probably worked with some of them as well. Increases in sales. That's great. I love that. That's a lot of fun. We want to increase coffee sales in this nationwide concept by 2%. And if we do that, it would weigh, weigh, weigh more than pay for this analytics program. And this is what I find, by the way, by the way, when you can increase things by a good point, percentage point that is, I just said 2%. That's going to, for a mid to large size organization, that's going to more than pay for your project. But of course, the trick is in making that happen, right? Efficiencies in processes. Reduction in inventory. You can measure all this. Automated processing, reducing returns and fraud. Those are fun. Saving procurement dollars and operational savings, squeezing out some savings that way, maybe from warehousing goods and so on and so forth. Every industry is going to be a little bit different in terms of what the tangible returns are in that industry. But everybody's selling something. So let's hone in on that first bullet for all of you. Increase in sales. What does that mean to you? Think about the company that you're in. What does that mean to you? Selling more pharmaceuticals, selling more telecommunications, selling more retail, selling more of your manufactured goods. What does it mean? And you could probably break it down more than than I just did. But intangible returns are different. These are fun. These are great. These are to be listed, but not to hang your hat on for the project most of the time. And definitely not a part of return on investment analysis. For return on investment, again, you need the cash flow and you need the returns and the investment. And then it's all math from there. But intangible returns. And these are things I hear. I hear these things from great analytics and data management professionals. But there's a gap there between this and what's going to work in the enterprise. And this is what I try to tell people that we can have this conversation because I enjoy improving customer loyalty. The customer success manager enjoys that. But at some level, somebody's going to look at it a scant and say, well, what does that mean to the bottom line? Let's say I have 2% more customer loyalty. How does that translate? Given all these things we're going to have to do to take care of those customers and keep them aboard, is it smart to do it? I mean, all of us have changed phone plans, right? And we've received offers maybe for a free month here or there. Or I'm going to move you to this level of service or better customer service or a free ball hat or what have you. But there are limits to where those communication companies will go in order to keep you. And that's what I'm talking about here. And that's what somebody is looking at. Somebody's looking at, let's do everything we possibly can to improve customer loyalty. But somebody else is going, well, within reason, within reason. And let's look at the bottom line. So these things are great, but somebody has to translate them to ROI. If you want to use them in the ROI. And I would say most of these things, you can, but a better list was the one on the prior slide. Enhanced decision making capabilities. Yeah, that's all great. But which decisions, what decisions are we going to make better? And what's that going to mean? Improved operational efficiency, by how much? What's the bottom line on that? Increased revenue and profitability. Now, that I would argue is mostly a tangible return. But if you just say it like that, and you don't say buy how much, then it's intangible. And you got to focus on something else. So that's not a great intangible return to me. Improved marketing effectiveness. Okay, great. How effective? And how many more customers are they going to translate into? And what's their customer lifetime values predicted to be, by the way? So I can understand what that revenue is going to look like. Improved marketing effectiveness include improved risk management, customer segmentation, the ability to do that. Yeah, that has a knock on effect on a lot of things, customer retention, and the grand old competitive advantage. Okay. But I would say gone are the days, especially 2023, it's not a good year to wave your hand around and say things like that, versus breaking it down into some examples. So let's do that together. Here's what you need to know. You need to know the company's discount rate. This is what they're getting on their money. This is kind of a hurdle rate for where the project needs to exceed in order to be justified, because why should I put money into analytics? When I can put it into inventory and I get a solid, you name it, 18% return, says so right here. It's been saying so for 20 years. And you want to do something else with my money. So what's that going to look like? So that's your discount rate. Usually you have to get almost to the CFO office before you can really understand what that discount rate is. Yet many times we're challenged with doing ROI and not knowing that. So that makes it difficult. You might have to do it a few ways if you don't know what it is. Duration, I say two to four years is a good duration for what I've seen work for ROI calculations. And it's not good enough usually to say, well, at the end of four years, the ROI is going to be X. You've got to break it down. You've got to show quarter by quarter what it's going to be. And you want to do this, because the first quarter is going to look pretty bad probably compared to the second compared to the third and so on. Usually these projections make the project look better and better as time goes on within limits, right? Because you get to a point maybe 10 years down the line where the project shows its age and we have to do something else, right? I've already mentioned that. We're not going 10 years out. I'm saying two to four years. And in that timeframe, things should continually get better ROI for the project. You've already recoup the initial investment. Now you're printing cash, right? You're printing cash every quarter or every half year, however you're measuring it. And that's what you're showing back to the business. In a spreadsheet, they look something like this. This is a very basic one. And again, I'm not going to hammer the math at you here today. This is really simple stuff. Let's say you're doing cost savings. You're not doing more revenue. And by the way, revenue is not good enough, is it? Because there's a cost of that revenue. And most of the time it's fairly substantial. So we want to have the profitable revenue if we're doing that. But that's, I'm getting aside. On this spreadsheet we are looking at, you have cost savings. In year one, you're not saving anything. Why are you not saving anything? Why are you not doing anything in that first year? Are you waiting? No, you're investing. You're building. And a year is a long time, by the way, but this is simplified for spreadsheet purposes here. So your hardware investment's going to be, let's just look at the year one column, $100,000, yay. So your total investment is $100,000. Now that's not realistic. You have hardware. You have other things, but I'm trying to keep it really simple. So the first year your investment looks like negative 100%. That's terrible. Doesn't get any worse than that. But things get better. They better get better, right? You better not show year one and say, can you fund this, please? You go to year two and you actually have $60,000 of cost savings. And since you already built it, you're just maintaining it at 10%, it costs you $10,000. You do the ROI on that. It's a negative 45% now cumulative speaking. Within that year is pretty good within that year. Now you get to year three. But again, okay, let me back up. Again, we're sitting at year zero as we're doing this. We're not sitting at year two saying, oh, what do we do now? Because if you're sitting at year two, you've already invested year one and year two. Now let's say you've already invested year one. So if you're going to, year two is going to look great. Go ahead and do that. For sure do that, right? But you invest $10,000, you're going to get $60,000 back. For sure do that. But if you're looking back from year zero, you haven't invested that $100,000 yet to make it possible to get the financial return in year two and year three for that matter where it gets even better, $80,000 for again, the $10,000 maintenance level of investment. Maintenance levels are pretty good, aren't they? Once you get there, probably going from $100,000 to $10,000, not really realistic. It's probably more than that. But this is an example and this yields 16.67%. And that's my cover page to this business justification ROI document, right? 16.67%. Is that good or is that bad? Well, it depends on the company. For most companies, that would be bad. That would not be good enough. But at least it's positive. And unless you have nothing else to do, you would still want to, if you had nothing else to do, you'd still want to go ahead and do this. But if you have better things, or let me put it a little bit differently, if you have executives buying into better things, then this might get kind of pushed aside, this 16.67%. And yes, a lot of it is in presentation. And this is why I get people that want to just gin up the numbers and show massive returns on investment. I say you got to keep it realistic and you got to talk about the great methodology that we're using to come up with this. And you got to let that win the day over somebody in some other department that brought a project that shows $500,000 return on investment here in year one. $5,000, excuse me, 5,000% ROI. Yes, but their methods were shaky. And you got to hope that that gets kind of shaken out of the process. Anyway, yeah, there's all that to it as well. So the ROI coming over from the prior slide, negative 100% year one, negative 45%, and then there's that 16.67%. I'm not going to get into all the math here, but the cash flow, you can see what the cash flow is just on the face of it. And then the cumulative cash flow, which is pretty important, is it starts with negative 100,000. It goes to negative 50. That's the cumulative. And it's positive 20,000 in year three. Yay, great. Finally. That's where we have broken even year three. That's too long, in my opinion, for a project to break even. But nonetheless, that's when this one broke even. The internal rate of return, again, I'll let you do the math on it. But that's 12% after year three. The net present value, the present value at a 4% discount rate, there's that discount rate. And by the way, I've kind of eroded the discount rate from the calculations that you've seen here today just for simplicity, but you might want to have to use it, might have to use it, might want to use it throughout. And anyway, that shows a positive $12,303 after year three. So it's the equivalent of somebody walking in the door in year zero and dropping $12,303 on the CEO's desk and saying, there you go, free. That's what this project is. And so again, extending that analogy a little bit, it's kind of like the CEO has a line of people outside his or her door with wads of cash in their pocket. And he only has time for a few of them to come in because he's got a lunch appointment, right? He or she's got a lunch appointment. So anyway, he's only going to let the ones in that have the most cash in the pocket. I don't know if that analogy went over great, but anyway. Worksheet probability distribution. So we're going to say the probability of ROI one, and that's your first set here, which I just showed you, that's 40%. That's your main line that you're kind of normal. That's what you really expect. In the back of your mind, that's what you really expect. However, could be worse than that. Look at ROI two. I've got that netting out to a negative 33% at the end of year three. Not good. Not good. If you have some very conservative CFO types in the organization, they might hone in on that and say, whoa, we can't do this. Look, it might happen at 30%. We might lose a third of our investment at 30%. That's kind of bad. But if you've got some more optimists in the room, they might zone in on ROI number three, which is also 30%, and that looks like 116% ROI. So depends on what the makeup of the people are, what they're going to believe, all the great stuff that went into these calculations. And I've shown you the easy part. So the harder part is really coming up, getting some people to fess up to what the returns might be. And that's not a great way of putting it. By the way, I should say you're helping them with that process. And if you're the ROI champion of this analytics effort, you might say, well, I'm sitting here, I'm just doing the analytics. And I may not be on the project team, but nonetheless, you know how to do it. You want the project done, you might have to do the ROI on it. And by the way, if you weigh out those ROIs using the probabilities at the top of the slide comes out to a 31.67%. After year three, you know, once we add those probabilities in, the thing looks a little bit better actually double as good. And 31% in my book, yeah, probably good enough for most organizations to want to do that project. So to do what we just did, you did get the discount rate, you know the revenue and the cost reduction. Yeah, that's the hard part. Know the cost to implement it. Estimate the cost for multiple scenarios that your probability distribution estimate the probabilities of each scenario and plot the results. So these are some examples that I found of companies out there using analytics and they are somewhat analogous to some of the projects that I've worked on. So Walmart used analytics to identify customer buying patterns and optimize product placement, resulting in an increase in sales. See how these all, well, most of them end up with increase in sales. The third one, increase in viewership for Netflix. Well, Netflix increases their viewership, we all know, and they know very tangibly what that means to their bottom line. So you've done something there if you can increase viewership. And the last one is a reduction in delivery times by UPS. And they know and we know that that means great stuff for the bottom line. So again, we're close to or sitting right on the bottom line in these examples. So I want you to think about your company, your industry, and how this all might work for you. By the way, if you have any questions, I'm going to have about five minutes at the end here in a few minutes for your questions. So go ahead and put them in the Q&A. And I look forward to those in a few minutes. So abstracting ROI, impossibly measurable at the detail level. So you have to measure this at the large group impact level. So something like customer lifetime value, translating that to what that means for which customers, which customers are going to have an increase in customer lifetime value? What is the decile of spend that they are in for you as a company? What does that decile do for the bottom line of the organization? And that is a barometer that you can use to drive a number to the bottom line for this calculation. I hope that hope I just made sense there. But if you're targeting something that's more, I would say abstract, okay, abstracted from the bottom line, not an increase in sales, but an increase in something like on the prior slide, the example of Netflix. Okay, that's a good one. They increase their viewership. What does that mean? Well, if you increase every 1% viewership increase that you do, however you define the 1%, that means the bottom line of, and I'm just pulling a number out of the air here, a million dollars a month. So there you go. You're almost there, but you're not quite. You have to add on to that. And that's what abstracted ROI means. That's what I mean by that. So if you're doing something like customer satisfaction index, you're trying to improve customer satisfaction, you know, that's okay. That's good. As long as you can translate, improve customer satisfaction to the bottom line. And some companies use something called a customer satisfaction index. Now, TCO is just, you can do it this way or you can do it that way. And if you do it this way, it'll cost this much. And if you do it that way, it'll cost that much. I have done innumerable TCO studies for clients in enterprise clients, as well as vendor clients. And I've published a number of those as benchmarks, by the way, out there. So if you want to see some more of this calculation in action, although you're probably not a vendor, so you can apply it to your situation. But nonetheless, you can see the calculation. They're out there. So TCO, and a lot of you are charted with things this way. You are charted to build a data warehouse or a data lake, or master data management that's on here as well, some sort of hub of data. Why would you want to do that? Why would you want to do that? These are architecture questions. There are great reasons why you'd want to do that. A lot of it comes back to it's lower TCO. It's lower TCO to the organization to do a data warehouse than to build, I don't know, 10 data marks, what have you. What you're going to have to do if you don't think about things like a data warehouse. And yet at the same time, none of us seem to be able to build one data warehouse for the entire company. I get that. But the fewer the better, I think we can all agree on that. It's just really hard. But the fewer the better, as long as they're not locked away by IT and 10 other things I could go into, which I won't today. The storage medium that you choose, the cloud that you want to use, the cloud instance that you want to use, that might be a TCO, that very well might be a TCO, master data management, whether you're upgrading or replacing something, whether you should outsource something. A lot of these are TCO questions. Total cost of ownership. What is cheaper to do ultimately. And nothing is total TCO, by the way. If you do, for example, if you do a going back, if you do a replacement instead of an upgrade, that comes with enhanced capabilities. All you're saying here is, I'm just going to measure the cost. And as far as the new capabilities, they'll come, but I'm not going to bother to measure them right now. Okay, so that makes that a total TCO example, but I'm saying that the ROI is inherent in something like that as well. You're just deciding not to measure it right now. Hope that made sense. So a summary of the methodology, you do your cost analysis, you do your tangible benefits analysis, not the intangible, that breaks it down to cash flow, where you can do, and then you add the probability distribution, the ROI calculation, list your intangible benefits, have fun, prioritize and plan things based upon ROI and other factors, and do continuous learning. It's not a one in done. Continuously get this thing better. The first time you do this, the first time is going to be hard, really hard, but it gets better. You get more practiced at it, and you get more comfortable with it. In summary, target the business deliverable of the project. Use a repeatable consistent process using governance. And I haven't hammered that home here, but definitely you want to get the business involved and you want to get the governance programs involved in what you're doing here for project justification. Use lower TCO for program justification. Let me just say that again to make sure that it comes across. Use lower TCO for program justification, justifying doing architecture A, the data mesh versus nothing. Why should you do a data mesh versus nothing? Well, it's going to save the organization cost in the long run. We could all sit here and argue, yes, it's going to increase capabilities, blah, blah, blah, but it's going to lower TCO. I'm suggesting it possibly will, okay? This isn't the data mesh talk, but anyway, look at it that way. Isolate project benefits and costs. If it's important, you can measure it and you can improve it. The project will almost always pay for itself. And a lot of projects die on the vine because nobody did the ROI for it because they found it too hard. Judgment remains essential. I'm not taking away any of your judgment today. You got to keep that in the game. Isolate project benefits and costs. If it's important, you can measure it, you can prove it. I think I just said that, sorry. And judgment is essential. That's a redundant slide. Anyway, this has been showing ROI for your analytic project. I will now turn it back to Shannon to see if you have any questions. Thank you so much for another great presentation as always. And thanks everyone for submitting your questions. If you have questions for William, feel free to submit them in the Q&A. Dan is the most commonly asked question. Just a reminder, I will send a follow-up email by end of day Monday for this webinar with links to the slides and links to the recording. So diving in here, William, sometimes a return might be the result of some other program project running in tandem with yours. So it can't be easily asserted that your project program alone moved the needle. How do you handle those types of situations? Yes, that's a great question, by the way. And this is true for so many of us. But I just say we've got to reach out to that organization, make them a partner in this, and try to get, let's say they're doing the returns, we're doing the investment, right? This is common, right? We got to get the returns out of them. We got to work together with them. And we're the ones chartered to do it, but we don't have full control. I get that. But we still got to reach out and get that. And by the way, this brings up a point I failed to mention, but a lot of times technology vendors would love to take credit for the whole project. Oh, this project that delivered X percent ROI to the bottom line, this was built on blah, blah, blah database. So look at the ROI that blah, blah, blah database provided to that company when that's not true at all. You could have used five other databases for that, you know, let's just say. So be very wary of studies like that that credit the technology so heavily with the ROI. So a project can deliver the benefit of quote unquote, save time. If so, if you save 1000 people's time at the total value of a million dollar estimated hours time labor costs per hour, but none of these people are moved out of their job and there are no tangible differences in their output. Is there any tangible benefit other than perhaps people experience? Well, I tend not to do ROI that that has a lot of that in it because of the reason that the questioner stated, it usually doesn't work out and we kind of all know this. And so why go into it with that as your as your guide? However, it's true that the that a lot of times, especially with today's projects is as they're moving into AI and ML, a lot of them are going to replace people functions. So they're going to replace people. That's a different question. But the people's functions, you know, we all have to continue to improve our skills. So we can we stay relevant to our company. We don't become redundant with the things that are going on. I think that is still a doable goal in the next few years that we all need to step up to for ourselves for our careers. Are there other things? I mean, if that's your thing, if that's the thing for this project, and of course, there's a million projects, right? So I don't know which one that we're thinking about here. But if the thing is, we're going to, you know, take people cycles out. I think that's still fair to do. Say, yeah, we're going to take people cycles out. So, you know, we're going to reapply X number of salary X percent of salary somewhere else. And here's the plan for that. And here's how that will benefit the organization, maybe more from an ROI perspective and so on. So you got to have a holistic plan. And you are right to be skeptical of that as your only benefit to an ROI. Well, William, such a great topic and so many questions. I'll get these questions over to you just so you can see the ones we didn't have a chance to get to. But that is all the time. I'm afraid that we have for this webinar. Thanks to all of our attendees for being so engaged in everything we do. Again, just a reminder, I will send a follow-up email by end of day Monday with links to the slides and links to the recording. William, thank you so much. Thank you. Thanks, everyone.