 Here we go. Hello and welcome. My name is Shannon Kemp and I'm the Chief Digital Manager of Data Diversity. We'd like to thank you for joining the latest installment of the monthly Data Diversity webinar series, Advanced Analytics with William McKnight. Sponsored today by Looker. Today William will be discussing showing ROI for your analytic project. Just a couple of points to get us started. Due to a large number of people that attend these sessions, you will be muted during the webinar. For questions, we will be collecting them by the Q&A section. Or if you'd like to tweet, we encourage you to share highlights or questions by Twitter using hashtag ADV Analytics. And if you'd like to chat with us with each other, we certainly encourage you to do so. To open the Q&A panel or the chat panel, you will find those icons in the bottom middle of your screen for those features. And as always, we will send a follow-up email within two business days containing links to the slides, the recording of this session, and any additional information requested throughout the webinar. Now, let me turn it over to Joel from Looker for a brief word from our sponsor, Joel. Hello and welcome. Great. Hey, thanks, Shannon. And I am going to show you just some quick slides that are going to lead us into what William is going to talk about around ROI. So give me just a moment here. And we'll get that up and going for you. I thought it would be really interesting to talk a little bit about a background around ROI before we get started. Share a little bit of information about Looker as well. Shannon mentioned Twitter. I'm at Joel McKelvie on Twitter if you're interested. Feel free to get there, although I won't be able to respond until after I'm done showing you this stuff. But feel free to hit me there and use the hashtag as well. Great. So just to talk a little bit about ROI, there's a reason why we do analytics. And there's a reason why we access data. And that is because companies that use data well fundamentally do better. Insights driven businesses, according to Forrester. And these are businesses that use data to find new information, to drive better decisions, to improve operations and efficiency. They just do better. They do better at scale. They grow faster. They they transform more easily and really relevant for 2020 and 2021, where we've been disrupted throughout so much of the business community, they're much more likely to succeed during disruption. So it's really key that we use data well. And as a result, we're seeing that business leaders are rebalancing their portfolio to invest more in data. So that's good for us that are those of us who are data teams. But business intelligence, data analytics, and using data is a huge place where investment is going going forward. And that is why it's so important that we understand the ROI and what William's going to share in just a little bit. As we invest, we have to make really solid use solid cases for the money that we spend. But I think that what we can find pretty quickly is that it is money well spent. So so certainly I'm looking forward to William's session. I'm sure you are as well. So what is Looker and how does it fit in this universe? So Looker is a business intelligence and data analytics tool that leverages the power of your database to deliver data to the people who need it when and where it's needed. And we certainly do BI. One of our customers, Sun Run is a classic example that's using us for self-service analytics to do things like they're a solar company. So it's for solar installs, they track it, they track their sales data. They have dashboards and reports that they send around the company and they really use Looker for comprehensive business intelligence and metrics management. But it turns out in the world of modern analytics that traditional BI reporting and dashboarding is just part of it. And certainly Sun Run knows that they've been doing a bunch of other things, but there are more ways to get more out of data. And I think that this is really important for us as data specialists to think about is integrating data, integrating insights right into where people are using data or need data. A good example here is Stripe, the financial company. Their reps have data from Looker embedded right in their Salesforce instance because really they're working in Salesforce these teams and they need it right at their fingertips. So the insights are integrated right into where these teams work. And Looker can do that as well to embed those insights right where they're needed. Further, there's a lot of workflows that could be enhanced by data, whether they're physical workflows on a manufacturing floor, whether they're virtual workflows with organizations like Salesloft, they're looking to reduce churn and automate mail campaigns and do some automation with third-party tools to make sure that data is used and consumed very, very efficiently. And so again, embedding or integrating data right into workflows is a key part of analytics today. That's something that Looker offers as well. And then the last one, which you may not be considering, but which is very important, is this idea of customizing data applications. So a lot of online applications, a lot of mobile applications really use and expose data to their users, whether it's gaming, exposing, let's say scores or leaderboards information, fitness applications, health applications, retail applications, portals for hospitality websites. These all have a lot of data on the back end. If you've ever ordered something from a retailer, you want to go look at previous orders or see how much you've spent, that's all data. And integrating that into applications can be painful because it takes a lot of work to connect to the back end data and to model the data appropriately and expose it. And so custom applications, we have a retailer that's maintaining inventory using a custom application and doing a lot of supply chain management through it. Those types of tools are also being built and are very important for data teams and can also be built on top of Looker. And Looker actually provides a robust API, powerful access to data in your database or multiple databases that gives you a lot of control over the data and lets you deliver these types of what we call data experiences. Yes, the traditional BI and analytics, but integrating insights, data driven workflows and custom data focused applications right to the people who need it, where the data can be used and can make a difference to your business and drive the ROI that William's going to talk about going forward. So why Looker is different is not just that we can fuel all those data experiences, but at its core we have a fundamentally different approach to how we make data accessible to users. We have an in database architecture which leverages the power of the database to run the queries and we can connect to multiple databases as necessary. The average Looker customers connecting to three or more databases with some with many more databases simultaneously through Looker. But we drive that in database query engine to perform the query that takes place, so we can take advantage of the power and flexibility of modern databases with their in database architecture. We have a robust semantic modeling layer which lets you model once and use many times across multiple data experiences so that you can leverage analytic experience and data knowledge very, very aggressively in a do not repeat yourself manner in a way that is get enabled and shareable and collaborative in that semantic modeling layer. So when we can drive data experiences through that semantic modeling layer as well. And then an API first extensibility. What we do, we can do via APIs. So if you've got a third party tool you're trying to automate, you're trying to connect us to another tool to develop your dashboard, whatever it is that you want to do, we can take that modeled powerful in database architecture, the data that's associated with it and deliver it as necessary wherein when you want to use it. And we have a multi-cloud approach which means whether your data is in Google Cloud or Amazon or Microsoft Azure or in all three or there and on-prem, we can connect to it. So you can form multiple database connections from Looker and access data in the cloud of your choice. We recommend that you put Looker in the cloud hosted in the cloud that is closest to the bulk of your data, so that makes a lot of sense to reduce latency and all the rest. But we really let Looker follow where you are doing data, where your choice is rather than forcing you into our ecosystem. So we're multi-cloud from the ground up and we are happy to host you in Google Cloud or in Amazon or in Microsoft Azure at this time. So I'm very excited to work with you on your multi-cloud strategy. So with that to kind of round out our part here because I know that I'm between you and William and he's got the interesting stuff going on, right? I think the key here for driving a good ROI is to make sure that people can make smarter use of data. Looker has become very popular in doing that, got lots of developers, lots of individual businesses and organizations using this. What's really compelling is that we have found that when you enable experiences outside of BI, so that people can use data outside of BI, more than half of users end up using data in these different types of experiences. So that really frees up the data to have the biggest ROI possible. And of course we're intercompatible with most of the big cloud players and supports more than 65 now different database connections if you're interested from different vendors of databases. So don't take my word for it. Feel free to go out to looker.com slash demo and get a demo yourself. So it would be really, if you're interested, I suggest you just go and take a look and judge for yourself whether or not you think that we can help you drive a lot of ROI. So with that, thank you for listening. Head out to looker.com slash demo and I'm going to hand it back to you, Shannon, and then off to William. Joel, thank you so much for kicking us off and thanks to Looker for sponsoring and help making these webinars happen. Really appreciate it as always. And if you have questions for Joel, feel free to submit the questions in the Q&A section in your screen. You can find that icon in the bottom middle of your screen for that icon. And now let me introduce to you our speaker for the series, William McKnight. William has buys many of the world's best known organizations. His strategies form the information management plan for leading companies in numerous industries. He is a prolific author and popular keynote speaker and trainer. He has performed dozens of benchmarks on leading database, data lake, streaming and data integration products. And with that, I would give the floor to William to get his presentation started. Hello and welcome. Hello. Thank you, Shannon. And thank you, Joel. It's always great to have one of our go-to products aboard here at Advanced Analytics. I trust that I am sharing the right screen, if not interrupt me, but we have a big topic today. As a matter of fact, I think this is one of the most important topics that I've ever talked about here at Advanced Analytics because no matter what else you do, no matter if you have the great architecture, you have the great tools, you know, you're doing the great execution of your projects and so on. It does come down to good old ROI many times. So we might sit here as analytic professionals and look at each other and say, well, isn't it evident that we need to do this? Isn't everybody doing it? This is the way of the future. And we're when we're in line with that thinking, I'm in line with that thinking. But if you've been in this industry long enough, you're going to run into the time when you're going to have to show ROI for your analytic project. And in my experience, it's been frequent. And it's been more often than not. As a matter of fact, we bake it right into our project methodology. And I suggest you do the same as well. So I'm going to help educate you a little bit on that process so that you can do that. And you can show ROI for your analytic project. So this is a jam-packed hour. Hopefully you don't mind. It's going to be a bit of a fire hose, but it's a big topic. And that's just how the topic falls. So I want you to be able to navigate your analytic project justification. Because even though, again, we sit around and say, well, isn't this great? And yes, we should be doing this. And look at the great things that we're doing and we've got users that are happy and all that good stuff. It's not enough for some level. And those pesky people in the C level, they ask for things like ROI. Can you believe it? And sometimes we resent them for it. But we should not do that. It's a perfectly legitimate question. It should be asked more, frankly. And we should be better at our answers. So I'm going to help you become better. Distinguish between tangible and intangible. Too often, we analytic professionals love those intangible benefits. But it's not good enough for ROI. We want to be able to present an itemized ROI because let's face it, we're not sousayers here about exactly what's going to happen in the future. So let's present some different options as to what may happen. Articulate the value of an information program. So I'm going to hammer on this program versus project business here to begin with, because you all, the hundreds of you here have come to this session with that many ideas about what ROI is. And so I'm going to help you understand at least what some other people might think about and adapt that methodology. So I've been running a consulting practice now for over 25 years. So you can believe that I have gone through the justification of big projects many, many times over. And frequently, I come aboard when there's a lot of early days, when there's a lot of skepticism and very vague ideas as to what may or may not be able to happen in the organization and so on and try to take that all the way through. Now, you might be saying, well, we're already doing an analytic project, William, we don't have to justify it, we're already doing it. Okay, that's fine. The day may come. And maybe the day is here today for you, where you're asked to state what the ROI is in order to get a future budget or maybe in order to get a promotion or some such thing like that. So this is actually a quite important topic for all of us. And I do find that in those days, like 2020 and still here today, 2021, when things are tighter out there, it's asked for more. Okay, so I'm going to try to help you wear both hats here. I know we have to, we have to do the right thing. And we have to provide this ROI, but we can't just slam an ROI in there on the business that's going to go away tomorrow because we failed with our architecture hat. So I am very well aware that we must wear both hats at the same time. Are you prepared to wear both hats at the same time? You must be, somebody must be, that's going to helm up this justification. Well, quote here, I've dragged around for quite a while here. And it's just about the importance of the firm's measurement system. All right, let's acknowledge the strategic. First of all, before we jump into the calculations and the details, I get that things are strategic. I love to work on strategic projects. I love it when the client says, hey, we've got unlimited money. And let's do things that may or may not work and maybe break into some new markets and so on. But we're not all Amazon. We're not all Google. We're not all in those places. But the strategic is a place for learning and innovation. I get that. There is the unknown upside where we kids can't even begin to predict what a new project may, where it may go in the market. And it's the intuitive thinking employed by the highly paid professionals, the highest paid person's opinions in the room. And so sometimes we just go that way without doing ROI calculation and just burn the bridges and just go for it. Okay, I get all that. And that does happen. But more often than not, a analytic professional such as the people that come to this type of webinar are usually under at least a little bit of pressure to provide some ROI. Now, what's important here to begin with is to divide our work up into workloads. Yeah, we're all familiar with the term workloads. But we have to divide and conquer when it comes to ROI. You're not asked to do the ROI for the entire organization. If you work at PepsiCo, nobody's asking you, what's the ROI of PepsiCo? No, it's for a project. And we have to put bounds around that project. I'm not going to belabor this point, but I am going to call it out as something very important in doing this because you have to match returns with investment. It's not fair to gobble up all the returns for things that are not included in the investment side of ROI. And I have worked with many a very wise CFO or somebody in the CFO office that will point this out right away. And you start to lose credibility when you do, shall we say, sideways, things like that. So let's be sure we come in straight. Let's be sure that we are impenetrable in terms of what we're doing. Now, ROI is useful for many things, but it is important to me for building out a roadmap for a client because it's simple. What's easy to do? Okay, that's important. What are prerequisites for other things? That's obviously pretty important to get in the right sequence. And then what's the ROI? Because I want to hit on some things that are going to produce the most ROI. Don't we all? Shouldn't we all be wanting to produce the highest ROI we possibly can for the organization? I think so. As a matter of fact, if we're not working on that project and nobody else is working on that project that's going to produce the biggest ROI, then we ought to drop the project that we're working on and go work on that project that is going to produce the biggest ROI. Okay, let's talk about ordered benefits. And this is what makes it difficult to do ROI. Benefits can be direct or indirect. And since we're not exactly building, let's pick on a artifact, a data warehouse, we're not exactly building a data warehouse that we're selling subscriptions to usually. That's a very rare scenario. And if that is your scenario, you got it easy for ROI. How many subscriptions are you going to sell to the data warehouse? And what's the cost of building? Okay, that's easy, but we don't do easy things. We do things that have second order, third order, fourth order, et cetera, benefits. And so we have to trace the chain of benefits all the way back to the point at which somebody is selling something. PepsiCo is selling more Pepsi. The department store is selling more socks. The insurance company is selling more insurance. Okay, that's how we need to be thinking if we're going to do ROI. As a matter of fact, when I go to a new client, I like to start asking people, so where's the ROI from this project coming from? And I asked maybe 10 people in the project and I'll get so many different responses. And it's kind of that's kind of a symptom of no wonder that we're not all on the same page rowing that boat in cadence. All right, so why don't you ask yourself and your teammates, how are we producing ROI here with this project that we're working on? Are we trying to sell more insurance? Okay, I was at Dunkin' Donuts. Are we trying to sell more coffee? Are we trying to sell more donuts? Are we trying to sell more out of each of the stores? And we're not so worried about the product mix. Are we trying to create a less costly supply chain, et cetera? Okay, let's get the right answer. Are we trying to start at, are we trying to improve income? Are we trying to reduce expenses? Because it all comes down to that when we're talking about ROI. Now historically, and this is when I get involved because we typically tend to think we're good across the board at this stuff until we start rolling up our sleeves and trying to do it, we find out it's really hard. This is hard stuff. But once you kind of get it down and you've done it a few times, it becomes like anything else, a little bit easier. We're good at the tasks and activities. We're okay at it. I'll say what are the activities that are required to build out an analytics platform? There are accelerators out there. There are experienced consultants out there. There are experienced people inside the organization out there. There are patterns that work within organizations. And we can kind of picture what the final outcome is for a lot of our activities. But what we're bad at is what are the returns? You know who's bad at returns? Everybody. We're bad in the builds on the build side of things and the business is kind of bad on the business side of things. And as a matter of fact, they try to duck and dodge the question just like we do. Just like we do because we just want the project to be justified. We don't want to deal with all this stuff. But I'm telling you, you may have to deal with it. You may be in that seat right now. And you may hate this stuff. I mean, personally, I like the architecture stuff better myself. I like the technology. I like making things happen. But this is part of the deal, I would say. Part of the deal for an analytic professional, at least anyone that wants to kind of make a real mark for themselves in this space. You got to know how to do this. We're bad at risk. We tend to not take it into consideration when we do our ROIs. We're bad at costs. We used to be good at costs. We used to be okay at it, I'd say. But now you think, oh, it's all cleaned up with the cloud, right? No, it's not. Because the cloud with all of its flexible spending options by usage and so on, those costs have tended to be higher than many shops think it's going to be. And the usage just tend to be higher. Well, I guess you hope that's a good thing, right? You hope the usage is translating somehow to ROI. But nonetheless, we're bad at costs. We're bad at timelines to, I don't think I need to belabor that. I think if you've been around this industry for a while, you know that a lot of timelines are missed, unfortunately. So that's historically what we're good at. We got to talk about the difference between a program and a project. Because again, we have different opinions of what it is that we're justifying. It's very important to hammer this home. All right, let's understand this. Only a project has straightforward ROI with a set of costs, say to build an analytics platform. Let's say you're putting in a new Synapse database with a looker on it. And you're bringing aboard a talent for the ETL. And you have your machine learning portfolio with you. You have your streaming. You're buying a data catalog for it, et cetera. Stuff I've covered on other sessions of advanced analytics, all right? So that's your costs. And how are you getting returns? And it's not easy, but you can kind of figure that out. But how about a program? Maybe you're thinking, well, you know what? I really don't know what the business is going to do with it. I'm justifying a program. I'm justifying a program approach. Now, when somebody tells me they're justifying a program approach of things or the right thing, the right way to do things, you know, I think about, well, you're trying to build a data warehouse instead of data marks, okay? You're trying to build a data lake instead of a bunch of one-off, non-congruent data clusters. I totally get that. You're trying to build master data management instead of everybody go roll their own master data. I get that completely. Maybe the approach there is more of a TCO type approach. Like, if you do it this way, it's going to cost this much. If you do it this way, it's going to cost this much. And you'll have plus or minus these kinds of benefits if you do it this way. The goal of a program is to enable the component of the application as supports. It all comes back to applications or projects. I probably shouldn't have used them interchangeably here, but it all comes back to that. It all comes back to those things that are put in place within organizations that hopefully are driving hard at sales or expenses. So, what's being justified? Now that you are starting maybe, hopefully, to wrap into this idea of program versus project, are you trying to justify an information management or analytics program which will store data for several projects? A sample question there is, why use the data warehouse architecture versus independent data markets? That's a vastly different question from should we do targeted marketing. That's a vastly different question from how do we optimize the supply chain or how do we do fraud detection? When you are doing projects like the three I just mentioned and the 25 other ones that you're doing in your organization, when you're doing projects, the artifacts become just support for those projects. Now, we're builders of data warehouses. We're builders of data lakes. We're builders of data hubs and MDM and blah, blah, blah, right? That's mostly the audience to this webinar. We build this stuff, but we don't exactly use it. So, sometimes we don't know how the projects translate into the bottom line of the organization. However, that doesn't get us off the hook for doing ROI for the entire project because most of these projects today are technology, not only are they technology based, they're technology dominant. It's the technology that what those projects are all about. So, now get out your protractors. I trust you all have brought your protractors to the session and get out your scientific notation calculators, which I think I put in the pre-rex. Everybody needs to bring to this. I'm just kidding, of course, but there is some math involved here, right? Return minus investment over investment, that's your simple formula. ROI should always be supported with a time period because if you tell me I'm going to get 10% return on my money, I'm happy if that's one day. I'm not so happy if it's over 10 years, right? So, we have to put the time period on there as well. ROI should be presented with assumptions and risks and be itemized by however you want to break it down. However, it makes sense to you to break down that analytics project. That way, the decision makers have something to decide. Oh, yeah, let's do it this way. It looks like this way will produce the most ROI with the least risk, blah, blah, blah. That's the kind of response that we want to generate with doing ROI for justification. Add the possibilities, but don't oversell. Don't get crazy with this. If anybody says to me, well, we've done the ROI and we're going to get 10,000% return next year, I'm like, I shouldn't say like, I'm saying that is way too much. You are overselling. You may believe it and we can always shoot for the moon here, but let's be realistic so that we are perceived as realistic. Again, we use ROI for predicting and measuring. Now, if you're doing a program, again, back to project versus program, can't get away from that. If you're doing a program, program helps because there's less impact on operational systems. We're not pulling data all the time from a source system to stored here, stored there, stored over there. We're not taxing out that operational system, which might impact its running, number one, but it also might be very costly to an organization. One way of doing things, there is merit to that. Tools, competence, there is merit to that. Consolidating expense streams, there is merit to that. Getting this notion of enterprise subject areas. Master of data management is kind of pure about this, right? We're building the one customer master for the entire organization. Well, if you think about it, the program approach is really about that, no matter what artifact you're using. As a matter of fact, let me jump to the next slide. If you're doing a data warehouse, you could justify that with ROI or TCO. Think about it. Some people, some shops will confuse, I guess, the data warehouse versus the applications. Let me put it this way. No organization needs a data warehouse. What they need are the things that the data warehouse enables for that organization. That's how we got to think about it. The target of marketing, the supply chain management, the customer management, the fraud detection, the IoT operation, et cetera. What are you justifying? The idea of doing this as a data warehouse? If you're doing an analytic data store for one project, to me, that's not a data warehouse. That's not shared. That gets wrapped into the justification for the project. If you're saying, well, let me look at the roadmap here for the organization, there's 10 projects coming up. They're all going to need this, that, and the other data. They all could go roll their own and be incongruent. It could cost, let's say, a million dollars each one to do it. It also could highly risk out those projects, or we could do it. We could build a data warehouse, build it once, use it many times. That's a TCO question. In larger organizations, a lot of times the builders do not have tremendous visibility into the usage. They just know that so-and-so needs, so-and-so data. Project B and C and D, they all need the customer data or the sales data, et cetera, and they seize upon that opportunity to build it once, use it many times. You can see down the list here what some of the preferred approaches might be. Again, you'll get the slides later, so you can come back to it. I'm going to go through some of these for you quickly, but it's not that difficult, really. Once you break it down into cash flow, I'm going to say that again. Once you break it down into cash flow, now I see the hairs are standing up on end out there, and people are saying, William, we shouldn't have to break this down into cash flow. It's just so good for the organization. It should be evident. Again, I'm back to that conversation that I had with you earlier. Yeah, I agree. I get that, but at some level, that doesn't fly. At some level, the CFO is not going to dig into analytics .com or dataversity.net and do all that reading and figure out, oh, yeah, it looks like this is a good strategic thing to do. They're going to ask the question about ROI. These are the four basic variations on the ROI theme. Now, my suggestion is let's not be Nostradamus about it here. Let's present at least three possible scenarios with how we present this. If I'm asked to justify a project, I'm going to say, yeah, this is the best case. A little goes wrong, and I'm not trying to, by the way, I'm not trying to pump the ball here and say, well, I won't, I'll just leave it all be up to you. I suppose you can look at this and say you're risk sharing here, and I accept that. I am, but I don't have all the answers. I do want to bring the sponsor, if it's a sponsor. I want to bring the sponsor into the decision-making process, and I want to show that I'm not here saying I can precisely predict exactly what's going to happen here. There is a worst case, and usually my worst case is that I show our negative ROI. We just, you know, it didn't work, and we just wasted $3 million, $4 million, you know, that could happen, and here's why it could happen. And by the way, that process of going through the why it could happen helps you, because then you start to take care of that, and make sure it doesn't happen, so that you get a little closer to the best case. All right, now let's talk about tangible versus intangible metrics, because again, I find a lot of billed professionals think about the intangible, but at some level that doesn't work. Again, if you're going to break it down to cash flow, you need something tangible. Now, there's a fine line between tangible and intangible. Some people like to say, well, it's hard and fast. No, it's not hard and fast, it's what do you choose to measure? What do you choose to measure? Do you choose to measure the fact that I actually had this scenario about 15 years ago, that we were going to build a data warehouse because it was going to save on paper costs, because everybody was printing out their report, and they couldn't just go to a screen and get the report. Kind of ridiculous in my opinion, kind of petty, but that is how that went down and that helped. It wasn't the only factor in justifying it, but it helped. And it turns out, it wasn't as petty as I thought, but still, but still, they chose to measure that. I would usually not choose to measure something like that, but you could say that that's intangible. If you choose not to measure it, it's tangible. If you choose to measure it. So these are some hard, tangible returns, increases in sales. Yeah, we like that. Now, you can only take back into ROI in my opinion. You can only take back into ROI. You have to take out the cost of sales, right? Because if we sell a t-shirt for 20 bucks at our store, that doesn't mean that 20 bucks went straight to the bottom line. Efficiencies in processes, reduction in inventory carrying costs, automated processing, reduction in returns and fraud. Interesting. Reduction in returns, reduction in fraud. That's a big one. Well, fraud has a cost. And if we can reduce the probability of that cost, that is a straight to the bottom line savings, procurement savings, savings in inventory holding costs, operational savings. Now, intangible. And we tend to throw these terms around. Speed to solutions. Don't we all want that? Well, how much faster and what does that mean? Clean data. Okay, you have clean data. But what's the downside of not having clean data? Why do we need that? The single version of the truth. Same thing with all of this. Faster response to customers. You might say, well, that's important for strategic purposes. Oh, yeah, it is. I don't doubt it. But can we translate that to dollars? Et cetera. So now we've got our cash flow. And I just summed it up there, but it's a lot of work, as I'll say again. Let's get into some examples. Now, what do you need to know? You need to know the discount rate because money has a differing value over the course of time. If you're going to give me a dollar today, I'm much happier than if you're going to give me that dollar a year from now, because it's going to be worthless. That's just the way of things. And so the difference there, if I were to take, let's say, if you were to say, well, I'll give you a dollar in, I don't know, pick a number, 15 cents a year from now, would I take that versus the dollar today? I'm like, maybe yes. So that would mean if I'm like on the fence, I shouldn't say like, sorry, if I'm on the fence, 15% is my discount rate. Organizations have a discount rate. CFO offices are usually well tuned into what that is. And you look really nice by asking about that and making that into your calculations. Your calculations look smart when you do this duration, two to four years, forget about the, oh, 10 years down the road, 15 years down the road, that tends to not fly. So I usually cap my ROI analysis at three years. Time blocks, minimum a quarter. Some of you want to break this down to month by month ROI. I just think that's too much. I think that's a really ridiculous level of granularity that's very hard. And so I bounce it up to a quarter or a half a year. I prefer a half a year. Half a year, three years duration, you're golden. Now here's one doing it at a full year. And this is a basic example here. Cost savings. You're going to save this company $140,000. How are you going to do it? I don't know. Let's say it's fraud detection. Let's say it's, yeah, let's just leave it at that. It's fraud detection. So you're going to save on this fraud. Year one, no savings at all. Hmm, why? Well, you're building it. It takes you a year. Again, these are just round numbers to kind of weight into this, right? Year two, you save $60,000. Year three, you save $80,000. So you're breaking even at the end of year three. That's actually an ROI measurement right there. When do we break even? The sooner the better, obviously. And you see your financial return there, and your investment. Your investment in getting to this cost savings was $120,000 in hardware, and that's going to break down to 100,000 in year one, 10,000 in year two, 10,000 in year three, et cetera. So it's a 10% caring cost or a maintenance fee, if you will, on that. So you do the math, return my investment over investment. Year one, you're in the whole 100%. You can't get worse than that. And the year two, it still doesn't look good. Year three, it's looking good. If you can go out that far, and again, I say you don't go out any further than this. If you can't justify itself in three years, heck, if it can't justify itself in one year, projects shouldn't be done in most cases when it's an ROI related project. Now, you may debate that, but find out what the culture is like in your organization. What is the aptitude? What am I trying to say? What is the aptitude for long ROIs? I find increasingly, it's shorter and shorter. It's down to about one year now in general. So now we did the ROI, so I just bring that over to this one. See what the cash flow is? You see the cumulative cash flow. You're actually 20,000 in the green after that third year. Your internal rate of return, it's simple math. I won't go through it. It's an Excel function. Net present value, again, an Excel function. Not hard once you get the cash flow. All this is easy once you get the cash flow. Now, you set it up to where there's three probabilities. And I say probability one, again, best case to worst case. Probability one, 40%, that's going to happen. Probability two, 30, and then 30 for probability three. That adds up to 100. It's not a coincidence. They need to add up to 100. So after year three, and I won't go through all this, but after year three, the probability of ROI one is 16.67%, as I already showed you. ROI two, well, that looks like it's the worst case, doesn't it? We're still at negative 33% after three years. But three, we're knocking it out of the park. That's a massive 116% return. Now, most organizations would go for that. But most organizations would not go for the negative 33. Now, if they're really looking at ROI, they would not go for that. There's probably other projects or maybe there's a reshaping, reformulating of this project that could happen that would make it more of a positive ROI. And this is where you get to sharpen the pencil and make that project work for the organization. So to use a worksheet, you can build this yourself. It's easy. You just got to get the discount rate, know the revenue generation, or cost reduction, and the work cost reduction by year for the next three years. Now, that's in years. You might do half years. You might do quarters. Again, figure out what the culture is. And I am giving you ideas as to what to do if you can't get the answers. So I say do it by half year. I say do it out three years. Discount rate. You can maybe Google on that and kind of figure something out if you can't get that number. And then it'll get shaped as time goes on in this calculation because it's not a one and done. It's never really been a one and done because once they see that you're serious, you're actually doing this. You were asked for ROI for that analytics project. You actually didn't. You know how many people would just turn away and say, well, they don't want to do it. This company, they want that ROI stuff. I'll never do that. So they don't want this analytics project. Most people will turn away. I won't turn away from that. And you shouldn't turn away from that because that's a very normal thing to do. And then the can gets kicked down the road and it just becomes harder for that organization to be successful the longer they delay things like, well, analytics in 2021. Know the cost implement. And that's five words, but that's a mouthful. And it comes down to hardware or cloud software. People costs. I've done extensive research on this recently published. Some extensive research on this. Azure versus GCP versus AWS versus some other stack. And just what the costs were to carry all that technology over the course of years. Estimate the costs and benefits for three scenarios. I'm just reviewing what I just showed you. Estimate the probability of each scenario. And I just mentioned where your costs come from. Now I have some other examples, but I'm not going to belabor them. But I'll show you, this is fraud reduction. I actually kind of labeled the other one fraud reduction too. That's okay. It's the idea that you are doing something generous to the bottom line of the organization one way or the other. You're saving on expenses like fraud is an expense. Or you are increasing revenue. Okay. So year one, again, I show zero. We are not, we are just investing in year one. It takes a year to get to the first outcome of this. Now, I don't like that. I don't think it should take that long. I'm not going to demand that the ROI be very high in year one, but it should be positive. It should be positive. Here it's negative 100%. That's terrible. We need to do better than that out there today. Anyway, you see the breakdown here, the financial return over the course of three years is 1.25 million to the bottom line. That sounds good. Let's get into the expenses. The expenses add up here, 105,000 for hardware, 144,000 for software, consulting, 50,000. That might be a little askew. Consulting tends to be equal parts to hardware and software in these kinds of projects, but whatever, the investment is 299,000. That's not much investment for 1.25 million return. And that shows up as a 90% return on investment after year two and a 318% return on investment after year three. You get the gist of this? See what I'm doing? All right. Here's another example, and I won't belabor this either, but this is healthcare. Some of you are in healthcare. You're saving on claim. I did this project, not with these numbers exactly, but I did this project. So we're trying to both add revenue by adding customers and reduce costs at the same time with this project. That's okay. That's actually great. So on the revenue generating side, how did this project, now this project was to reroute claims to best of breed providers. So if you're having a C-section, you're having an ACL repair, etc., you go to U.S. a patient, go to the, quote, unquote, better physicians for such things. And that means better results. That means fewer claims down the road. And that's what I'm trying to capture in cost reduction here. We're saving on claims. What's the average cost of a claim? Well, 25,000. So do the math. Total impact of just saving on claims is going to be, in one year, year three, 4.5 million from this project. And we're going to, at the same time, attract customers. Customers like that they're going to better doctors for what they need to do. And that's something that we can certainly advertise if we're doing it. And we're adding, say, 30 customers in year three. Now, that doesn't sound like a big number, but we're talking about corporation, big corporations here. And the average customer premium, etc., the gross profit from the customer premium, again, it's not fair to take the whole premium to the bottom line, etc. And here we have a breakdown of that. And here are some other examples. Increase revenue per customer, increase customer acquisition, blah, blah, blah, reduced IT spend in supporting databases, more of a TCO thing. Reduce likelihood of regulatory fines. What is the probability that you're going to have a regulatory fine today if you just keep doing what you're doing? Versus, if you do project X, what will that drop it to? Well, what's the cost if we actually were to have a regulatory fine? Oh, it's about 20 million dollars or whatever. So is it worth it? Okay, that's how these things are thought of at some level. Abstracted ROI. Let's say it's hard to get to that ROI. We're not doing some of the things like I just showed you. Impossibly measurable at the individual level, must measure at the large group level. Stuff like, well, what if we improve the customer satisfaction? Or my mind's drawing a blank, but what's the net promoter score? Okay, what if we improve that? Let's say, for example, for our customers, if we target that improving the net promoter score, how does that affect our bottom line? Well, customers are happier. They stay longer. They tell their friends, etc., etc. So what does that mean? Sometimes you have to do it that way. You can't think, well, what's the impact of this going to be on Joe Smith? And how much do we make off of him, by the way? Okay, now let's go on to Mary Jones. Okay, not possible. Not possible. So at the large group impact level, you're doing things like that. All right. Now TCO, those of you in the program category of things doing TCO, should I use data, warehouse or data Mars? Maybe these are the questions that you are trying to address right now. And I say go the TCO route for these questions. Now you might be diverted over to ROI, and that's okay too, but this is where I'm starting. Is the cloud worth it? Should I use a big data solution or a relational database? Hmm. I think TCO is part of that today. Is master data management worth it? Why don't we just let everybody roll their own customer, roll their own product, not bother with a risky master data management project? Well, you can do the math on that. And that is a TCO example because we're not including taking it all the way to we're going to sell more stocks or we're going to reduce X expenses in our supply chain. Okay. Maybe these are some of the questions that you are asking. And by the way, some of these are questions that I have addressed in full sessions of advanced analytics in the past and in the future because this is evolving, right? So generally the cloud is worth it, but you have to be able to do the math and show it, right? So a framework for that is important. I've just given you the framework that some people need in your organization. You might say, well, I don't know anybody like that. I don't know anybody that needs that. Well, maybe your boss does and maybe this is partly why projects are getting stalled out or you're not getting the credibility that you need within your department because you're not showing ROI, but the guy over there is showing ROI for his project, even though if you did yours, it would be more than his, but you're not doing it. So hopefully you've gotten some tools here today, some ideas as a summary, ROI methodology. You can do cost analysis, tangible benefits analysis, cash flow analysis, do a probability distribution. Okay. Again, we're not soothsayers here, but we must have an idea of where these projects are going. Do an intangible benefits analysis. I know I said poo poo on the intangible benefits, but they're still important and they're a nice addition. So when I'm doing a project justification document or project, yeah, of course, I'm going to do exactly what I showed you here. I'm going to show the ROI and that's going to be like 30 pages of assumptions and so on, but then I'm going to throw in, hey, don't forget these are the intangible benefits. You get that single version of the truth, that warm and fuzzy clean data stuff. That might resonate with somebody and I'm not ignoring it, but really at some level, they don't care, but throw it on in there anyway. Just don't try to do ROI on it. Prioritization and planning decisions. So again, getting back to how we do projects in organizations. One of the things is what's the ROI on the project? Higher ROI should mean higher priority. So how do you know until you do it? All right. So in summary of the presentation, target the business deliverable of the project. Use a repeatable consistent process using governance for project justification. And by the way, I've given you a process here today. It may or may not fly exactly how it is. Within your organization. That's okay. That's okay. I flex this. I can't tell you how many times to make it work within organizations and you'll have to as well. ROI is an important component of justification, but I'm acknowledging it's not the only one. There's still that strategic out there which I talked about. Use the fact that a program drives to lower TCO. A data warehouse is better than going data Mars for that reason. Master data management is better. Et cetera. Isolate project benefits and costs. Remember my sheep? All right. I had put fences around the sheep so I could do some isolation. Make sure that you articulate very well what your isolation level is going to be. What your application really is. Put those fences around it. Make sure people understand these are the fences. These are the bounds I'm putting around this justification. Because if you are doing that wrong, if people are thinking something different, that could drop a bomb on your whole ROI. If it's important, you can measure it and you can improve it. The project will almost always pay for itself. Yeah. If it's important, you can measure it and improve it. But if you can't do those things, and again, many organizations, people building this stuff, they walk around not even knowing. Are they trying to improve revenue or save on expenses? That's essential. Judgment is essential and I'm not trying to remove that at all. Not one iota. I'm trying to improve your abilities to have great judgment really here with advanced analytics, but especially today. Judgment is essential. I'm leaving that in the equation for anything and everything that you do. That is so important. That's why experience matters. That'll be the conclusion of this part. I saw that quite a few questions were coming in. I'm going to turn it back to Shannon to fire away Joel and I, those questions. Awesome. Thank you, William. Thank you, Joel, for these great presentations and I am just going to dive in here to get, as there's a lot of great questions coming in. How do you account for diminishing window of opportunity as your competitors also start advanced analytics programs? Well, that's outside the bounds of this now, isn't it? Because that's saying that there are reasons we want to do this project that don't have to do with ROI, that have to do with keeping up with the Joneses, if you will, but keeping up with the market, keeping up with the technology that's important. I say that's an intangible benefit that you still need to work on. You need to take that another level and say, well, if we wait, here's what the scenario is going to look like and here's going to be the bottom line ROI. But if we act now, here's what the ROI is going to be. Here are some opportunities that we can capitalize on immediately the sooner we do this. I had one client, they were a home video organization and we were doing master data management for their videos and they were able to, when there was a market opportunity, let's say for example, a famous actor, one in Emmy or I don't know what is it, an Oscar. One in Oscar, there's a mad rush to, well, there used to be, now it's all digital, but there was a mad rush to rent all the videos for that actor. And if they couldn't agree what the videos were for the actor, then they would miss that opportunity to a competitor or to nothing. And so they were able to measure the ROI of our activity by saying, well, we would get way more rentals immediately if we actually, as soon as we do this. And so let's do it now instead of three years down the road and get that opportunity. So I'd say it's really a matter of continuing to work on that equation. All right, Joel, anything you want to add? You're muted. I would say accounting for a competitor activity is always hard when you're doing financial modeling. It's non-trivial and it's very specific to your industry. I've seen it modeled as a standard discount rate over time where you say, this will have a lower impact in differentiation. But what's interesting is that ROI is not a modeling of your impact compared to a competitor. It's really a modeling of the impact to your businesses. And so competition will be included in your estimates of how much sales are increased or how more efficient you become. And it'll be included there natively. You probably don't need to compare it externally unless that's how you're justifying it to your CFO or whomever at which point it probably benefits you to not include it in the calculation but call it out as a sort of a separate part of your presentation or your pitch. Love it. So how do you justify an analytical solution that will increase sales or decrease costs? Well, exactly how we showed you with ROI. That's great. I mean, that's if your analytical solution is increasing sales, let's say, well, how much sales are being increased? A million dollars in a year, whatever. What's the cost of those sales? 500,000? Okay, you're increasing by 500,000. How much does it cost to build the analytics solution? 300,000. Okay, there's a 200,000 to the bottom line. That's a positive, I think, 66% return. Whatever like that. That is actually a very healthy return. Of course, I just made up the numbers. But if you're doing something great for the organization, I kind of left this out of presentation. Let me just slip it in here. If you're doing something great for the organization, you know you are, get the thing justified. However, you need to. And I'm giving you tools to do that with, but it's incumbent upon you to justify projects that are great for the organization, not to hold those projects close to the best. So get out there and get them justified. Yeah, you know, William, I think that's a really great point. I read the question a little bit differently, which is how can you say that better access to data or better analytics will actually cause an increase in sales or decrease costs? And that's really going to be specific to your business, right? But I think that we can all sort of intuit that, for example, if I know that an analytic solution will be able to suggest a subsequent purchase to someone who buys something online, that it can probably drive some business. And it might make sense if you're really stuck trying to say, well, will something like that drive business is to do it maybe manually at first and see whether or not you actually do impact business or do some lightweight experimentation before you go for a full project. But I think that we're all, as data teams, we're pretty sure that we can use data to drive the business forward. And if we are, then I believe that there is actual data you can find that will show the justification for whatever your ROI is, increase sales or decrease costs, the higher efficiency, whatever it might be. Well, thank you both. But I'm afraid that does bring us to the top of the hour. So many great questions here still. I'll get these questions over to everybody. So we can get you guys some additional answers there as well. And just to answer the most commonly asked questions, just a reminder, I will send a follow-up email by end of Monday for this presentation with links to the slides and links to the recording. Thank you both so much for these great presentations. Thanks for our attendees for being so engaged in everything we do. I just love it as always. You guys are just amazing. And thanks to Looker for sponsoring today and for helping to make these webinars happen. Well, thank you, everybody. I hope you all have a great day. Thanks, guys. Thank you. Thank you.