 So let's start with the flaw of averages. There's a mathematical name for it. It's called Jensen's inequality. And we don't have to do a poll. Three of you have heard about it. And of those, maybe one of them knows what it means. But I'll explain it in Laman's terms. So Jensen's inequality, I call the flaw of averages. Why? Because I want people to understand it. It says that plans based on average assumptions are wrong on average. Look, when you flip a coin and you say that you're going to get a half of the head and a half of the tail, you're right on average. This is worse. This is worse than that. So for example, consider the statistician who drowns in the river that is on average three feet deep. That's a classic case of the flaw of averages. It explains why so many projects are behind schedule, beyond budget, and below projection. Now, to understand the flaw of averages, well, oh, where does it come from? What causes it? What causes it is when you are dealing with uncertainty, like rolling a die, for example. This is uncertain, right? It is. When I roll that thing, it's uncertain. And I show this to you, and I say, I want you to build this into your model. And you say, oh, OK, I'll just use this average die here with 3 and 1 half dots per side. Every time you replace an uncertainty with an average, you are doing something as ridiculous as this. By the way, these flat dice are available. I don't know, get me an address over there. I can send you some if you cover the postage. So this is no joke, and yet we almost all do it. So you have to understand the arithmetic of uncertainty. Now, let me just define it for you. And by the way, there are definitions of all these things in mathematical terms, which I do not use because they tend to trigger PTSD. So arithmetic tells us that x plus y equals z. Easy enough, you've all done it. The arithmetic of uncertainty says, what do you want z to be? Here are your chances. Right, I'm sure all your organizations have standard protocols for estimating the chances of achieving your goals or avoiding risk. No, you don't. At probabilitymanage.org, we are trying to develop such standards. So again, I don't assume any statistical knowledge. I can repair the damage. It's called PTSD. I want to give you an example of how I try to cure PTSD in people. Because even if you don't have it and judging from the statistics looks like maybe 80% of you do have it, let's take the 16 per whatever it was that didn't have it. Well, I don't want you to induce it in others. So I'm going to give you a little constant building exercise I use to get people across PTSD. So I'm going to show you how to solve the following equations. We do it right now. Oh my god, that reminds me of my statistics course. That's ridiculous. Those are the differential equations of motion of a bicycle. So you already know how to solve those things. Not through the seats of your intellect, but through the seats of your pants. I'm deadly serious here. Steve Jobs referred to the computer as a bicycle for the mind. It's a beautiful metaphor. Computers are so fast that we can now actually solve statistical problems with the seats of our pants instead of the seats of our intellects. But the best thing is when you connect the two together. I don't know if there are any pilots of small planes out there or sail planes that used to fly gliders. It was wonderful. You could not learn to do that either by reading a book or by just jumping in the airplane. Either would have led to death. But by connecting the seat of the intellect to the seat of the pants, you achieve what I call limbic analytics. The name of my blog is limbic analytics. That means the limbic system is that part of the brain that connects the reptilian brain to the rest of the stuff between your ears. Once you engage the limbic system, you have just tens, hundreds of millions of evolution on your side. And I want to give you a nice metaphor of limbic analytics. I want you to think about this. There's a wonderful scene in Jurassic Park where Sam Neill is communicating with a tyrannosaurus. And he grabs a railroad flare, and he waves it back and forth, and the tyrannosaurus follows it with its eyes, and he throws it off into the jungle. So when you're explaining one of these things to a layperson or your boss or whatever, think of them as the tyrannosaurus. And what are you going to do to get their attention and keep their attention and keep from getting eaten? OK, so let me give you an example here of the flaw of averages. Let's look at a scheduling example. Your organization is releasing a website and 10 pages. I don't know, it's got some sort of date on it. And you need to get all 10 pages done. I don't know, there's a legal page and this page and different kinds of data page. And on average, it takes six weeks to complete each page. These are done in parallel. It's the simplest kind of task in the world, because I got team one doing page one, team two doing page two. There is uncertainty. And I want to point out that the way to view that is that could happen, or that could happen, or that could happen, right? Do I have the attention of the tyrannosaurus in the audience? Something moved, triggered something deep down in the limbic system. All right, notice you can't finish the project until the final task is done. So you don't go live until all 10 are done. And they're all uncertain. Yes, but the average is six weeks. So your boss comes to you and says, when do we go live? And you say, I don't know, boss, because I don't know how long team one will take, or team two, or team three. And because you said you didn't know, the boss says, give me a number. I don't know, do we only say that in the US? Has anyone in this group heard the term saying, give me a number when you're uncertain about something? That is the fork in the road to hell, right there. That's when you say, oh, well, OK, if you need a number, it's 3 and 1 half. Right. By the way, these are not totally useless. They're actually coasters. So you can get a set of these and put your beer on there. All right, well, so we're going to ask a little question here. And this is going to be another poll question. So what's the chance that will actually be done in six weeks? Oh, because what happened when the boss, by the way, hold on, hold on, the boss said, give me a number, right? And you responded with, would you accept an average? And the boss said, sure, if it's all you can do. And then you said, oh, well, on average, we'll be done in six weeks. Because if I put all the averages in here, I'll get the average of six weeks out, OK? So what are the chances we'll be done in six weeks? You want to pop that up, Charlotte? Hey, I want to presence this by saying, none of these are right. Poor lesson here. When it comes to estimating chances, it's like horseshoes and hand grenades, close counts. It's one of the first things we sort of have to teach in limbic analytics and in dealing with uncertainty. So yeah, one of these is a lot closer than all the rest. I love these polls. See, when I teach at Stanford, right? If I do this with a show of hands, everyone's looking around the room at everyone else, I never get clean results. I'll just leave a few more seconds. We've got 67% of people responded. So. Yes, it's always interesting to see how many don't respond to that. That gives you another piece of information about the question. The lowest answer at 2% is one in four. And the highest is at 26%, one in 10. Can you display the graph for us? There we go. Well, this is good. This is good. So in the US, this is a profound study for some graduate students out there. In the US, 50%, at least the audience that I've been doing, 50% is usually the winner. And in the UK, the winner is one in 10. Great. OK, the right answer is one in 1,000. And it's not enough to know that. You've got to be able to explain it to a 10-year-old or the Tyrannosaurus or whatever. But it's really simple to see that it's one in 1,000. So let's remember that could happen, or that could happen, or that could happen. Imagine, and I didn't tell you. And so you've got each team. Did we get this far? That could happen, or that could happen, right? Every team is uncertain. I didn't tell you anything about the distribution of these times. But I'm not going to, because nature doesn't always tell you that. But suppose there are 50-50 chance of each team being over or under six weeks. Then it's like every team is flipping a coin. Heads it's under six weeks, tails it's over six weeks. And the only way you're done in six weeks is if all 10 teams flip ahead on a coin. Can you do the math? That's 1 half to the 10th. 10-24, actually very close to 1 in 1,000. OK, so there we are. So that was the flow of averages. It is insidious. It is everywhere. Let me give you a more sobering example of the flow of averages, so you grasp it right away. You saw the statistician in the river. But here's a more sobering example. Consider a drunk wandering back and forth along a busy highway. His average position is the centerline. This means that the state of the drunk at his average position is alive, but on average, he's dead. In the US, we say that that's not close enough even for government work. I don't know if you have a similar saying in the UK. Now, the flow of averages has been understood since the Danish mathematician Jensen came up with it in 1906, published a paper. Again, the mathematician is called Jensen's Inequality, which is stupid because the guy's name was Jensen. And anyway, he was going to remember that. So I call it the flow of averages. But I started my career in OR. It's what the British would call operational research. Is that something anyone? Do I see any nods of heads? Operational research? OR, in the US, we call it operations research. And I started there, but if you just increment the first letter by one, this is easy to see here. Does this show up? Oh, R. Kind of tiny, isn't it? But even there, you can see what would happen if I incremented the first letter by just one. I get PR, Public Relations, which is my current field. And so as a public relations guy, Jensen's Inequality is my anchor tenant. Here's the book, the flow of averages. And I want to bring you now the audio logo. So when you're promoting something, let's see. Here's an audio logo. Oops, come on. Don't tell me my audio logo isn't working. There we go. Did you guys hear that? Do you know what that audio logo is? Did that ring a bell? No one knows. Recognize it, but couldn't tell you what it was. OK, OK. That was Intel. Here's the flow of averages audio logo. All right, now let me show you a little about how we're actually solving this problem and performing the arithmetic of uncertainty. So I'm going to do it with dice. And the way we're going to represent dice, remember what we need to do here. We need to represent this as data, which obeys both the laws of arithmetic and probability. So I've got to be able to answer questions like, what's the chance that something happened here, right? So the way I represent the dice is with arrays of Monte Carlo simulations of the dice. So I rolled three dice 10,000 times. And I stored the results in arrays. Now, the arrays are called SIPs. I'll explain that in a sec. Now, by the way, everything you're looking at is native Excel. This is kind of important because not that you shouldn't be using R and Python and all those things, but if you can get dashboards in native Excel that don't require any other software, then you immediately have 1.2 billion users, which is useful. So there is a formula in Excel called the index formula. And that's pointing down into these arrays. So there's roll 1, roll 2, roll 9,876 is 3 ones. Well, oops, it isn't. Oh, 9,000. No, it's 9,765. There it is, 3 ones. So I haven't memorized them all 10,000, but it's auditable. It's just data. And now there is a formula in Excel. There's a command called it's under what if analysis. It's called the data table. And let me just show you what happens. This is now buried on another sheet here. All this stuff is available on our website. But it's just really interesting to notice what happens now. I can, if I want to roll one of the dice, I just say show me what that die does. I hit the Enter key. And that's 10,000 Monte Carlo trials for the die. And then what happens if I add two dice together? They're more seven than anything else. And then you probably never heard of unsimulating, but here, Control Z minus 10,000, Control Y plus 10,000. So now we have simulation readily available. It's just a native Excel file. You can share it with 1.2 billion of your closest friends. And it's interactive. So you can do limbic analytics with it. The nonprofit has been building standards to do this cross-platform our Python Excel because we suddenly now have this capability to do interactive simulation. Now, let me do an environmental example. So let's take, say, sea level rise. We've got a little town here. And I think I need to adjust this a little. So I have a river. The river has dice around it. I have a potential flood. What if I know the statistics of the flood? The statistics are that the average is 2.4 meters. And I've got a 95% confidence range of plus or minus 2 meters from that. On the lower left here, over here, I have a little table of damages. So if the flood goes up to 1.82 meters, there's $1 million, $2.8 meters, $5 million, and so on. And the average flood of 2.4 meters, if I plug that in here, 2.4 meters across, then the damage is zero. Sure, but what would happen? Let's look through some of the trials. I have a library, a SIP library of floods in here. So that could happen, or that could happen. You can go, oh, look, we had a flood that time. That could happen. Oh, we had a real bad flood that time. This model is also available on the website. You might ask yourself, what is the average damage? So the damage of the average flood is, over here, zero. The average damage up here is $5.5 million. So you can see this is very central to problems in the environment. And where it comes in is when you have a nonlinear penalty function or a nonlinear reward function. So once the flood goes over the dikes, then, of course, it gets very nonlinear and very bad. Watch what we can do, though. So our average up here is $5.49 million in damage. Let me start cranking up the height of the dikes. What if I pop the dikes up to 3 meters? See, they got taller. Oh, but watch this over here. Watch the average over here. Control Z. I want you to watch that and that. Control Y. Control Z. Waving the railroad flare back and forth. Do something that moves. And again, this is 1,000 trials per keystroke. Hold on. Let me just check my timer. Good. It says I have about three minutes left. So let me now put this in kind of a bigger context in terms of environmental modeling, which seems to be on everyone's mind here. And let's see. I think in this case I'm going to share my screen. All right. So now you should see my whole screen. There we go. So remember, I talked about doing the arithmetic of uncertainty here. And going back to the dice for a minute, first of all, why did I call those elements sips? So I'm trying to replace numbers with uncertainties. I already know that everybody has bit and bite firmly emblazoned in their brains. I need something that's going to remind you of bit and bite. So I knew right away I would need sip and then something else. Sip, like bit, sip, right. So sip is stochastic information packet. And that's why they're called sips. That was easy, but quick. What would correspond to bite? I've got bits and bytes. I've got sips and slurps, obviously. OK. But what does slurp mean for God's sakes? Well, this was tortured. But it's very important. It stands for stochastic library unit with relationships preserved. And I'm going to give you an example of one now. So remember, we have to be able to do math with these. You've got to be able to add them, multiply them. So let's imagine that we have some model of global climate change. And here it is. It turned on my laser pointer. So there's a model here. Actually, people have made more than one model, right? And I don't care how good your model is. Sea level rise is uncertain. We all know that. In fact, one of the key outcomes of these models is that weather systems are more uncertain. And we do see that now. It's one of the early confirmations of the model. So it says I'm out of time, but I'm almost done. So it is uncertain. Now, you're not just interested in sea level rise. By the way, suppose there were no temperature effects, just sea level rise. Let's just ask ourselves what the economic impact of sea level rise is. Oh, well, I have to go to 500,000 coastal regions. I have to model the economic damage at 500,000 coastal regions and add that in to this model up here. Right. This model is already collapsing under its own weight. There is absolutely no way you could add 500,000 more models to it. But you don't need to, because if you're doing sip math, you just publish the sips of sea level rise in the cloud. The local models all take these in. And you see, it's not just the same distribution, everybody. It is the same trials. Once on the trial where sea level goes up two meters, it goes up two meters all around the world. And then when it goes up one meter, it goes up one meter all around the world. And then once you've learned that, you put in local knowledge. But you're adding your local knowledge on top of the coherent sea level rise sips that are coming in. We all get six meters on one trial. We all get two meters on another trial. Then we add in our knowledge of the tidal basins and junk like that down there. And then, oops, wrong direction. Now, these numbers, these sips are a slurp. Why? Because the relationships have been preserved of the global things like sea level rise. And by the way, I mean, you really have another set of slurps on temperature. And whether or not we get hit by an asteroid or stuff like that. But these then go up to the star of the show, the summation sign, because sips can be added together just like numbers. I have to understand on the dice, when I added two dice together, I just could have summed every element. And then the sum of the sips is the sip of the sum. My god, they're additive. They're multiplicative. Anyway, that gives you your distribution of global impact. We decompose the problem. Why? Because we had arithmetic. We could break into pieces and add them up. Think of the sip as the Hindu Arabic numeral of uncertainty. And hopefully, after all we've been through, maybe that means something to you at the end now. So thank you very much. I ran a couple of minutes over. But I blame that on technical difficulties. Good. Thank you very much, Sam. So we don't have any questions yet. But maybe some of them. I think I'm going to be able to share my screen. Yeah, hold on. I can't unshare my screen. Can someone in charge there try to unshare my screen? This happens occasionally. I think I've done that. There you go. I'm like, Derek, thank you so much. Yeah, I mean, well, people are so thinking. I mean, what would your sort of recommendation for people, Sam, who've never dealt with really probabilities and estimations before, they may be used to doing all their calculations in anything Excel, R, or Python, whatever. But what would your advice for them to get started in terms of? Yeah. So that's a good question. Here's an analogy for you. First of all, they should visit our website and play with some of the models. So everything is just free and fun. Here's probabilitymanagement.org. SipMath. Go to models. And so I mean, we've got, oh, there's very interesting COVID stuff that we're modeling. But let me see. The dice are going to be down here somewhere. Oh, here comes the flood. There are the dice coming up. We have then applications. If you, all of these are just demonstration models. But they're big companies are actually using this stuff. If I go to applications, I mean, there's everything from climate change. That's where our little flood model lives, financial management, military readiness. Military readiness is another place where people are unable to add things up. And agriculture is a big one where you and I have been working, Keith. But what I was going to say is this, that there's an analogy with electrification. So the reason, and I'll bet that everyone has heard of computer simulation or monocular simulation. I would bet that 80% of your audience has heard of it. And the problem with monocular simulation is that you have to be a statistical expert to know how to generate the right random numbers to go into it. And let me just define monocular simulation for those who haven't heard of it. It's basically analogous to what you do before you climb on a ladder to paint your house. You shake the ladder. That is, you bombard the ladder with random forces to find out if it's stable. It's a computerized version of that. And many people will say, well, you know, with simulations, garbage in, garbage out. Well, that's wrong. When you shake a ladder, the distribution of forces on the ladder is nothing like the distribution of forces when you climb on the ladder. But I'll bet that you're all gonna keep shaking ladders before you climb on them, even though I've told you you've been using the wrong distribution your whole life. But let's go back to serious monocular simulations where the input distributions do matter. Most people cannot figure out how to generate the random shakes for their model. But about 25 years ago, I came to the realization that the whole world is using light bulbs and most of them don't know how to generate the electricity. And that was a real eye-opener for me. And I did some research. It turns out they take electrical engineers and they put them in a big building called a power generation plant and then they distribute the electricity over a standard. And in the US, we use a 60 cycle AC current standard. In the UK, you use 50 cycles and I think it's like 200 volts or 220 or something. Anyway, so that's what probabilitymanager.org is doing is developing the current standards for the probability management grid. And here's what I would really recommend. People in your organization who have data that they think is uncertain and they wanna try to quantify should get in touch with us. Our mission in life is to spread the word, right? And we will help you do it. Great, Sam, I got, thanks for that. I got a question, very interesting question for you and here and from the Zepeah Holland. Are there any thematical domain projects curating SIP databases with validation of the SIPs? I guess the question is, do you have any examples of any organizations who actually have created SIP databases and kind of using them at an organizational level or what are they? Yes, yes, here's a very fun one. Very fun and very grassroots. Let's see if I have the file still up. I'm gonna open it. Okay, just a second. This is really, this is so fun. We work with an outfit called the Government Finance Officers Association. It is a, it's a nonprofit of 20,000 chief financial officers of municipalities around the country. And so what they've done is they have created a SIP library of the accuracy of tax forecasts. And now we're able to pass this out to other people in the organization. So for example, you know, you're the CFO of, you know, nowhere Kansas. And, you know, you're expecting a million in business taxes, three million in property taxes, four million in sales taxes adds up to eight million. And then I come to you and I say, well, what's the chance you're actually gonna get your eight million? And you say, people don't do that. Like that's unheard of. How would we possibly do that? Well, here's how you do it. So we have a, this has not quite been released, but it's all gonna be free. It's a little add in called chance calc. And you're gonna have to learn three keystrokes. So keystroke one is I'm gonna open a SIP library, open the SIP library. And it's gonna be the GFOA Historical Accuracy Library. Accuracy Library. And you saw it flash by for a second. I don't have time to go into the details. This one is actually a virtual SIP library. Instead of millions and millions of trials, this one is something the size of a big tweet. Quite amazing technology. It's now is going from DC direct current to AC current. It's much more efficient to ship. But anyway, there's the library. Okay, it includes little shapes over here. It's cute. But now what I do is I'm gonna put in a column, Historical Accuracy. And so the historical accuracy, I'm going to select those cells and I go to SIP input. I just have a little menu here and I'm just gonna pull these off. Now, by the way, this is a macro which is free. Let me click the button. Oh boy, there are the distributions. But the model is always native Excel. I could send this off to a billion closest friends. Notice I now have trials in here. So I'm looking at trial one, trial two. Because these are virtual SIPs, you know, I could look at trial eight, seven, six, five, four, three. There it is. But now my simulated revenue is the following. These numbers, think of these as percentages. That is to say on the first trial, I got 79% of my million dollars, I got 100% of my three million, I got 95% of my four million. And so the simulated revenue is that times that. And so there's my total, 762. Oh, but I wanna know what the chance is that that number is greater than or equal to eight million. So you have to, so far, open a library, insert the SIP. There's one more button you have to learn. And I'm hoping that you once played Monopoly as a kid. And so I'm like, hey camera, I'm over here, back here. All right, thanks. So if you played Monopoly as a kid, oh, and also I'm presuming that you somewhere in school learned what greater than or equal to means, right? And we already know, you know what eight million means. So I just go down here and I need one more keystroke for chance of whatever button. And so I'm just gonna ask the chance that that cell bears that relationship to that cell and the answer is 68%. I mean, people like me can do this stuff in our head, but most people require simulation. No, of course I can't do it in my head. And by the way, if you were looking at this, you said, oh yeah, we're gonna hire a bunch of firemen and policemen, I don't want a 68% chance of blowing my budget. How about if I go down to like, you know, 7.75, that's up to 89%. The fact is, however, that these numbers came in from a very rosy economic period. And what if I replace these with numbers from the depths of the 2009, 2010 recession? Let's do that, because I've got, oh my gosh, now I'm down to 54% again over here. So here's the pitch. We are looking for people who wanna do this stuff. We will help them. We've got funding from some large corporate donors. We always need more funding, but we're always looking for exciting things to do. So we start out working with people for free. And if all the tools are free, if you can do it, then we will use you as a poster child to promote our work. All right, that's the thing. Thanks, Sam. I think at times just about that, but I would just also like to point out that there's a couple of things which also make these tools really easy to use. And one that I found particularly useful is that often people get stuck on how to create probability distributions, or you now have integrated the meta log distributions which enable already anyone to say, well, what's the most likely value? What's the chance that it's can't be 10% lower than this value? What's the chance it can't be 10% higher than this value? Or you can get that from your data and then immediately generate your probability distributions. And I think that's a tremendous supplement to these tools. And then, yeah, and I think the other one was how it's a random number generator which standardizes the seeds so that you're always getting the same results and different repetitions. So I think I'll just like to point out those, I think is two really useful supplements. Look, I've shown you the tip of one of three icebergs. I hope you folks will stay in touch. The meta log distribution deserves at least a lecture of its own, the new random number generator. Well, that was invented by Doug Hubbard who you guys saw last week or so, I believe. We've teamed up with some pretty bright people. Doug is one of them and like I named Tom Keel and there's another who's invented the, what I'd call the Taylor series of probability distributions. So I hope that in a group of this size we end up with some permanent friends and collaborators. And... Well, there's plenty of people out there in the network making all kinds of measurements and analysis. So I hope some of them might be tempted to get in touch. So thanks again, Sam. Great to have you online, great privilege to have you and I hope that's been useful for people and I hope to see you again soon, yeah.