 I love eating noodles out of this new thermos. It's got a metal jacket. When President John F. Kennedy appointed Robert McNamara as Secretary of Defense in 1961, it was pretty clear from the get-go what sort of ship he wanted to run. McNamara was the president of the Ford Motor Company, one of the Whiz kids who had then hired on as executive staff straight out of the Air Force, and a pioneer of the use of metrics to improve company performance. This approach had worked well for him. He had a lot of success using analysis and careful planning to optimize organizations under his control, and when he became the head of the U.S. military machine after World War II, he went about his new job the same way, by the numbers. And there were a lot of numbers. The rising threat of nuclear war between the Soviet Union and the United States and the ideological battles over the spread of communist forms of government meant that McNamara had his work cut out for him. He regularly ordered massive reports that incorporated all sorts of variables. Troop counts, inventory, military budgets, project budgets, a five-year plan for spending. There were some military leaders who speculated that the sheer volume and complexity of the information was just a rhetorical strategy, that the findings were massaged to support whatever McNamara wanted to do in the first place, and the tables upon tables of numbers were simply there to overwhelm any opposition. And then Vietnam happened. Although the U.S. and France started out simply training anti-communist South Vietnamese soldiers, it became clear as time wore on that the North Vietnamese were going to take control of the whole country and institute a socialist republic without further intervention. The U.S. was faced with a tough decision, either double down on holding off the communist government with their own military forces if necessary, or let them run the country. Unfortunately, we'd already talked a pretty big game to the rest of the world about how we were going to kick the commies out of Vietnam. And let's see here. Freedom, liberty, no taxation without representation, rock and roll. No, there isn't anything in here about restraint. McNamara's management of the Vietnam War was also heavily quantitative, but the shortcomings of this strategy became increasingly apparent over time. Even as military leaders described just how bad the situation was becoming and the draft scooped up millions of young men to fight on the other side of the world, McNamara assured everyone that by the numbers everything was going very well, citing a few key metrics that kept going up and up and up. Megatons of bombs dropped, number of supply vessels intercepted, square miles of land controlled by U.S. forces. But the war just kept going. In order to wrap things up quicker, he prioritized one of these quantities. Enemy combatants killed. After all he reasoned. With a limited number of Viet Cong soldiers, the faster that U.S. troops could burn through those reserves, the faster they'd win the war, right? To the military's credit, this top-level strategic directive was communicated to every soldier, make as many bodies as fast as possible, and they followed orders. The frequently indiscriminate killing caught many Vietnamese civilians in the crossfire, which fueled resentment for U.S. presence in the region, increasing local support for the Viet Cong. In a perfect microcosm for the entire disaster, there was a military strategy session where a Brigadier General suggested that the attitude of the Vietnamese toward U.S. troops might be an important factor in winning the war. McNamara wrote this point down, then on further reflection erased it, saying that he couldn't measure it, so it must not be important. One interpretation of this course of events might be something like, well, that's what you get for letting an egghead and his numbers run the armed forces. Numbers have no place in the war room, it's all about battle-hardened experience in your gut. I'd be very surprised if many military leaders shared this opinion. Measures and analysis are critical to modern warfare, and I think that anyone who tried to run the armed forces on pure intuition would stumble headlong into an immediate catastrophe. Another take might be something like, McNamara was focusing on the wrong numbers. He should have been looking at stuff like the recruitment rate for the V.C. and percentage of the population in favor of the communist government. It's true that the right measures might have given him a better idea of what was going on during the war, a more complete and accurate picture, but it's not like there weren't people trying to tell him that his chosen metrics weren't representative of the situation. It seems like the biggest, most glaring issue was that McNamara was offered that insight on multiple occasions, and each time chose to disqualify it because it wasn't quantifiable. If there wasn't an obvious way to record some aspect of the situation, like the attitude of the locals, and put it into a spreadsheet so that he could optimize for it, he simply wasn't interested. This error is so common that it has its own title, the quantitative fallacy, or sometimes the McNamara fallacy. The line of reasoning that leads to that sort of attitude is flawed, but it's an easy mistake to make. Numbers and metrics are astonishingly useful for figuring out patterns that are otherwise too subtle to detect, or too personal to perceive in an unbiased way. The success leads some people a step too far, saying that if something can't be measured, if we can't fit it into a numeric framework, then it doesn't matter, or maybe doesn't exist. Numbers good. Therefore, not numbers. Bad. Funnily enough, people who actually do work collecting and analyzing numbers for a living are often happy to detail the myriad ways that metrics can be misleading, and just how many ways you can go wrong, navigating the treacherous abyss between a set of numbers, and what those numbers might mean, what philosophers of science call the inferential gap. Unfortunately, no matter how many measurements you pile up in one place, there's always several different stories you can tell about why they're doing whatever they're doing. Take the sequence one, two, three. What comes next? Four is a decent guess, but so is five if we're doing the Fibonacci sequence. Maybe it's one again if we're counting a waltz, or it might be two if we're listing radicals of positive integers, which are the products of the distinct prime numbers dividing them. There are literally an infinite number of rules that will produce any sequence of numbers, and picking one rule, one story over any other, is always going to be a stab in the dark, which is why, ideally, that story fits nicely with all the information that's available, including non-numeric observations. Many researchers familiar with the sometimes slippery relationship between numbers and the story, or stories supposedly implied by those numbers, advocate for an integrative approach, a strategy that emphasizes casting a wide net and accommodating as much information from as many sources as possible. Meta-analyses and reviews of scientific literature are a good example of this sort of thing. You don't read one study and take its results as gospel. You read 50 or 100. You compare their methodologies. You look for trends or exceptions, and hopefully you walk away from the whole exercise with a nuanced interpretation of what's likely to be true, a narrative that accounts for most of what you're seeing. If you're very lucky, you'll be able to compare radically different approaches to the same question and find a single thread that unites them. If you find the same story being told in an economics journal and a biology journal and an anthropology journal, you're probably onto something. If McNamara had taken a more integrative approach, assimilating anything that might help rather than systematically ignoring anything that wasn't on his short list of supposedly key metrics, it's possible that he'd have recognized some issues with his story about body count and winning sooner. That he'd have imagined some alternate explanations that would account for the numbers that he was seeing, as well as the non-quantitative observations about how the war was going. But it's hard to shake people of that conviction that a tale told with numbers is a fact, or that a fact without numbers is irrelevant. Now, if you'll excuse me, I have a wow rate that I have to get to. Apparently we have a 32.33% chance of survival. Repeating, of course. Where have you seen the McNamara fallacy at work? Can you think of anywhere where a more integrative approach to data might help us understand more of what's going on? Please, leave a comment below and let me know what you think. Thank you very much for watching. Don't forget to blah, blah, subscribe, blah, share, and don't stop thunking.