 Hi everyone, I'm Rachel White, Senior Vice President at New America Foundation, and it's a pleasure to welcome you here today. I hope you've seen the books outside. I'd encourage everyone to buy the leading indicators. We're really lucky to have Zachary Carabel here. Zachary's been part of our New America universe for a long time, well before I was here, and I've got eight years in this place. Zach is a member of our board of directors and a prolific writer. I think this is book number 12. One of the things that I always enjoy the most about Zachary is his ability to take complicated subjects and make them understandable for the likes of all of us and me in particular. And I think this is another stunning example of just numbers, which seem inapproachable, and the way Zachary has distilled it down to something understandable and interesting. We'll be joined a little bit later by Steve Clemens, who's moderating the Q&A. Zachary will start off with a talk for about 15 or 20 minutes, and Steve will come in when he comes in. In the meantime, welcome Zachary, and thanks very much. So yes, I feel like I'm going to end up moderating Steve Clemens at the end of this. So I'm going to try to talk for 15 or 20 minutes, maybe 15, maybe 20. And then hopefully Steve will come and we will have a conversation, and if Steve does not, then we'll have a conversation anyway. Sonia, thank you for coming for lunch. And as Rachel said, I've written a bunch of books before. This is the first time I've done kind of a purely economic study. And let me give you just a sense of where this book came from in my own history. So I was trained as a historian and a political scientist, and then I kind of inadvertently acquired a Wall Street career after having had a journalism career, and I never stopped a journalism career. And one thing that struck me as I became a fund manager and became kind of a Wall Street economist, de facto, without having had an academic background in that. So kind of like learning a language when you move to a country but you don't have the grammar and the study behind it, was the degree to which vastly consequential decisions, particularly in markets but increasingly in politics, revolve around what a limited set of very simple numbers say about what's going on in the world. Versus what it is these numbers say about what's going on in the world or what they're capable of saying. And that led to sort of a study and an awareness of the fact that whether you kind of want to or not, political, economic, and increasingly our social sense of what's going on in a country and in a nation. And this is true almost everywhere in the world is dictated by this barrage of economic statistics, particularly GDP and the unemployment rate and trade figures and inflation that come at us in metronomic waves weekly that then shape our sense of are we doing well or are we doing badly. And the degree to which all of these are just a set of numbers that human beings made up to measure something versus absolute representations of reality is a disjuncture that I thought we don't sufficiently pay attention to because we treat these numbers as kind of an absolute, as a representation of what's happening in the world. Rather than as a set of numbers that we made up to measure something and that these numbers have a story and they have a history just like human beings have a history and a story. They were invented at a moment in time to answer a certain set of questions at that moment in time and that world and that moment in time is remarkably recent given how much stock literally and metaphorically we place in this set of numbers. So that's kind of where the book came from, right? How do we come to live in this world shaped by these numbers and who invented these numbers and how did that happen in the first place? What is the story? And as you'll see from the book and as you'll hear from me right now, there's a remarkably recent time frame in which we came to live in a world where things like is GDP going up or down or what's the unemployment rate or what's inflation or what's our trade balance have come to shape elections, politics, trillions of dollars of spending and massive amounts of individual decisions about do I buy a home? Do I get this job? Do I take out a college loan? And that that world is in the space of several generations, not several hundred years. So if you look and if you take the tool, there's something Google has done called Engram which is to digitize every single thing written until copyright kicks in. So you can go into Google Engram and you can type in just about anything. It gives you a nice little chart of the frequency with which this phrase or set of words appears in everything printed ever. And if you type in the words the economy, you will find that until about 1930 it goes like this. Essentially until about 1930 the phrase the economy by and large doesn't exist. It exists a little bit. And then in 1930 it does this sort of hockey stick. And suddenly from 1930 to the present the idea of the economy as a term that has meaning becomes interwoven into what we write and how we speak. But until then nobody talked about the economy. There was no, you know, Lincoln Douglas debate about what's the state of the economy. Teddy Roosevelt did not get up and talk about under his watch the economy had grown and he had made it bigger. And in fact until the late 1920s until the onset of the Great Depression the notion of there being this thing, this abstract thing called the economy hardly existed at all. Not until these numbers are created which give form to this abstract thing called the economy is there even a thing called the economy to talk about. And then one of the early titles of the book was the invention of the economy and other was like the matrix economy. You know that the world, the numbers, not the matrix has a nice metaphor, the matrix has that cheesy pop film that we all, at least I certainly watched, I don't know, four or five times. That the world that these numbers say that we're living in and the world that we actually are living in are not the same world. And that if you took the world that the numbers tell us you'd think that we exist in this particular universe and the world that we actually inhabit bears only tangential relationship to it. So in many ways part of the book is about how did we invent the economy? How did we come to live in a world where the economy is a thing? Because before the numbers what was it? I mean nations from time immemorial had kept trade statistics and agricultural statistics very loosely. They did that purely because the only two sources of revenue for most governments until the 20th century were how much farming was there in the land that you controlled and how much trade. Those were the two things you could kind of tax. But beyond that you know we didn't really measure this stuff. The impetus for it a little bit was progressive America in the 19th century. It was some sense that there was this thing called industrialization and factories that was creating a different kind of labor world. And wanting to get a hold of that so that people who were real labor advocates could prove that there were some disruptions going on. But other than that it took the Great Depression to act as a spur to the development of most of the numbers that we now think of as being so consequential to our lives. And it really took the dislocations of 1929-1930 for President Hoover, who was himself a real apostle of scientific management and measurement, to fund the Bureau of Labor Statistics with the support of Congress to start figuring out just how many people were actually unemployed. And having created some study of unemployment which showed that things were really really bad even though you could probably look at your window and tell. Hoover then proceeded to lose the 1932 election. Roosevelt who uses this nascent set of numbers to prove that the government wasn't doing enough to deal with the crisis. Hoover there is sort of hoist on his own statistical patard. So it took the spur of the Great Depression to start studying this thing and unemployment was one of the first. And it's not like there was an unemployment statistic that just emerged whole cloth. First you had to define what it didn't even mean to be unemployed. Then you had to figure out how do you count because you can't go around and count every single person who has a job and not. And this was before there were really good sampling sets and statistics itself was a profession that was emerging. So it's a pretty rudimentary snapshot. But even the very definition was a new thing in the 1930s because for most of the 19th century there was no unemployment either. In a world where there was scarce labor and abundant work. The idea that you could be without work was sort of inconceivable because it either meant you were dead or you were a vagrant or you were drunk. But there was always work that needed doing and you always needed to do work in order to live in a world where there was zero social safety net other than the kindness of churches and the willingness of your community to support you. So the 1930s become this spur as does the 1930s to developing the foundations of what we now know as GDP. And that was based on if you're going to spend all this money as governments in Britain and the United States were doing to try to address this massive problem that some of these early employment numbers were showing. You would want to be able to prove that there was some relationship between the money you were spending and some sort of constructive outcome. But the only way to do that was to be able to measure what the output of this thing called the economy was and how much the spending and activity you were doing to make things better was actually making things better. So that becomes the genesis of creating national accounts which in turn become the basis of GNP which in turn becomes the basis of GDP. And the other spur is World War II where the big question for the United States and for Great Britain was how much domestic production could you turn to making tanks and planes and guns without so imperiling the domestic manufacturing economy that you would ruin the domestic economy while winning the war. And again you needed to know what was the capacity to construct and you needed to know how much you could turn toward war versus domestic industry. And then there was also the inflation which was a major concern as it had been in every war that people get worried about scarcity because of war so they start hoarding and they don't spend and you create domestic crisis as well. So it was also spur to start measuring prices and measuring price stability and having the government take a much more active role in price stability. So what we have by 1950 is a rudimentary set of these numbers and so that's 60, 65 years ago. That's the entire time between the invention of these numbers in any constructive or coherent fashion and a world where it seems like everything we talk about hinges on what these numbers say. And what they dictate. So what I argue in the book is that we're very good because of the legacy of what these numbers measure and what they can measure at assessing and describing and sort of quantifying a 1950s world. We have a set of numbers that are really good at capturing 1950s industrial nation states that make stuff. And we are much, much less up to the task of describing and capturing and quantifying a 21st century world where information technologies and ideas generate any kind of economic growth that there is where labor is much more fluid where capital flows globally and not just nationally. And where the ability of the statistical framework to capture those realities is much, much more limited whereas the ability of these numbers to capture things like factories making cars is much more established and successful. Really good at measuring that world, not so good at measuring this world. And a lot of the argument of the book is that if I gave you a 1950s roadmap and told you to get from point A to point B, there is a high likelihood that you would get lost because it simply would not have factored in all the changes and new roads and new towns and intersections. And in many ways using these numbers as intimately as we now do to navigate this world that we're in is a formula for both the law of diminishing returns and for getting lost. And it's one of the reasons why so much of what we think is going to happen in the world isn't happening. So what do we think is going to happen that isn't happening? Let me give you an example of that. And let me give you an example of it's probably that we've become ever more dependent on using these numbers particularly in a national conversation even as these numbers tell us far less than what we expect them to or what we demand them to. So one of the most repeated statements in the 2012 election was no president has ever been reelected with an unemployment rate greater than 7.2%. Everyone here I'm sure heard this right or not everyone here, everyone except for the people who haven't heard it, heard about it. It wasn't all repeated you know and it was used by the media and it was used every time that the unemployment rate came out and it was kind of debated and it led to conspiracy theories about whether or not there was some relationship between the White House and the people publishing the numbers that were going to make the numbers look better in order to raise re-election chances. So that statement as presented as such seems like a pretty strong formula, a pretty strong connection between this economic reality and this economic number and real world political outcomes. However it was not until the late 1940s and really until the 1950s that the US government and the Bureau of Labor Statistics actually published an unemployment rate. So let's say for the sake of argument there was this thing called an unemployment rate in 1948 which there might have been but it wasn't published so it wasn't really known so it wasn't really discussed. Between 1948 and 2012 there have been 16 presidential elections, 7 people have run for re-election and 2 people have lost, Jimmy Carter and George Herbert Walker Bush, Jimmy Carter in 1980 and Bush in 1992. So to make the statement no president has ever been re-elected with an unemployment rate greater than 7.2% is basically saying in the 7 times that this has happened it's never happened. Which any statistician would tell you is a meaningless set. I mean you could make that generalization with a set of 7 but your standard deviation, your margin of error would be so vast as to make that statement completely unsupportable. And yet we use these numbers in common parlance constantly as if they can tell us that. Now I would say look if in a thousand years we've been keeping these numbers exactly the same way and impossibility but let's just say theoretically. We will probably have enough information to make some really strong statements about when X happens, why it happens. But the kind of assumptions we currently make about laws of economics, laws of macroeconomics and how they will play out. We simply do not have enough information collected over enough time to make any of the conclusions we're making. So take the stimulus act which has been much in the news lately, the stimulus act of 2009. And this is not by the way just full disclosure, this is not meant at all as a partisan statement for better or for worse. It is simply a commentary on how we get to spend what we spend and what the consequences of it are. So in February of 2009 President Obama newly elected as we know gets up in the midst of this intense crisis that's getting worse. And one of the things that his transition staff of economists had done in the fall of 2008 and then into 2009 was to move as quickly as possible to try to figure out just how bad this recession crisis was. And to try to figure out just how much money needed to be spent immediately by the government to stem the economic bleeding and to turn things around. So what that leads to is that Obama gets up in Arizona in February and announces what's the American Investment and Recovery Act that becomes known as the stimulus or the bailout. And that's $787 billion bill which then becomes like an $800 billion bill. Obama announces that in passing that act and spending that sum of nearly $800 billion, 3.5 million jobs will be saved or created. Now you flash forward to the present as has often been noted by partisans who were against that and those 3.5 million jobs didn't materialize. It's certainly true that they may have materialized in terms of jobs saved but the advantage of making that claim cynically is that you can never prove that those jobs were saved because you don't get to replay the tape with a different amount of money spent or no bill passed and show what the outcome would have been in the absence of it. So it's a convenient statement in that it's inherently unfalsifiable to say that 3.5 million jobs were saved by that money. By the way, it's one of those laws of like if you're going to forecast impending doom and the collapse of everything, just don't give a date. So why is there this mismatch between that? First of all, what allowed the President of the United States to get up and make an absolutely formulaic statement of spending this much money will create that many jobs? It's a very specific correlation. He didn't say we're going to spend $800 billion and many jobs will be created. He didn't say lots of jobs. He didn't say more jobs than would be created if we didn't spend this or more people will be out of work. He said 3.5 million. In order to get that number, you had to have a formula. There had to be a calculus. There had to be an input and an output. In order to create an input and an output, there had to be a team of economists who looked over the legacy of government spending that they could possibly look over in the 20th century and try to figure out the relationship between additional non-planned emergency crisis government spending and jobs created relative to the size of the workforce as measured at that time by a statistical agency and how many new jobs got added in terms of employment as defined in a period of time after that spending. And then figure out what the current size of the economy was in 2008, what the amount of recession and depression was doing to the potential size versus the actual size and what the amount of spending then would do based on those formulas. And all those formulas are, because of what I said at the beginning, 60 years of information in the 20th century. Very little of it was kept in exactly the same form as it was in 2008. So a lot of economists, wow, that was very cool. What was it? Did I just like tap into some, you know, okay. So a lot of economists went back and they did regression analysis, not the Freudian type, although that probably would have been helpful too. The statistical type where you backfill information based on current methods and you try to find the data that might have been kept at the time but wasn't rigorously kept in the form of the same statistics. So you do a lot of regression analysis of the 20th century and you try to figure out what the relationship is between a dollar of government spending and actual hiring. That's an extremely mechanistic way of viewing a thing called the economy. If you get the inputs right and you get the outputs right, you can actually turn the levers such that X amount of spending turns into Y amount of jobs. And it's not working that way. So the question is why is it not working that way? One answer is, well, actually not enough was spent, you know, that the Output Gap was greater and that they understood it at the time but it wasn't politically feasible. And another is, oh, you know, government can't spend and it's always going to create problems. Part of what I try to highlight in the book is that it rests on such a flimsy foundation of information and data that to believe that you could make that kind of formula play out in exactly that way is to misunderstand exactly what these numbers can tell you and not. If they're very good at measuring closed economic systems in the 20th century where there were industrial nation states and they're not, as I argue, very good at measuring the world we're in, it should not come as a surprise that those formulas break down in the real world. But we are in a framework where those formulas are absolutely essential to how we spend anything. In the past weeks, this Congressional Budget Office has been in the news increasingly because we rely on the Congressional Budget Office to make formulaic assumptions about the outcome of current spending on future deficits, all based on assumptions of inflation rates and GDP and tax revenues, all based on this limited set of information over the past 30 or 40 years. The CBO didn't even exist and do this until the late 1970s and now we have a framework that constrains everything through this very narrow aperture. One of the reasons that this breaks down is that if you had this closed economic system called the economy that's based on output in the 1950s and that output is based on people making stuff and it's based on people making stuff within a country, then maybe there is a relationship between what these numbers say and the world that we're living in that's very tight such that you could make this kind of formulaic assumption about what's going on in the world. But if we live in this world of kind of global capital flows where supply chains are spread out around the world where nothing is quite made anywhere, then the idea that with more input of money into this closed system called an economy employers will simply add jobs because that's what they did at an earlier point in time is I think to mismeasure the world we're living in. One of the reasons these things break down is because we don't live in closed systems. We don't live in worlds where more income and more easy access to cash naturally translates into increased demand which then naturally translates into increased hiring. Think of the manufacturing renaissance that's much touted in the United States. We could build factory after factory after factory in the United States today as we are. And that same factory will have 350 people trained in software and just-in-time manufacturing who can maneuver robotic floors whose software packages change weekly to build new things. Whereas that same factory would have been not just one factory but multiple ones of multiple parts and multiple chains and 10 to 15 times as many people 30 or 40 years ago. Just like 30 million farmers produce barely enough food to feed a much smaller population in 1900. Two and a half million farmers in a world of agribusiness and massive combines and all this stuff produce far more food that we can then export and then we even need in a world today. We live in a world where manufacturing just does not necessarily lead to bodies being hired so the belief that more money and more income and more output will naturally lead to more hiring is again I think to mis-measure the world and yet all of our funding and all of our spending revolves around these easy conceits of if you simply get the calculation of these numbers correct you will get outputs that you expect. And I think that just continues to break down. You know employment as well we live in this world where there is a chronic fiction of simple synthetic numbers describing complicated multifaceted realities. There's not a single one of these indicators that is not a very basic very loose average. That's all they are. They're just averages of very complicated systems. You know the unemployment rate is essentially that and we act as if there's this one synthetic unemployment rate that's true for California and true for Washington true for Texas and true for New York. And it's not you know we do not meaningfully help ourselves talking about the unemployment rate as a national average that pertains to this system because the variations are so immense by race and by education and by geography. If you are in Nebraska in the past five years the unemployment rate has never been above five percent and if you're in greater Las Vegas the unemployment rate has never been below ten percent. And if you're in Detroit it's rarely been below fifteen and if you're an African-American male without a high school diploma you have an unemployment rate that may be ten percent and an unemployment rate approaching eighty or ninety percent. If you're a college educated woman you have an unemployment rate of four percent. All of these things get lost in the fray of talking about one easy number and then we design policies to lower that number which even the Fed I think is now saying you know what. Designing central policies to lower a national number that itself is not so meaningful probably is not the best use of our time and number after number after number has these limitations. They are large averages for complicated systems and those averages don't even adequately capture the complicated systems we're in because of the changes in the world. And again we can talk about this ad infinitum and probably should. So just to wrap up what I propose in the book is not that we chuck all of the indicators we currently use and then here's a better set that we should use. Now that would probably be a more satisfying answer because it would be an answer. It would probably be easier to latch on you know if we're not going to use this set of indicators let's use this set. We're not going to use that set of numbers let's just invent a new set. The part of the point that I'm trying to make is that any set of really simple numbers to describe multifaceted three dimensional realities will be wrong even if they're wrong for different reasons because all statistics reduce complicated realities to simple numbers. And we're talking a lot these days about income inequality and the problems and the challenges of income inequality which we ought to but it's not clear that we should be talking about those in terms of per capita income because all per capita income is GDP divided by population. If Bill Gates walks into this room and sits down we're all per capita millionaires that tells us nothing about struggles where skills are where acumen meets reality where need meets hope. None of that gets embraced in a simple average like per capita income so to talk about that as a number we ought to be changing doesn't get you to anything meaningful that you should be gotten to. So part of what I suggest in the book is that we live in a world now where the ease with which we can find the information and the data to answer the questions we have has never been greater and the degree to which we cleave to a few simple averages to answer those questions has also never been greater. And then we ought to wean ourselves from our dependence on these numbers and use the ability to tap into the information that we all have at our fingertips to answer the questions we need. Back to the unemployment rate. None of you have an unemployment rate of 6.5%. Most of us have a personal unemployment rate of zero or 100%. So using an unemployment rate as a gauge to kind of decisions you're going to make tells you nothing. Using an employment rate for the area in which you live with the profession and the skills that you have combined with some sense of the income you might earn based on where you live, based on the income you might have. And then using that to frame whether or not you should purchase a home at the price that that home is selling for in the 10 mile radius that those skills and that reality combine. That's a meaningful set of statistics and indicators that you can compile by going on to Zillow and looking at whatever other informational site you have for free, none of which shows up in GDP. The ability to use those sites has no statistical reality within our GDP figures because we have no way of capturing the market output of that process. You go online, you find out what the home prices are in your neighborhood, you find out what the income is for the job that you're trained for, you find out what the prospects are for that. All of what you can do more easily now than you could have done 10 years ago where it might have taken you hours if not days to find that information if you could have found it at all. Referencing GDP and unemployment and national housing statistics to get at that question is essentially meaningless, pointless, unhelpful, distracting. Same thing for a small business, same thing for a large business. You know GDP could go up 3% or down 3%. It won't necessarily matter whether or not Amazon is capturing market share. It won't necessarily matter to the small business that's trying to serve a particular need communally. And even governments need to be much more circumspect about what these numbers can tell us and what they can't in making decisions like a stimulus bill with the conceit that you can calculate these things so precisely that you can get input in and output out rather than much more creative and dynamic ways to solve the problems we have without reference to a statistical framework that's imprisoning us rather than liberating us. So what I call for at the end is bespoke indicators. You know make your indicators to measure. Create the ones that you need to answer the questions you have using the power of big data that we all have at our fingertips much more than anyone had in 1950 or 1970 or 1990 or even recently. And I think that's where you need to learn or we need to all be aware of what these numbers tell us and what they don't and not to live in a world that is unduly described by them and constrained by them as opposed to the world that we could live in that can be liberated by our ability to use the data we have at our fingertips to answer the questions that we all have. So that was 24 minutes instead of 15 or 20 but you know for me that's pretty good. And I guess I will now self-moderate my own question as if I were Steve Clements. But thank you very much for listening. He can come and inject his own view into this. Sir and then here. Yes. I haven't read your book but thank you for the presentation. You said a couple of things. You talked about the 20th century. Of course the Depression and World War II being the cataclysmic events that drove the numbers the 1950 numbers. And then you talked about the recommendations being to create our own questions to answer the questions we need. My question to you is in the absence of the cataclysmic events at a policy level in terms of what the nation needs. There have been some other initiatives over the last couple of years whether it be genuine progress indicators or others to come at the numbers. But in the absence of some agreement about what those big questions are how do you at a nation state level get to the appropriate questions to be answered. So it's a really important point and you know I'm not naive enough. I hope I'm not naive enough to believe that simply indicating an issue and pointing it out will lead to some sort of like you know Capitol Hill sea change where people will the waters will part and people will go. Oh wow that's right we need to spend massive amounts of money. Hi Steve Clements on on revising our statistical framework. And you know there is no question that that at any kind of intense policy level that this is anywhere near the top 10 of most people's list of things that need doing. So what you have is a series of government agencies that are funded at whatever level they're funded you know not optimally but not totally suboptimally to collect the information that we have. Staffed by a lot of people who are highly trained quite intelligent I think very diligent. You know the people I've talked to in the Census Department and the Bureau of Economic Analysis and the BLS and you know the White House are all aware of everything I've just said. It's not like anyone sitting around going oh wow I thought these numbers were perfect and we're articulating reality exactly as they should. But you know without the money and the mandate they're going to keep keeping these numbers within the framework they do. And then people externally will do what human beings do which is they'll come up with better what they think are better ways of doing it. And you know you have the UN which 20 years ago came up with the UN development and the development index. You've got happiness indicators you've got different ways of measuring inflation you've got cheaper ways of measuring inflation. There are some economists at MIT who came up with a bot that scans all prices real time online as a way of getting price volatility. Weirdly enough it doesn't diverge so greatly from official inflation it's just a lot cheaper. So I don't expect in the real world anything to happen other than has been happening. You know I'm not the most astute gauge of really really you know topics that have action items attached to them. Like if it was it would have been called like you know the coming number Armageddon or something like that. But it's more of a way of saying as we each disseminate and try to navigate the world and that's true if you're like in government thinking about policy and it's true if you're in business thinking about what your business strategy should be and it's true I think individually as we navigate our lives to be much more circumspect about what any of these numbers can tell us and much more aware of what they're designed to do. Because whether it's the Happiness Index or UNDP or Genuine Progress all these gauges have to make choices about what you're going to rank above something else. And those choices will matter greatly to what the number says and is probably always going to constitute a set of limitations. So you know I'd love to say look we're all going to figure this out and what it would mean. Because what it would mean to be a much more kind of case by case complicated iteration of how do we solve this problem. And I'm not sure human beings are either desirous or quite yet capable of encountering reality in that particular way. Hello Steve. Hello everyone. Good to see you Zach. Good to see you. Good to see you. This is like the guy who jumped to the front of the line with the little pass at the airport when everyone else is waiting for questions. For those of you who don't know and hopefully they made a comment before I am. I'm Steve Plemmons. I'm a senior fellow here at New America Foundation was a founder of the American Strategy Program. I'm an editor at large of both National Journal and the Atlantic. And I am I basically just to be quite honest about it because it made me think about the coming age and how we're so unprepared for it. Whether it's data or our gadgets but I had put in today's forum when I was in Munich Germany at the Munich Security Conference and failed to account for the fact that it would change the time zone here. And had obligated myself to help open the ARPA e-conference today over in National Harbor where I interviewed two really fascinating guys one from BASF and the other from Waste Management looking at sort of avant-garde kind of the new horizon in energy. And when you really think about what's coming and you think about how transformed our world is going to look five, ten years from now and you can measure that by how different it is from today. The things that I just want to put on my own views on the table about your fascinating book and treatment is that there's a revolution of change coming and yet the resilience of our data and statistics to remain rather unchanged is phenomenal. And what many people here may not know is that the roots of this institution in part were in an article that Ted Halstead, our founding CEO wrote years ago for the cover of the Atlantic magazine of all places called if America so, if the GDP is so up why is America so down? And you know if I were to rename your book which is a well-named book nonetheless I'd call it Data and the Soul. Sort of looking at you know how do you look at these kind of. I would rename my book. What would be the other contending names? That would be good. I was going to do the Matrix Economy. The Matrix Economy my book will sell more. Data and the Soul would have been good. I guess the first thing when I was sitting over in this ARPA-E conference I was thinking how should we start out. And it does, I guess I just want to say you talk about the inadequacy of so many different measures. So why can't Zach Carabelle on his own go out and create a firm that creates your own set of data to more appropriately measure things? Wouldn't there be a great business there because there would be an arbitrage between. It would certainly be a great business. And I mean in the sense of helping people navigate their own lives with the data available. I mean the one really good thing and we also get to other questions about the current statistical framework governmentally and in private industry is the ways in which they have become the justification for collecting vast amounts of really useful data. So in a weird way it's like you wouldn't want to do without the 50 pages of tables and data that simply come affixed to every single one of these numbers. You just would want to do without the number and the press release and then you'd have to help people navigate. What do I do with this stuff? The unemployment data on underemployed the frustrated. It's all there, it's just that's not the number we're fed. And it keeps getting better. Because the one thing that these agencies will do and do really well and they can do within their mandate is they can improve their data collection. They can expand it if it's cost effective and increasingly because of technology. You can expand data collection. And you can create some really interesting correlations and stuff underneath. It's just you can't get out of the framework and nobody wants to deal with the nuts and bolts. So yes you could definitely and all of you are welcome to do so. You could create a really interesting business. It's almost like helping people data navigate and ignore. And that isn't what big data is going to allow us to do. And so I think what you will get in time is not that the statistical framework will really change much because I see no mandate to do so. But it will get subsumed. Its prominence will lessen in the face of the noise of everything else in a productive way. Before I know you start questions with this but the other question I had driving racing over here to think about is when you look back at the history of data and these statistics, these measures of the world that were there. Are there any that went bankrupt that just stopped being used? No. I think that's a little bit like it's easier to start government programs than it is to kill them. Once these are embedded and once there's a time series and once someone has made policy based on it. We talked briefly and I'm going to have an article in the Washington Post other this Saturday the next on the front page of the business about the CBO. Another really interesting recent data trap or statistic trap. Where all our spending has to be funneled through a very narrow statistical aperture. I don't know of any one of these that have, if anything they've gone from the public sector they've been acquired by the private sector and then they've proliferated even more because companies will pay for this to justify. So the very term the leading indicators was a series of numbers kept by the Commerce Department that was then bought by the conference board that is then disseminated but also sold by the conference board to businesses as a consumer sentiment. So there may be one but I did not find, I find only proliferation of numbers. And then finally does it lead us to make bad decisions because the other data point that I've spent some time writing a little bit about lately was the difference between private sector debt and an economy and public sector debt. And we're entirely trained and beaten over the head every day on how bad government debt is and focusing on government debt levels. And when the taxpayer and the voter gets angry whether that voters in Spain or Italy or Japan it's about government debt. But very often if you go back and look that government debt spiked because of the problems created over a private sector debt crisis. If we had a private sector debt level out there every day would that change how people thought about who's accountable or who's responsible? So part of my point in this in which I made a little bit before is again no matter what that were we don't know enough to make the kind of conclusions we do about where the breakpoints are or are not. We certainly don't know at a public level so I do touch on this in the book. There was in the aftermath of the recession all the government spending also a lot of debate about what level of public sector debt. What would the ratio be where where growth becomes imperiled by too much public debt. Obviously there was a famous you know article about this and it became part of that discussion. My issue with this was always that you know until the 1930s or 40s there was no clear record keeping of either public sector or private sector debt. That to create this information before the 1950s you do a lot of regression analysis you go back you fill it in you you try to make it fit a framework. But for the most part we just don't know enough about any of this to speak with certainty about what we think ratios are that are going to be damaging or not. And it's the it's kind of the arrogance of statistics not of statisticians it's the arrogance of we have enough data points to make the kind of conclusions we do. You know we have enough to make some interesting observations about what may or may not be true. But unless you can show me that everything is apples apples apples you know and that the China debt levels now are the same kind of debt in the same kind of way as US debt levels in the 1920s that are the same kind of debt as Argentina. You know then maybe we could come up with something but I think what these what these numbers when they're used politically and used to some degree academically they create a false sense of being able to know. And the one of the things historians are good at you know and I was trained as an historian said I history was not theoretical enough in many ways. What historians are very good at is saying there is such vast difference in every single thing that you have to be incredibly careful about what generalizations you make. You know economists and political scientists go to the other extreme which is what's the pattern that's conclusive that we can draw and then politicians want to prove that they're just in sync with that pattern. So I don't think we know enough. I don't think we know enough about private debt or public debt to know for sure about any of it other than the fact that if you've got a creditor and you've got an amount due and you don't have enough money to pay there's probably going to be a problem. Other than that I don't know. We'll come back anyway let's open them in the back right right here. So I'm Dan Estia professor at Yale and I've spent. Dan knows data better than. Well I've spent 15 years working to develop an environmental performance index. And so I appreciate and really think you're right on the mark in terms of your critique of our current set of indicators. And I guess I want to just take issue with your diagnosis or prescription beyond that of where to go. So you're highlighting the fact that there's a lot of embedded assumptions in these numbers that are not often brought to light. But your conclusion was we should all create bespoke indicators. And I guess that might work for many people but not people in this city in particular. It would work if you are trying to buy a house and figure out whether the timing is right. But in this city the use of data is really to drive policy and fundamental to policy is to be able to say something not just about one particular example of something that fits against others that are similarly situated. So the conclusion I would come to is not bespoke indicators. And I've seen this in ranking countries every country complains and says oh you didn't treat us fairly. We're different from everybody else. If you and this pick an example you're the desert countries of the Middle East. We shouldn't rank low on sustainability of water we don't have water. I said what that's the definition of sustainability is if you don't have it you have a problem. So you know on their bespoke measures and we've set this up so you can define your own analysis they look great because they don't care about availability of water. But here's what I would say why not focus instead on sensitivity analysis highlighting the assumptions that matter and demonstrating how much they matter as opposed to giving everyone a chance to make their own numbers up. Which adds you out of the business of being able to identify leaders and laggards and identifying best practices. But I would say you know the index you're developing and some of the environmental you know the newer sustainability indices are what I would call bespoke they're just macro bespoke. They are designed to address a certain specific set of big policy problems with the awareness that yes you need some then and now you need some basis of comparison. But in many ways you're deciding which variables make sense for that set of policies. That's definitely true in the sustainability you know it's definitely true in the global reporting initiative which I think you could say goes way too far in the data side and not enough in the OK what do we do with 10,000 variables that we've now measured globally. So you're entirely right to take issue with the way I talk about it and I'm still you know I am trying in the way I talk about this to calibrate that somewhat unsatisfying answer to different areas of society. So there's the individual right part of the part of the message is just our individualized are almost never meaningfully dictated by unemployment rates and GDP and inflation. One because they're not meant to help us to because they don't and three because we shouldn't use them to guide ourselves. Even for businesses by the way I think the bespoke carries through in that one thing I learned as an investor is you know the macro numbers could look really bad or really good and your business could either be thriving or screwed regardless. Because you're not you're not you're not strategizing correctly given the nature of your metrics. You're totally right that a kind of that governmental macro international level you need data to prove it. But again I would say that the kind of stuff you're talking about is by definition bespoke in that you're creating a framework of data analysis and numbers to help guide and answer a set of very particularly defined problems that you have. Can I just ask Dan a question. Dan is for those of you who don't know is one of the smartest people in environmental policy that I know his wife happens to be a terrific member of Congress. And I'm not to have you speak for her but the question is you know when I worked in the Senate we struggled with these data issues a lot. It was it was a big issue when you look at what was coming out of the BLS you looked at things like CPI you looked at a whole variety of issues on whether the calipers were right or whether they were distorting because the function comes from a public policy. The policy perspective is were you were you essentially trying to prescribe things that were going to be you know they weren't in touch with actually the effects that we were having. And I'm interested in today because you must have helped with your wife's campaign. Is this is this problem since then Congress today. Do they know they've got crappy numbers. Well I would say and I don't mean to dominate the conversation. It's actually curious how data disputes have become big business in terms of the congressional battles are being fought out. And that does I think mean that Zach's timing is excellent. I mean I can't remember a week in the last 10 years where our biggest battle is over a CBO number related to in this case what's the impact of a minimum wage. And actually what's curious in this case is the problem is that they did what I suggested that they should do which is they gave a range from zero to a million lost jobs. That's just too big a range. And in effect the people who wanted to argue there's no problem are fixing on the low end of the spectrum people want to argue there's a big problem or fixing on the high end. And that's a problem too if you don't actually use the sensitivity analysis to highlight the assumptions that are careful about it. So I think there is a sense in Congress that the data is now politicized and one has to be careful because we no longer have and that's what I was really pushing Zach on any common fact foundation on which to frame policy. And then we should move on but I mean on those I would push back even further and I would say we should not certainly at a public policy level be making even with sensitivity analysis future predictions about outcomes where there is both far too many variables and far too little history to make any of those conclusions. I mean I personally do not think that any human institution has enough data about the effect of a raise of a minimum wage the last time it happened you know it's happened twice in 25 years. Whereby we could make any conclusions numerically at all that have any weight and that we shouldn't be in the business you should not be in the business of forecasting numbers that are inherently on forecast. Why is that though when we are in an era where computational capacity and speed and association are so much more greater than any thought. Why can't we throw that in a super computer and come out with a better number. I just think too many variables that we do not yet understand the relationship between. Okay right up here in the front and then did you have a question. Right here. There's a microphone. Mike McDonald US resilience system and I guess first of all we measure a lot of mission critical functions at the community level and I think there's a paradigm shift that's taking place because not only do you measure it at the national level. Now people actually within their communities can decide whether they're succeeding or failing and I think that the numbers that you're talking about actually do have great utility but the utility is mythic. If you don't want the facts to screw up your ideology they're very useful because you can argue one way or the other. And what we found in the disaster areas after Sandy for example the mayor hired a paramilitary group to collect the data and then when the community said well are you going to share the data with us that he said yes we'll share it but they never did. So this issue of public versus private information is extremely important and it seems to me in this new paradigm shift. The key is having publicly available data that's actually meaningful that tells the community whether they're declining or whether they're emerging and could they potentially collapse. No I mean I totally agree with that. I met some 25 year old who moved to Detroit and had set up a database of every parcel. So if you wanted to know who had bought the land in your neighborhood and if you were living there and you wanted to know what was totally sinking or swimming or who bought the parcel next to you they started creating a kind of a Google map view where you could click on. And all the records were public but every time someone went to City Hall prior to the past year and asked for the public records they'd say well you've got to come back tomorrow or you know I mean it's back to this whole big data of groups now creating meaningful information for the kind of questions you have that would not have been the case. I guess even five years ago. Yes right here in the front. I'm going to bring you the microphone right here. I'm Jason Stern of Bradley Communications. I work with a lot of small companies through associations and many of the small businesses don't know what statistics are really real. Do you have some suggestions as to what they should be referring to and which would be very constructive for the folks who are on Main Street. Yeah I mean again I think it really depends on what is your business. Where are you located. What is your supply chain cost. I mean again what would an example be. I mean if you're probably we probably wouldn't open a small town hardware store on Main Street because you know that would be inherently silly but I mean if you were starting a small travel agency right. You know how much are people spending on tourism in the area. What's the local income. What's the unemployment rate. How do you reach them. Those kind of metrics I think it's all dependent on what's your business. What's the local environment. What are the metrics that you can find relatively easily to answer those. None of what you will get from kind of you know National Association of Travel Industry or none of what you'll get by overnight lending rates because it you know it doesn't really matter what overnight lending rates. Or interbank rates or 30 day rates or two year rates. It matters whether or not in the industry you're in banks are lending money in the area that you live. Again I think this is more of a processing like you're talking about where you kind of you go in you do an audit with somebody about what do you need to know. And then you literally go and you you create the database for it because I don't. From a bottom up way I mean OJ runs the helps run the Wharton Business Economic Outlook Forum here in Washington right. The interesting to sort of you know to sort of survey those businesses as to what they feel they they they need to traverse their environment you know to sort of look at what those those sort of broader things are. But I think on top of that they do look I mean I think part of confidence you know part of business confidence is shaped by these mega data items like you know what they see is happening with debt what they see is happening. And I remember you know and things like the budget shutdown and the debt talks we had at the Atlantic a lot of forums with small business people I think you participated. You asked them what is their biggest worry. They would say that right because they were obsessed with that issue. They were so convinced that it becomes tangible in the business behavior and decisions that they're making. This is another thing about limits of numbers right so I have a whole chapter on consumer confidence which has become suffused with media. And it's it's one of our kind of leading indicators that we're all aware of and yet as a predictive number people feeling incredibly anxious about the state of the world says almost nothing about future spending patterns and people feeling incredibly hopeful about the state of the world says almost nothing about future spending patterns with the exception of buying a car. And the only the only thing could be that you know cars require a little more lead time it's seen as a bigger purchase. Same thing with business confidence all the surveys we have a manufacturing regional and domestic come out monthly. Half of those surveys of whether it's ISM or other stuff that we hear about are bus based on asking business owners do they feel orders are going to pick up in the next three to six months. Which is a which is a sentiment snapshot. Now I don't think it tells you nothing but I also think the only thing that has ever been proven to dictate future economic patterns. Are how much money people have in pocket. You know businesses are a little bit different in that they could be making more much sector debt levels matter. But it's how much money you have in pocket. It's not how much if you know if you know one what's in your pocket because people it's debt levels are not high levels of debt do not preclude spending high levels of interest rate. Is that real quick. Robert shredded president of international investor one time I served on a research institute vice president. We were always amazed how as hard as we would try to be objective about our analysis. It was twisted and turned of course according to who was using our numbers. But that's what my question comes to you. It sounds like you're getting so pessimistic about some of these numbers that they're almost meaningless. And yet we need some international benchmarks and especially societies need to be able to say how well our leaders doing their vis-a-vis other leaderships in the world. Are we improving our standards of living our health care whatever it might be. What do you think about international rating systems where they try to at least rate nations. Can I piggyback on that. I just say the OECD sort of does that but but what so the question is beyond what Robert saying what's wrong with the OECD ratings if something or how can they be tightened up or made more relevant. So should there be a happiness. I agree that I do sound pessimistic about these and I am. I'm not pessimistic about again about the data that underlies them. I mean even the government doesn't know what's wrong with the OECD ratings if something or how can they be tightened up or made more relevant. So should there be a happiness. I agree that I do sound pessimistic about these and I am. I'm not pessimistic about again about the data that underlies them. I mean even things like GDP ranking of nations is highly problematic because you can make an argument as many people have that in many ways the aggregate output of the United States is dramatically understated by the numbers because of intellectual property which we've only started recently factoring in. Because the trade numbers treat a final country of origin as being a real thing rather than as the OECD has done show the ways in which almost nothing is made anywhere but the value add of what's in a lot of things is flowing to the U.S. Rather disproportionately but it doesn't look that way optically at least in these numbers. And there are almost no international numbers understood as measuring an international system. A lot of national numbers divided added and divided. There are almost no international statistics because nobody is paid to do so. Even the U.N. depends almost entirely on the national reporting of member nations in order to create international numbers with the exception of the World Health Organization and to some degree UNDP which have been able to trace transnational patterns statistically a little bit better. So I would say look we absolutely should have better international information. I don't know who's going to pay for that given the degree to which many of these like prices right. Prices are increasingly set as is the cost of capital by non-nation state phenomenon but we have no way of really getting it you know what international price movements are. I think you can start to get there with a lot of the data you have and a lot of these things. I'm very cautious about things like GDP rankings where you're measuring an economy like the United States against an economy like Zimbabwe with a similar output number that I think is actually measuring very different things. Or climate you know carbon emissions or things like that I assume. You know Nobo Tanaka the former director of the International Energy Agency you know sort of used to bitch a lot about Chinese economic data. You know China is not a member of the IEA but he would basically look at their reported energy use numbers and compare them to GDP levels that they were reporting economic growth levels and said these don't fit. And he would try to get people and you know it just sort of is an unresolved matter for discussion but it is interesting. I think this gentleman had a question. Thank you. Yali Friedman scientific American. I wonder if you're not asking a bit much from numbers and from people. I do a ranking of 54 countries globally on biotechnology innovation capacity and it comes down from about 60 different metrics. It is built into one number. And I'm very transparent about how it's all calculated but guess what people focus on. Just the final number. And they completely ignore the methodologies. In parallel if you take a look at biotechnology one of the one of the things which people often say in agricultural biotechnology is that if you can just educate people more then they'll understand how safe these things are. So I was in a forum and I said how many people here can tell me how VCR works and nobody knows but we all use them. And so you know the the Obama jobless numbers those were only put forward for political objectives and people like single numbers. You can't argue saying well technically there's only seven seven cases and what have you. It just sounds like a weak argument. And so are you asking too much of people and of numbers. Well look in the real world I'm clearly asking too much of people and numbers but in the ideal world of I get to put out ideas in a book that I think are important for us to integrate into our own lives that I think would more meaningfully shape the world we're living in. You know in almost a philosophical way right I get to make that injunction even while I recognize you know the effect of books are like ripples and a pebbles in a pond. You know the ripples are going to go and at some point they're going to hit a hither shore but how they're going to impact things unless you're a really black and white ideologue you know you don't you're never going to know. So I respect greatly that there are things that I'm suggesting in this that are both unrealistic and unlikely and and they're also legitimate places of disagreement. I mean but your your index of biotech is a perfect kind of bespoke indicator thing it's like you're trying to get at one element of what's going on in illustrated numbers. I'm not saying we shouldn't do that. I am saying and I and one where I don't think I'm expecting too much of people is that we have only very recently become highly dependent on a very small range of statistics to shape vast amounts of public spending and private investment. And that world did not exist when the interstate highway bill was passed or when you know lots of initiatives that we undertook didn't have to go through a CBO scoring system and if it did it would never have been passed. So I think that there is some element of realism in that we should step away from something that happened very recently in the way in which we consider future outcomes both publicly and privately. And that this statistical framework is much more recently rigidly imposed than we think given how few options we think we have. Before I go to this gentleman I just a fun data point paying tribute to Scientific American. If any of you see the fun but not very good movie RoboCop which is pretends to be about 2028. Scientific American and the Atlantic. Scientific American and the Atlantic are still existing as magazines in that era. Yes but the Atlantic has the voice of the past and Scientific American has the voice of the future although the voice of the past is the one that prevails humanistically at the end so who knows. Which yeah we had nothing to do. Anyway this gentleman right here. What's it. Not that I think they I mean we all found out later but I think that we did do it. We'll come back to you if you like. You had your hand up as well. Oh no. I was one Bruce Wallman I was wondering if you're trying to explain a problem that's mostly political rather than a problem with the data. I mean I think there's already a rather enormous industry in this country. Anyone that objects to any of the indexes or statistics out there and has the money can easily hire a firm to come up with an alternative which will come with a different conclusion. And the public is completely confused because there's thousands of these things out there so there's no real. Would you say that on unemployment. Well level. I mean I would say that's a counterpoint. Figures that we do see are out there as I think someone else said for political marketing reasons not nobody is really being steered by these numbers. These are out there as a way to explain it to people. And just looking at the employment numbers it's interesting how they shift the interpretations as the political forces want to have another spin on it. So I think the problem is really not that we don't have all these bespoke. Gathering of information statistics because I think most people in their own experience like you're talking about do use bespoke numbers or indexes and things like that. And it's usually things that they don't aren't in their daily experience whatever where they're looking for these outside sources. And it's the same political problems that everything else we have. Great question in the sense that you know just to piggyback I mean it political also means winners and losers. So you know aren't you really just fighting between aren't isn't the fight about winners and losers and thus adjusting them. And thus it's fundamentally a corrupt measuring of our system anyway. Yeah I mean look I we're all going to be anecdotal but the degree to which we think these things shape things or not. I try to point to the increasing ways in which again through CBO through government spending that a very limited set of numbers now do in fact constrain vast amounts of public spending. Not just you know public perception but actual how we can spend or whether we can spend you know makes it much harder to invest much easier to spend. I talk to a lot of audiences you know throughout the country both in the financial world and also colleges you know college kids who are easily buffeted by whatever the news du jour is are certainly their sense of what's possible is impacted by a pretty basic digestion of a pretty limited set of numbers. You know people hear about the unemployment rate and they hear about there's a big unemployment problem for the youth as least as measured somehow. And then that creates a good degree of both anxiety and past choices that are different. But look I don't think I can prove that. So who determines that the use of the media. Right now the data is something that I just think look the media have become in the past 20 years because of a need to fill space. There is a limited number of numbers that come out regularly that become a source of conversation that shaped the economy. I've been involved in the unemployment figure debate a little bit on the edge writing about it. It's not a million people on a list but you know Leo Henry who has been very interested in labor issues has been for a long time hammering very directly on journalists TV journalists broadcast journalists economic writers on how the stated unemployment figure is a complete illusion and that beneath that. So you know what if you go back and you track you know what what had come after Leo began hammering this out years ago. More and more people are beginning to raise this issue more and more people using real unemployment as a figure and they're coming in and people seem to be interested. So there's an incremental and slow transition. I assume that that stated official unemployment figure is going to go out of out of business at some point because it will look so inane compared to other dimensions. As we move into a more mechanized world with more robotics more whatever which is putting real impact on labor rates. That notion of frustrated workers and and what what's happening on Main Street is is going to be more in demand. So maybe I'm wrong maybe it will you know be more resilient but it's just I don't think it's all a function of just marketing and kind of a political play but there are winners and losers in it. But I think that that the unemployment one is the one I've seen where you see a transition underway. And I think part of this is back to you know your point I think it's complicated. I think it requires more words you know it requires more explanation and it cuts you know multiple fashions. Each one of these inflation to just you know it's not meant to measure whether or not you have enough money to buy what you want or need. It's like that's not the nature of that but that's how we discuss it. And I would hope some of these things change. I think we have one more. Let me just take these two here and we'll see how these two go. Let's take them both together. Hi my name is Oliver Grimame a correspondent in DC for an Austrian newspaper called the press. I've been interested in your view on the paradox that on the one hand we have the huge amounts of data big data and more granular data or whatever. At the same time there seems to be an increasing inability of the political actors to act upon them. I mean even somebody like former Mayor Bloomberg who makes very active views of these things gets lost in a silly campaign to tell people in which size of bottles they should buy their soda. But he doesn't really have a solution for the increase in rents the increase in income inequality in New York and so forth you know. Do you see any relationship with the amount of data? His answer would be no answer yet. He had another term. But I mean the question is serious. Do you see a relationship between this huge amount of data and the inability to act upon it or is it just correlation. We're going to take these together. This gentleman back here and then we'll jump to you. I'd like to get back to the unemployment question. The Gallup organization has developed the ability to take worldwide questionnaires. And a couple of years ago they did that and they asked everybody a fairly simple set of questions. Would you like a job equivalent to Walmart Greeter 30 hours a week a paycheck on Friday no benefits nothing fancy. And the numbers they came up with where there are 7 billion people in the world about 5 billion of them are of working age. About 3 of them responded that they would like such a job. 3 people or 3 billion. 3 billion. I understand the confusion but 3 billion. And there are 1.2 billion such jobs in the world. So more than half the people are really unemployed. The way that's gotten around is lots of countries when they take unemployment statistics. They allow somebody who is a dumpster diver to they he's he's listed as a self employed dumpster diver. Okay great. Let me just take this quick this gentleman's quick comment question. We have the microphone right here. You've talked a lot about resilience and US resilience system is trying to measure resilience and vulnerability. So we've come up with a common core data set. If you look at the insurance industry the insurance industry has just scrapped actuarial data and now they're using simulations because the actuarial data in a time of climate change isn't meaningful anymore. So what if we thought about a green field approach to this given what we think is coming now which is disruptive. Could we possibly come up with a common core data set that would actually be meaningful to communities and societies that could aggregate up from the very local level that where you can really see whether it's mattering or not. So I think this is a very interesting point. It kind of segues with what you're saying about about big data. So there are two things and we're at the end. One is a green field approach is a really good idea. The problem is what a lot of this demands if you were going to liberate yourselves from a set of frameworks that have some utility but don't work as well is there doesn't need to be a degree of trust right. There's because basically it's it's do you believe whoever's in a position of shaping things is going to act with integrity to shape them constructively using the information at hand to do so. And there is throughout all this kind of data creation and manipulation a good degree of legitimate tension and suspicion that numbers particularly officially disseminate ones are just a way with that government's control reality. And that we didn't get into this in this conversation but it's an element of it and that to liberate and allow for the creation of kind of newer and use that data constructively is an act of trust that whoever's going to do that. That is doing so constructively at least the advantage of the numbers that we have that are legacy numbers is that they are somewhat less susceptible to those questions of manipulation. If you believe government's out to control you you're going to have a conspiracy theory about inflation and employment anyway. But you know most countries in the world are not like Argentina where Kirchner actually did fire the entire statistical agency responsible for compiling inflation in 2009 because the numbers were higher than they wanted for their for their reelection. Most countries don't do that in fact and if they do they get thrown out of the WTO and they get thrown out of the IMF and that's a problem. So I think that's something to think about which is it would require us to believe that we are collectively engaged in solving problems to be really green field about our numbers because it would mean that you're using the data at hand with some level of integrity. And I think that's a lot of the problem behind it is you know that trust is in short supply and it would probably obviously be better if it were in greater supply but that's a whole other. You know just to include what I think is the terrific contribution that your book makes is it comes back and shakes the assumptions of these statistics and data that we do and then really throws open the notion of should we not rethink a number of these things. Should we don't we have trends in place should we worry about different things than we've been been worrying about you know in this question about green field approaches you know I'm one who sort of probably am a little bit more along the line where you are. I think there's a great opportunity there because you know when you see us able to simulate the landing of a rover on Mars which I sat down with some of the people who did that math and began thinking about it. You think about complex systems well that may not account for everything in the human system there's a lot or you sit down with Facebook marketers and people that are taking vast amounts of data and they know what I want before I get there and they're feeding it to me in an individual basis. You know and then you come down here to the US government figure well they're not quite there yet but the capacity may exist to begin looking at things differently. Talk about city mayors Mike Bloomberg. Mike Bloomberg in my book was one of the most interesting templates for a modern mayor in the sense of forcing and compelling data resistant divisions within the New York the city of New York to begin thinking more seriously about data input output outcomes. And Rahm Emanuel is today in Chicago problem and Rahm this may be a silly example and not as complex we are but Rahm you know was out there trying to figure out how do you I save you know kindergarten education or early start education. And needed to find ways to save money and one of the things he looked at was the contrast between how his dump trucks in Chicago were operating and how UPS trucks were operating and got the grid. They got rid of right turns or certain right turns save 80 million bucks. And it's interesting to look at these transformational changes that data are bringing and that is a step away from the larger thing of saying OK we need to change the measures or change the you know really the metaphorical notion of what we think is important to measure society and I think you do a great job in opening that up so congratulations to Zach Carabelle. Let's give him a round of applause for making us think this afternoon.