 Welcome back everybody and good afternoon. I know a number of people have joined us and there are some watching online And so I wanted to say again that I am Susan Collins the Joan and Sanford Wildein of the Gerald R Ford School of Public Policy And we're delighted to have all of you here with us on the occasion of both the 100th anniversary of the Federal Reserve Opening its doors for business and also the 100th anniversary of public policy at the University of Michigan the Ford School Centennial Well today's keynote speaker is Betsy Stevenson who is a highly accomplished economist who is currently on leave from the University of Michigan Serving as a member of President Obama's Council of Economic Advisors and I'm really proud and honored to have her as a member of the Ford School faculty She previously served as Chief Economist at the Labor Department and a research associate with the National Bureau of Economic Research An affiliate there her widely read research and public discussions on the impact of public policies on the labor market have been well recognized And have had a large policy impact as well and I know that's one of the topics that she is spending some of her time at the CEA looking at And she has focused extensively on the role of data, subjective well-being and in a variety of other areas as well for how we think about public policy Her topic today as you'll learn more about in just a few moments is the role of research for the Council of Economic Advisors And that seems a particularly appropriate topic given the broader context of our conference I did want to mention that we have set aside a little bit of time for questions following her remarks And if you have questions we do have staff who can bring you a microphone please wait for the microphone And for those of you who are watching online I invite you to tweet your questions please use the hashtag Ford School Gramlich And one of our staff will read your questions to our speaker And with no further ado it is my great pleasure to invite Betsy Stevenson to the podium Thank you so much it is a tremendous pleasure to be here And I was just saying to Marina she was asking how I was liking the world of the Council of Economic Advisors And it is one of the most thrilling exciting places I've ever been and I think part of that is the fact that I have to move so rapidly from very very disparate issues And that is both its challenge and its excitement So what I wanted to do was to take this talk as a chance to step back a little bit and think about the role of the Council of Economic Advisors And it gave me an excuse to read what a lot of former CEA chairs have written about the Council of Economic Advisors And you know I thought after I'd read it that it was both very helpful and I was somewhat glad I hadn't read it right before I came in Because I think it allowed me to go in with fresh thinking But I think there's some ways in which they all paint the same picture and without having read their pieces I would have said the exact same things that they said In the sense that they also paint the same picture of the Council as providing the President and senior officials with objective Adviced based on the principles and tools of economics The other thing I found really interesting is that many of them mentioned the fact that the Council of Economic Advisors Looks after the national interest by considering how policy impacts societal welfare in a process dominated by the views of interests Usually called special interests and many remarked how the Council of Economic Advisors special interest was the national interest What I found interesting about this is that it helped me understand actually why I sometimes face challenges in my job that I hadn't quite recognized before Because it always seems obvious to me that the national interest is the interest And I am learning over time that there are many interests but I fully stand behind the job of the Council of Economic Advisors As trying to figure out what is really the best thing to do in terms of maximizing societal welfare So that description of the Council remains true nearly 20 years later And of course many of you I know have actually spent time at the Council So the structure has really stayed the same You know the Council is comprised of a chair and two members, senior staff of senior economists, professional statisticians, a director of forecasting and many junior economists The Council members and senior economists typically come for a short spell in government from academia or a think tank A model that former chair Joe Stiglitz described as citizen bureaucrats While sometimes clunky this citizen bureaucrat model does facilitate the Council's role as an effective conduit between the research community and the policymaking process And the Council continues to evaluate decisions in terms of the overall impact on welfare So what has changed? I think the biggest thing that's changed is technology And that's not just a change in the Council, rather that's the broader change that we've seen in our profession And the Council has of course reflected this change The economics profession is in the midst of a revolution, we're becoming a more empirical field Rather than just theorize about the effects of policy, we turn to hard data to tell us about how the world really works This revolution is the result of a technological change that's given us an explosion of data And importantly the computing power with which to analyze it In short there are now opportunities to analyze public policy in a way that our predecessors could have only dreamed about These opportunities make academic research more connected to reality and policy In turn our deeper connection to reality makes us as economists more accessible, more relevant and actually more central to the policy debate Implicit in this shift is the hope that empirical evidence will improve our understanding of government programs and the economy Now while this change is impacting all of economics, I think it's a particular boom for microeconomics And in reading many of the reviews of what the Council has done, microe has sort of been, shall we say, the Cinderella in the family And I say Cinderella on purpose as in the sort of ignored stepchild who I think is rising in prominence with our ability to do more analysis Now hopefully many of you will disagree with that And say that the former chairs had a bias when they wrote up their reviews of the Council of Economic Advisors But I do think that what definitely has occurred is a notable shift in the amount of applied work that we're doing The work and the staff of the Council has shifted along with the economics profession Our junior economists are no longer only advanced graduate students but include computer savvy undergraduate economics majors Following the full-time RA model that many empirically oriented economists are using to crunch literally millions of data points And our staff economists have noted roofily that they are expected to conduct research at the Council that a few decades ago would have easily earned them their dissertation And now they're expected to do it in a few short weeks, a month or two if they're lucky There also, this type of sophisticated analysis of large-scale data sets in brief time is I think one of the advantages for graduate students of spending time at the Council Not only are you exposed to an enormous range of ideas but you actually learn how to get them done and get them done quickly The explosion of empirical economics has also increased the number of competing analysis that are relevant to any particular public policy problem And this explosion represents both an opportunity and a caution for policymakers in the public The caution is becoming obvious. There are conflicting studies on many important policy issues And I think we see this across a range of fields, not just economics, in a field where I know very little health I'm still confused about whether coffee is good for me or not good for me given the number of studies that come out And so the most important advice for policymakers is to understand that a given research result should not be thought of as either right or wrong But as a guide that should shift one's thinking The question is how far? We're informal basians and to us the CEA, the question of how far to shift our views depends on both the precision of our priors That is how much we already knew about the issues and the precision of the new signal which is largely shaped by your interpretation of the credibility of the research design The challenge for the council and for others trying to inform policymakers about research is that policymakers tend not to be basians The council's job is therefore to bring Bayesian updating to policymakers The opportunity is that the proliferation of studies truly does improve the evidence base available to us Making it easier for us to make the case that policy should be guided by robust research For the council of economic advisors the explosion in empirical research and the tools available to scholars has deepened our role I believe as a conduit of research to White House senior advisors into the president So let me turn now to giving you some concrete examples of what CEA does and how I think this revolution has changed or caused us to tweak a little bit about what we do So the CEA as many of you know is required by statute to write an economic report of the president each year And what it says in the statute is that that report should set forth current and foreseeable trends in the levels of employment production and purchasing power And a program for carrying out the policy to promote conditions under which there will be afforded useful employment for those able-willing and seeking to work So in most years that I have looked through the ERP has attempted to do more than that and has attempted to be broader and to speak more to the broader policy debate that is going on And I think one of the things that's changed is that the rapid timeline under which things are moving and the technology available to us means that we can release reports on a more timely basis and feed into the policy making process In a way that provides more timely analysis on economic policy So we've recently begun to reconsider how we should best produce the economic report And if you look at the most the 2014 economic report of the president you'll notice that several of the chapters have been previously released as reports that fed into a more timely aspect of where the policy cycle was I think the ERP to me serves two really important roles One is it provides a record for what the administration was thinking and doing at the time But it also provides, it also feeds into real time policy process and this more continuous model of releasing chapters I believe better serves that latter purpose So to give you an example of one of those chapters this past January marked the 50th anniversary of the announcement of the war on poverty Something Sheldon made sure I understood clearly as I went to the council this past summer And walking into the council with a president who was interested in thinking about poverty and inequality We saw one of our jobs is making sure that senior advisors understood that there was a broader historical framework to think about And that there would be questions naturally raised as we got closer to that anniversary about how good of a job has our government programs done Now the challenge that we faced is you'd think that 50 years later it would be an easy question to answer Whether or not our government policies have reduced or not succeeded in reducing poverty It turns out that is not actually an easy question to answer It's not easy for what is either a simple or arcane perhaps reason which is that our official poverty measure excludes almost every poverty fighting program that we have So if you look at the official poverty rate and use that as a tool to measure or as a statistic to measure our tools in fighting poverty You're failing to understand that that measure in fact excludes our very tools So CEA what we really wanted was something that seemed like a simple concept What would poverty be if there was no government market based poverty and what is poverty once we put the government programs on top And we turned to our staff we worked with our senior economists we said how are we going to answer this question Very very smart senior economist who graduated from the Ford school Jordan Matsudaira said I cannot answer this question on my own in the next couple of months This is a big idea question and he went out and canvassed and talked to academics who worked on this issue And he discovered a team of academics who were trying to extend the supplemental poverty measure which does more to include our policy programs as measured back in time But Jordan said to them that's great I'm really glad you're doing this work The problem is if we want to say something about how effective our programs are over time supplemental poverty measures quasi-relative So as you extend it back in time you're shifting the goalpost and if I use your measure to compare how poverty as measured by the SPM is today compared to 50 years ago I'm still going to get something that doesn't tell me did our programs work or not So he worked with them and came up with the idea of creating an anchored version of the SPM that's not quasi-relative These researchers came in and presented to the overall administration they worked with our team and they released a paper that I hope they felt was better targeted at how they could influence the policy debate And we released a chapter which was able to explicitly try to answer the question of how much has public policies reduced poverty And we were able to release that in time for the 50th anniversary so we could feed directly into the debate that was ongoing That is I think a great model and let me say that was one of the things we did where we did actually spend months doing it not weeks doing it But we still spend way less time than academics would typically spend Let me give you one other example that I think really illustrates the modern era of economics The policy team and senior officials inside the administration were grappling with whether the president should raise the minimum wage for federal contractors And there were a lot of questions what would happen if we raise the minimum wage for federal contractors How would this impact employment? How would this impact productivity? How would it impact the cost to the taxpayer? And I think why I think this example really illustrates the modern era is because of course the first thing we did is what CEA would have always done And we had many people who were reading a full range of literature not just on minimum wage changes, living wage ordinances, changes in federal contracting pay And providing a summary of what do we know? We took a look at in terms of the employment effects at all of the many, many studies that had been done to sort of see where does the literature seem to suggest what are the employment effects from raising the minimum wage And then we did our own empirical research It turns out there's federal dollars spent in every single state and every state has raised its minimum wage What happened to spending on procurement in each state as they raise their minimum wage Now is where I talk about something that not that long ago would have helped someone make progress on their dissertation A very nice piece of work which showed that there was zero effect on procurement costs in a state following its rate increase in the minimum wage So that pulling together existing research, doing our own primary research and doing our own detective work in the data One thing I should mention is that a natural question you might have is how many people are there working for federal contractors who are in the minimum wage That data doesn't actually exist, we don't collect that data So that's when I talk about data detective work, developing models and looking at downloading federal contracts and trying to figure out what kind of work these people do What city they're in and what's the likelihood that they're being paid the minimum wage given other people doing that kind of work in that kind of town This overall analysis, doing primary research, investigating the current research This is the kind of thing that I think is really only possible in the modern era where millions of data points are at our fingertips Enormous amounts of computing power at our fingertips And frankly our students have all been better trained coming out of undergraduate programs, coming out of MPP programs, coming out of first or second year graduate school To use a conometric program to do the kind of analysis to help us shed light on these issues Now beyond these issues that are directly feeding into policy decisions The council keeps abreast of the new economic literature for topics that might be important for policy making in the future and where the profession stands on critical issues Maintaining a close connection to academia is useful here too as we're able to offer advice on new policy suggestions that is grounded in cutting edge work For instance we're regularly finding, looking at what is the latest working paper coming out of NBER and is there information in there that we want to make sure policy makers Know about not related to any particular policy but just in the back of their head as they're trying to understand how the world is changing We also invite prominent scholars to discuss their work And in recent weeks have enjoyed sessions with Tomas Piketty, Bob Hall, Atif Mian, Amir Sufi to discuss their prominent work in person And I think it's really important to emphasize that the White House is always interested in policy recommendations even if they can't be readily implemented Because policy ideas that may seem unrealistic at first can, the policy recommendations that flow out of research can eventually become common sense approaches to policy So for instance as many others have noted tradable permits and spectrum auctions were once only academic exercises And the Council of Economic Advisers was at the forefront of saying no these may actually be real live policy solutions by continuously beating that drum the world eventually moved towards that The other thing that's happening is of course as the world is changing there is greater potential value and certainly greater opportunity for a CEA to engage not only inside the administration But to engage with policy makers outside the administration and to reach a more general audience whether it's journalists or influential advocacy groups And so CEA now engages talking to the broader public much more directly And I think again technology holds the key We're making fuller use of a modern media platforms both Jason and I have Twitter accounts and I have many times after the employment report was released whatever way we possibly can Now the last thing I wanted to say and I think that this is important for people who take a look at the empirical revolution and economics and sometimes mistakenly think that the shift toward a more empirically grounded economics means theory is less important And I think this is just wrong and so I always when I talk about this want to make sure that I have a chance to get that out because what I think has happened instead is not that theory is less important but that the role of theory has changed When facts were rare we use theory to fill in for facts today facts are abundant we have too many facts and we use theory to help us weed out facts And to give us a set of tools to identify empirical research results that are unlikely to be true So let's return to the example of research on the minimum wage when I said you know that we took a look at as many of the results in the literature that we can I think one of the lesson for how theory weeds out facts is really apparent when you look at a scatter plot of the impact of the minimum wage on employment because it has a huge left tail and no right tail And this is evidence of publication bias it's probably sensical publication bias if I produce a result that tells you that the greatest way I can increase employment in my community would be to raise the minimum wage And that would enormously boost employment no journal editor is likely to believe that is true Now if instead I produce a result that says the minimum wage has a negative impact on employment those results are more likely to end up getting published But I think this again reminds us of the role of the council and other academics with inside government other economists and empirical researchers at reminding people again about that Bayesian updating Thinking about what are the results you have in front of you and what they mean And every time we as a CEA produce that scatter plot I always get a few people on Twitter who say to me but yes look at all the there's still all these negative effects on employment Look at these estimates on the left hand side and there's none on the right hand side and it's useful to understand that those have been sensibly I think weeded out But I think in addition to thinking about the fact that observations are thrown away because of the fact that they don't match our theory I think that the more important or perhaps most important role now in interaction between theory and empirics is the growth of behavioral economics Because you know the growth of behavioral economics depended on data to reveal to economists and psychologists the systematic ways in which people make mistakes Without evidence economists had known that people make mistakes I think sometimes people forget this economists did know people made mistakes We just assumed these mistakes were random and we didn't have any reason to believe that they were systematic and so on average people made calculated rational decisions What data is helping us understand is that there are ways in which people make systematic mistakes And this this revolution in theory coming out of the revolution and empirical science I think is very important for policymakers and something for that CEA needs to help policymakers understand What does this mean it doesn't you know what does this mean and what is the role of policy and helping people in particular areas where we now know that they make mistakes And I think this has opened up a whole new type of research for us to be paying attention to and a whole new way for us to be thinking about how we can inform the policy process So let me conclude by saying that this administration and and I always I should say it's really the only one I know because I am kind of young Is really has been dedicated to building a smarter more innovative more accountable government and has been focused on integrating research into policy In fact I I just left a meeting where the president of the United States attended and one of the cabinet members talked very excitedly about working with the chief statistician of the United States To make sure that you know was we were going to get the data that we needed and so these this idea that we have to know what we're trying to do and be able to measure it is at every level of government now And many agencies and offices in the White House including the Council of Economic Advisers are helping to make research more prominent in the minds of policymakers and helping to set an evidence based agenda to improve governance performance The Council's role in this is a continuation of a long standing model that's been successful across many administrations and while our approach is changing somewhat with the times the fundamental goal looking out for the national interest by bringing rigorous economic thinking to matters of national importance is resolute The revolution in economics has increased both the opportunity and the challenges for economists and policy and the Council of Economic Advisers is happily embracing both I have a question that has some it's a little unfair it's self-interest but it seemed to me first of all it's a meeting about policy and academia and bridging that gap as Ned Gramlich did better than anyone I know But you said a couple things about policy that I'm not sure we prepare students for namely the use of simulation simulation techniques and also behavioral economics so when you come back in the year would you have an interest Would you recommend that maybe econ departments and policy schools begin to train people to use those tools You know I think that that is a great question and I do think that if we think about the revolution and empirical economics and what it means for policymakers it does have implications for how we train students and the types of skills we need them to have You know at its core a grounding in economic theory is essential you know if you're going to be an economist advising in policymakers or playing a role in economic policy you need to understand the role of incentives that hasn't changed And that's been true in you know since 19 before the Council was set in place in 1946 but there is the sense in which you know our it is our theories are changing our understanding of the ways in which people make mistakes are changing And mistakes people making systematic mistakes is perhaps you know if I think about where where is the role of government greatest it's where there's market failure asymmetric information you know we when the markets doing well the markets doing well When people are making mistakes are there things we can do to help them make fewer mistakes and when there's problems with information are there things we can do to remove those blockages when and and we certainly I think that is the direction in which we should be making sure our students are trained to focus on why why is there a role for government what is you know what is different from if what would happen if there was no government and to do that today I think we do need to have a grounding in the types of mistakes people make And then the tools to be able as you said to to turn to the data and to consider you know potential counterfactuals and to think of the most creative way to say if we enact this policy what do I think is likely to happen Jim you can Are you at all concerned that the empirical it sounds terrific with a more empirically based you know policy analysis and randomized controlled trials but the most obvious application for those are on short run cut reactions and short run problems what if we're does it mean that we have less emphasis maybe Or how do we take that method and apply it to long run issues you know like deficits that go on 20 years environmental degradation global warming changes in income distribution over long periods of time how do we can we does it apply to that as well Or is it mostly kind of short term things and if it isn't the long term things then how do we address those analytically those long run problems So you know that I actually spend a lot of time trying to think about this because if you I do think one of the important roles of the council of economic advisors is that because we don't we're all going back to our academic jobs and I should have said you know I do think this is actually Many for other people pointed this out but this idea that in some sense we really don't care what people think of us because we have a real job to go back to is actually what one of the things that gives us a degree of like independence that's kind of important you know there's not the yeah there's There's not career positioning it's all fun and when it's not you know you can take your bomb back and go home and the you know so the council thinks a lot about long run and I'm going to apply micro economist and I actually spend a lot of time thinking about long run and in fact But I think that there are things we learn from the empirical data that can help inform us about long run issues. I will say that there is empirical work on this relationship between debt and growth little That you know I think is very interesting and should actually inform policymakers won't dig into that but it's not like those issues don't have empirical research that but for things that I think about because I focus mostly on social policy. You know I think that we have a couple of big opportunities for long run growth ahead of us right now. One of them is doing more to promote minority men in the labor market and I think that that's grounded in empirical research which shows that when we Invest more in preschool and boys in preschool they end up having better performance being more likely to work in the labor force. Someone in academics we had a bunch of academics to talk about work family issues in with senior officials just this week and one of them told me about a study that Showed that you can see the effects of paid maternity leave on the wages of children whose mom stayed home at age 30. So I haven't looked up the paper but I like was on my list of things to do this weekend because you can look for empirical results and what we have to do is then say well what does this mean for the long run. I think that you know it is important that we keep asking that question and I think the nature of the council is designed to think about the long run but for me I think you know if you look at women's labor force participation I think it was the growth of women's labor force participation was extremely important for Economic growth in the last half of the 20th century and what we do to support it in this century is going to be important for long run economic growth is in the same thing for minority men. Immigration is something where we think a lot about what's the effect of that on long run growth. So I do think that there is you know a role to aggregate up the micro results and ask what do they mean for the long run. Having been on a member of the CEA more than 40 years ago I was fascinated by your account of how things have changed over that period both for the CEA and for our profession. And I hardly endorse what you said at the beginning about the mission of the CEA but and I would include paying attention to the long run. But there's one thing you didn't mention and this is something that was first raised to me by Walter Heller and that since then I've thought a lot about. And that is another goal which is to some extent intention with all the ones you mentioned. And that is that when all is said and done and the internal fights have been won or lost by the CEA you are also the defenders of the administration's economic policy. And there certainly times do arise when that defense is not exactly congruent with the best possible advice you might give as an economist. And I wondered if that had yet concerned you and how you would deal with it. Well see one of the ways I dealt with it was I didn't address it today. I find that things aren't nice to talk about. We just don't talk about. So I mean that's that's obviously true. I mean nobody labors under the impression that it wouldn't make sense for the president and senior advisors to take the advice of the economists all the time because we should not be giving advice that makes any that that is conditional on making political sense. We should give the best economic advice and sometimes that advice is ridiculous given the political situation. And so that's obviously not going to be taken all the time. And also as you said there are people have differing views. I think that it's important that we continue to give you know the you know that we continue to just give the best advice that we can. And to look for you know and to follow the policy if you don't have anything nice to say don't say anything at all. I guess the one last thing I'll say on that is I think that for me the hardest thing is that I is what I did say in the talk which is that I think about the world as an uncertain place. And politics does not really allow you to talk about uncertainty. We talk about point estimates and that is very unnatural for me. We talk about 90 percent confidence intervals. And I think basically everyone in the administration who wasn't an economist would tell me that not only does that open you up to trouble but it bore it's boring boring so you know it doesn't serve any purpose. But I think that that doesn't impact it doesn't impact the advice that we give. And it just means that in our public engagement we have to choose carefully. Questions from people like me. Let me just say that the problem you asserted about confidence intervals is not confined to the government. I confronted exactly the same problem in the corporate world. Nobody wanted to hear about confidence intervals. They wanted point estimates. I'm really glad you said that because actually I have had a little bit of a brief stint myself in the corporate world and I found the exact same thing. And it's actually what drove me back to academia. So I think it's it's exactly right. It is true that you know people don't grow up comfortable with statistics. It would be good if more of our preschoolers were comfortable with statistics more of our elementary school students. But that's just not the world that that we currently live in. And I think that that's why it isn't just about government the public the world. They want a point estimate. When I