 Welcome to a kind of a joint event sponsored by close-up and the IPC, the support of close-ups education policy initiative where we bring in a series of speakers throughout the year. Today I'm pleased to present Rick Handiget who is an economist studying education policy and labor economics who's done many many things over the course of his career. In fact, I started reading him in graduate school and it's actually interesting. It's one of those cases where at least two or three times in my relatively short career thus far I've started a project, I started investigating a different area and then I get into it a little bit and I realized that in 1976, Rick has written a consumable paper on it. I just happened to talk just once but it's happened two or three times through thus far. So I'm very pleased that he has agreed to come speak with us. He is the Paul and Dean Hanna senior fellow at the Hoover Institution Standard University and in Fire Lives he's held academic appointments at the Air Force Academy, University of Rochester. He's actually quite involved in policy circles as well. He currently serves as the chair of the board of directors of the National Board of Education Sciences. So he is kind of the boss of John Easton, the director of IES in some sense. He's been very, very involved in kind of school reform and school policy debates in California. He currently resides in Texas where he's done a lot of work. And in recent years he's started doing much more work on education from a cross-national perspective and explaining differences across countries. And this is what kind of motivated the interest of IBC and close up to having joint sponsorship with the panel. And I'm going to turn it over to John Svénard, director of IBC, to say a word to you. Thank you. Thank you. I'll just add a few words indeed. It's a pleasure to have Eric Cushier with us. I too started reading the stuff when I was in graduate school. And then you sort of, 20 years later you go back and do a new reading list and you realize that it's a totally different area. He's contributed there as well. For us at the International Policy Center the great appeal of course is that he is one of those renaissance men who works not only on the United States but he's done important contributions actually internationally around the world, very important contributions in the context of the developing countries. In addition to what you've already heard he works with the OECD doing significant work in the advanced countries as well as the developing countries. And he really links labor and development in a unique way. And so for all of you who are IBC enthusiasts, great, great, we're here today. And it really is a better person for our annual joint IBC and close up lecture series. So Eric, it's all yours. You can use the usual health control techniques. Thank you very much. Thanks for the nice introductions. I aged a lot in those introductions. More than that, worse than that, I look around and I see many of my old friends in this room. And additionally my old friends come from a variety of different areas and so they know different parts of this talk and know it better than I do. All kinds of little parts of this talk so I'm a little bit nervous up here. Let me start out by giving you a quick overview of what I want to talk about. First, a simple answer that I'm going to give you probably in more detail than you care to have it. Does it matter what students know? And I'm going to try to provide an answer to that question. How well is the U.S. doing? And thirdly, sort of anticipating the answer to the second question. What can be done to change things? Now, spending too much time in universities and knowing that late on a Wednesday afternoon I can hold your attention for a little bit of time and then after that it all breaks down. So let me start with the answers. Yes, it matters what people know and it matters a lot. Not so well. The U.S. is not doing so well and we're not competitive internationally. And third, what can be done about this, I'm going to argue that improving teacher quality is the key. That nothing is quite as important as the teacher and teacher quality. And that unfortunately this is a tough thing to try to change though because there's a lot of resistance to it. So that's what I'm going to try to fill in some details. If you're satisfied with that answer you can leave now and I'll just go on for the residual who aren't quite satisfied with those answers. Does it matter how much what students know? The only answer that we get for that question comes really from looking at individual labor market returns for the most part. And there's been a history of half a century of people, economists in particular, estimating functions that relate earnings of individuals to schooling and other things. What I'm going to argue is that the recent evidence we're getting focuses not so much on school attainment, which is still part of the public debate, but it focuses more on what people actually know, their measures of cognitive skills of individuals. And then we're getting increasing amounts of evidence at the individual level that what people know is really important. Related to how much schooling they got, but not necessarily. But I'm going to skip over this because everybody in this room presumably walked into this room believing that. You can't be at a university without believing that. But I'm going to go to something else. I'm going to talk about economic growth. And in particular, I live in Stanford, California, which is right next to Mountain View, California, which everybody knows where Mountain View is now. I'm your computer. It's the home of Google. And so I'm going to start with the Google Earth view of education, and then I'm going to come down closer to where we are. The Google Earth view of education has to do with how education fits into economic growth. Now, economists have recently taken up again looking at economic growth. For a while it was kind of a dead subject, and macroeconomists in particular were much more interested in business cycles. That they are now. The whole news of today has to do with the current recession and so forth. But what I'm going to argue is that that's mistaken in many ways. But what I'm also going to look at is the developing country part of this story. We have the Education for All initiative, which is part of the World Bank's effort to ensure that everybody in the world gets basically an eighth grade education, lower secondary school education. There's an equivalent of that at the UN Millennium Development Goals that has essentially the same target. And in order to keep you awake, I'm going to start this discussion by saying that I think these are mistaken goals and that this should not be the way we phrase our goals. Now, usually people start wondering what kind of crazy guy is this? Why are you against development in Latin America, sub-Saharan Africa? And the answer will be here in just one second. Here is the Google Earth view of economics. What I have here is along the horizontal axis is test scores. I'll explain conditional in a minute. These are measures of what people know in terms of math and science ability. And on the vertical axis is growth rates. And what you see is that countries fall on almost a straight line or pretty close to a straight line with a little scatter off that line. Now, why does it say conditional on both of these? Well, the only thing else that's behind the scenes that you don't see here is that these are long-run growth rates between 1960 and 2000, annual growth rates and per capita GDP. But we've allowed for the fact that countries started in 1960 at different levels of income. And why is that important? Well, it's important because if you start behind, all you have to do is copy somebody else. If you start ahead, like the U.S., then you've got to invent new things, and that's harder. So what you find is that countries that have lower initial income, initial GDP, grow faster. And I've taken that into account here, just where they were in 1960. But once you do that, what you find is that you can explain a large portion of differences in international growth rates from Peru and South Africa and the Philippines up to Taiwan and Singapore and Korea, Hong Kong. And somewhere in this block here, you find the U.S. in the middle. And what this says, and I'm going to give you the numbers, the implications of this in a minute, what this says is countries that know more grow faster. That is important. All the economists in the room immediately understand that growth is what determines our current level of income. We started out the U.S. at the beginning of the 20th century with a pretty good economy. But then we grew more rapidly than other economies across the 20th century, leading us to be at the top of the economic pyramid. Can we keep doing that as the question? Now the radical change of this picture, by the way, is that we're now measuring what people know as opposed to how much schooling they got. Let me look at what happens if we look at how much schooling they got. Here is the picture of conditional years of school, again taking care of 1960 GDP, against growth rates. What you see is that there's a positive slope through this line, right? But countries are scattered quite a bit farther away from this line. The simple summary statement is that the R squared of this regression is .25. We can explain a quarter of the variation in growth rates with years of schooling or school attainment. In the previous slide that I gave you, the R squared is .75. I can explain three quarters of the variations in growth rates with just test scores. Now if I take this picture and add into it test scores into the same regression line that's behind this, what do I see? What I see is that this line is perfectly flat. You won't believe how long it took to get to this simple conclusion. The simple conclusion is if you go to school and you don't learn anything, it doesn't count. That's what this says. But it took a lot of effort to get to that, but that's what that's saying. And that's why I can make these statements about education for all and millennium development goals that I did at the beginning. By having a standard that says that you have to be in school for eight years, lots of countries said we're going to make that. But 30% of the time the teachers don't come to school and so on. You know the story if you've been into development work, and they don't learn anything. So like South America has had fairly high levels of attainment of schooling, they just don't learn anything. And that's been what's behind lots of the growth failures in our opinion of schooling in South America. Now let's go to how well is the U.S. doing. That's N.1. It does really matter, and I'm going to give you a summary in just a second of exactly how much. But it really does matter how much you know as a nation. How well is the U.S. doing? Well the simple story that actually catches a lot of people by surprise is if I just look at years of schooling, we all know that the U.S. provides all this schooling and the rest of the world doesn't. So here's an ordering of the countries, mainly the developed countries, the OECD countries, but some other developing countries, ordered in terms of years of school attained for the current cohorts. This is basically expected schooling for the current cohorts. And what you see is that the U.S. is this red hashed line. The OECD average is this green line, and there are a number of countries, Finland, Denmark, Korea, and so forth that are at the top. But we beat out Zimbabwe, where this is actually probably too large for Zimbabwe today, and the Philippines and a few developing countries. But in terms of just attainment, we don't do well. But I told you that wasn't the answer, right? Let's go to the other picture. This is the picture in terms of this is one of the international tests. This is mathematics performance on PISA. PISA is the program for international student assessment run by the OECD. Many countries around the world, everybody in the country knows what PISA means. In the U.S., it's still thought to be a bell tower in Italy. But PISA gives us a ranking of how much our 15-year-olds in this picture, that's what takes PISA, know compared to other countries. Same code I had before. The U.S. is the red hashed line. Here's the OECD average. And you have a few Brazil, Tunisia, Indonesia at the bottom. Can you see this large portion of the OECD or developed countries that are learning more than we do? This is not specific to math. I can do the same for science if you want. I can do it in different years and different age groups and so forth. Now, if I look at this and I start up at the top, you know, there's Hong Kong, you know, and realistically, we can't expect our kids to work as hard as those kids from Hong Kong, right? That's not possible. And then there's Finland and, you know, what else do you do in Finland? You either study or drink, right? But there's Canada. And Canada looks kind of like us, right? It's hard to tell, except for a few funny pronunciations, that the Canadians aren't us. And this difference, I want you to keep that in mind because I'm going to come back to this difference in performance when we go on between the U.S. and Canada and what that means. Now, I told you two things that might strike you as incongruous. I said that the U.S. is in its current position because we do so well on economic growth. We're in our current economic position because our growth has been so high. And then I'm also telling you that our performance on these measures is low and I'm making the story that the two are related. So why, how could that be? Well, I think that the first thing to point out is that the U.S. has the best economic institutions in the world. We have free and open labor and capital markets. We have limited government intrusion. We have secure property rights. The things that at least economists believe are extraordinarily important to determining well-functioning economies. We have done better than everybody else that having been said, lots of other countries in the world have recognized this and have in fact made significant strides to improve their economies. All you have to do is look at India and China which have had over 10% growth rates for 20 years and what have they done? Well, all they've done is tweak some of the horrid economic institutions they had to make them a little bit better and they've gotten a lot of growth out of this but they're still moving along. They are only part of the story of the improving economic institutions around the world. Historically we've substituted more years of schooling for low quality per year schooling. We've substituted quantity for quality but I already showed you the picture that we're near the OECD average, the OECD countries average in terms of years of attainment. I think we've always had much better higher education than any other country in the world but lots of other countries take advantage of our higher education system these days and in fact that's part of the story. We've had skilled immigrants, both people who come here for education in the U.S. and people who are educating their home country and come here to work. I only add a bit of caution. One of my squash opponents at Stanford was an electrical engineering PhD student who finished his PhD two years ago and he had the gall to go back to India. He's sitting in the middle of Silicon Valley with a Stanford double E degree and he went back to India arguing well, the opportunities are just better there suggesting that maybe we won't have this monopoly on retaining and getting skilled immigrants to help us out. So what does this all mean? I'm going to give you the summary cost estimates of what it means not to try to improve our schools and the way I'm going to do that is I'm going to take that little model of how test scores relate to economic growth and I'm going to project out, simulate what would happen if we could have a reform policy that in fact improved our schools. I'm going to make this what I would say a realistic policy of it's going to take 20 years to get any reform through to get the results of any reform so it takes time. And what I'm going to do is look at the economic benefits over the lifetime of somebody born today. Now somebody born today is going to live until about 2090, expected lifetime. And the reason why that's important is that some of the gains, lots of the gains, many of the gains from these reforms won't be seen for some time into the future. This is a historic political problem that politicians might look very close to present but also economists or you, you would rather have a dollar today than a dollar in 2050. So what I'm going to do is calculate the present value. I'm going to try to put everything on today's dollars terms of the economic gains so that I can compare it to other kinds of policies that we might think about. So what do we do? I'm going to assume, oh I'm also the first line there, I'm going to assume that the next future looks like the last 40 years that I had encapsulated in that picture. So here's the gains in rough terms of a 25 point PISA improvement. Now 25 points, I didn't give you the scale of the picture I had of mathematics on PISA. That has a standard deviation of 100. So this is a quarter of a standard deviation improvement for those of you who like to think in standard deviation terms. For those of you like normal people that don't like to think in terms of standard deviation terms. A 25% improvement is moving the U.S. to roughly the U.K. or Germany. So it's a noticeable change but not outlandish. What this is, the dark line is taking the growth rate picture that I had before and just projecting it into the future. So starting in 2010, well you don't see any results here. Why? Well we haven't changed the schools and more than that the people that are in school are in school, they're not in the labor market. So eventually though people start knowing more with this improvement and they start becoming a larger and larger portion of the labor market and you get this dark line. The other two lines are just some confidence intervals around this because there's uncertainty about these projections. But for the most part I'm going to ignore the uncertainty here and just give you the projection out there. And what I said was that I was going to summarize the gains out to 2090 which is the life expectancy of somebody born today. Now you start looking at these and if you look at these well this is zero, it's below 5% for quite a while and so forth, that doesn't look like very much because you know small percentages. But there's two things that are important to this picture. One is what we want is not this line and the point, this gives us how much higher is GDP at any point in the future compared to what it would be without an improvement. This is relative to everything else operating as it would. What I really want is the area under this curve because I get these gains that add up each and every year into the future. That's the first thing to point out. The second thing to point out is that these are small percentages but these are multiplied times $14.5 trillion today which is what our GDP is today. So just as we talk about how many Indian engineers there are there's only a tenth of a percent of Indian students go off to be engineers but there are a lot of Indians and so you get a lot of engineers in the same way a small percentage increase in GDP is real money. How real is it? Well, here's another simulation that I do of what would happen if all the OECD countries could catch up to Finland in terms of their performance. We had that picture in the past that Finland was up at the top. Well, if we go all the way over here you can barely see it but Finland doesn't gain anything because they're already Finland. Korea and Japan only gain a little because they're pretty close to Finland. But as we start moving across here we get through for Yand, Czech Republic here they're doing better, they're closer. The OECD average here is here in red and the United States is over here. Now what's this say? For the OECD average it says it corresponds to 600%. The present value of the gains on average for the OECD is six times the current OECD GDP. For the U.S., which is over here it's larger. In present dollar terms being Finland is worth $103 trillion. Now for the last six months all that we've talked about in Washington D.C. is the battle over one trillion dollars worth of stimulus money and this is the impact of improving according to history. History might be wrong but this is what history says. Let me look at one other simulation that might interest you. It looks sort of like no child left behind in some sense. What if everybody could come up to we would call minimum proficiency levels of performance which on these tests the PISA has a mean of 500 and a standard deviation of 100. This is one standard deviation below the mean which by the psychometricians in the audience will recognize that we sometimes attach labels to these things. This is level one which means in mathematics performance at level one is right around 400 points. That means if you have a well-defined problem that is completely specified a person can solve it as opposed to adding anything beyond a completely specified problem. Now what do we see there? I didn't point out before by the way Mexico and Turkey are OECD countries and they're sort of hopeless because they're way behind but the U.S. does a little bit better here but that policy for the U.S. is $72 trillion. Here's a summary of what I just gave you. I've got the OECD total of a 25 point improvement in PISA OECD total is $114 trillion Equaling Finland across the OECD is $260 trillion. Now there's two things going on in the pictures I showed you before. One is the U.S. is behind the OECD average so its performance isn't that good and secondly the U.S. economy is really big so the U.S. is 40% of this improvement by those two standards. The other thing I would point out I pointed out that you should remember this 25 points in the PISA is one half of the difference between the U.S. and Canada and one half of the way to Canada is $41 trillion. It matters a lot what we know and we're not doing too well. So how do we change things? So here's where I start getting in trouble with various people in the audience perhaps. The first answer across all OECD countries and across the U.S. is a simple resource solution put more money into schools to increase the investment in our education because it's so valuable. The problem is it hasn't worked very much. Here's the OECD picture just to get you the international thing. This is cumulative educational spending on students against scores on the PISA test same ones I gave you before and the red line is the simple regression line that I would put through these countries that is entirely driven by Mexico and Greece and if you think that Mexico and Greece will come up to the OECD average with a few more dollars then you probably don't want to stay for the rest of this talk. But if you drop out Mexico and Greece you get this perfectly flat green line where there's no relationship across countries in a simple answer. So what about the U.S.? Well here's U.S. performance. U.S. performance is measured by the National Assessment of Educational Progress or NAPE. This is a test consistently given over time and this is for the 17-year-olds basically at the end of high school. What I've given you is it's given in different subjects so there's math here, reading, science, and writing. Writing is not a very good test it's been kind of dropped but I'll give it anyways. What these bars are are essentially end of decade performance of U.S. students. So the first line is performance in 1970, 1980, 1990, and 2000. And so the way to look at this picture in a very sophisticated way is to draw a line across the top of these. So there's a slight uptick in math performance over this 30-year period. There's a slight uptick in reading performance. There's a more significant down-tick in science performance and writing as I say we shouldn't probably pay too much attention to that but when we had it measured it went down. So in a sophisticated way my argument is this says that performance has been flat in the U.S. for 30-plus years. Okay. So let's compare that to putting the public schools. This is just 60 to 2,000. We've done just what people say we should do. We've reduced pupil-teacher ratios from 26 to 16. Actually by giving 2,000 I'm inflating that number because pupil-teacher ratios in class size has fallen throughout the last decade. Master degrees close here to the education school Debra. We've doubled the master's degrees in the nation from a quarter to over half of our teachers have master's degrees. Experience has sort of marched up that sort of demographic of when teachers entered but it's as high in the post-war period I think and so when you take these three things together that basically determines per-pupil expenditures because degree levels and experience determine teacher salaries and pupil-teacher ratios tell you how many kids you spread those salaries across and so what you see at the bottom is that from 1960 in constant dollars we're 2,500 dollars they're at 8,690 today if I looked at 1960 to today it'll be over a four times higher expenditures per pupil. Now I won't go back to the previous slide but dramatically increased expenditures in the ways that many people have advocated that we should increase expenditures and flat performance. So something else we have to look at. This is total expenditures over a much longer period for those who argue that I picked a selective period of time here. This is constant dollars of spending where we've had essentially between 2.5 and 3% real increase in per-pupil expenditures for a century. Now this increase has been beaten recently by healthcare expenditures per pupil but only recently. If I looked across the whole century it would not be the case. With healthcare expenditures we say that we got to do something to control these. I presume that the president made a speech today about that subject and with education expenditures we say well we ought to spend more. Part of the story is this is just a little aside. It's not really central to the whole talk but there was a survey given a couple years ago that asked a random sample of parents how much do you spend how much is spent on your school and the blue number is looking at the actual expenditures on the school and the red number is the response of the average respondent that was about half of reality. For me the more interesting part is actually not this. A quarter of the respondents said that less than a thousand dollars per pupil was being spent on their schools when in fact the reality is ten thousand dollars per student. Similarly how much does the average public teacher in your state make it's closer it's only a third off instead of half off in terms of expenditures. So it's easy to convince a lot of people that we're really short changing our schools because they don't have any idea what's actually being done. So let me preface the next slide in a second. One of the things that's happening that's going to happen in California and I'm not sure if it's going to happen in Michigan it's probably likely in my opinion without any knowledge I predict that it's likely. Is that over time more and more people have called upon the states to in fact I'm called on the courts to in fact intervene in what's being done in schools. This started out in California this is one of the bad inventions from California in a case called Serrano versus Priest in which the argument was made in state courts that there was inequity in the funding of schools and that there was no reason why Beverly Hills should be spending that much more than Baldwin Park on their schools and asked the courts to intervene in a classic sort of basically 14th amendment but occurring state constitution due process equal protection kind of argument. More recently there's been a set of court cases called adequacy suits. The previous were equity suits making it equal across districts. Adequacy suits are ones where people have gone into court and said schooling spending is kind of even but it's just not enough and asked the courts to intervene to increase the amount of schooling. The argument is more sophisticated than just give us more money. It comes down to saying the state constitution says that you have to provide some sort of education. Every state has a different constitution with a series of adjectives. For example, New York state has a sound basic education. Other states have equal education or free and equal education efficient education and so on. These court cases were very popular actually they were started in Kentucky and they've gone other places. The one I want to give you because it has evidence on this resource question is Wyoming. Now most of you vaguely know that Wyoming is somewhere in the central center part of the country up to the north but there's no people there in this small state and nobody knows anything about it. Wyoming had a court case on the adequacy of its funding in the mid 1990s in which on the basis of a very vague sound basic education clause in their constitution, the judge said that the schools of Wyoming were required to be visionary and unsurpassed Now obviously you can't do this everywhere because you have Lake Wobigong problems. If you're a state like Wyoming you can do it without any repercussions. You can be unsurpassed but all it required was that if somebody had a new vision the state was supposed to pay for it. We can look at one side of this. This is spending in Wyoming versus the U.S. average. The U.S. average is the blue line here over time. You see the court case here in mid 1990s and here's the red line is spending in Wyoming per pupil and all of a sudden you see that it's going near vertical you know this is going supersonic and it keeps going past this by the way 2006. So that Wyoming most of you don't know this but if you just made any allowance for the difference in the cost of living between Wyoming and the East Coast Wyoming would be the highest spending state in the nation. So we might ask well this is a good experiment right visionary schools in Wyoming are being visionary let's look at their NAPE performance. So this is for white students in Wyoming of which 85% are white students this is fourth grade reading and what I'm going to look at is the growth rate in performance between 1992 and either 2007 or 2009 on these NAPE test and I'm going to look at how well did Wyoming do versus the rest of the nation. So the blue line is the national average that didn't get I just showed you they didn't get this fusion of money. They're just sort of growing at 3% as opposed to 9% I think it was for Wyoming. And here on fourth grade reading we have Wyoming performance. So they started out above the national average and managed to slip below. Fourth grade math Wyoming starts at the national average and falls behind. Eighth grade math Wyoming starts above the national average and falls behind. So well maybe these suits are aimed more at the disadvantaged students so let's look at the Hispanic students which are the only noticeable minority group in Wyoming. And what do you see? Well in every case the red line, dashed line is flatter than the blue line. So that with unsurpassed vision and money this is what you got. So what's this all mean? Well to me it means that you've got to look at something else and the thing that I look at based upon some of my research some of Brian's research some other people's research here is that teacher quality is really the key element. I'll give you a couple summaries that I would give from things that I've done. One is that there's a very wide variation in the effectiveness of teachers so that even if you look within the same school some people are doing much better than others as measured by student performance whether students are learning more or over time. How big? Well some of the estimates that I've done on Texas kids suggest that if you take a good teacher which I'm going to define as one standard deviation above the mean 86 percentile versus an average teacher three or four years in a row of a good teacher is sufficient to close the gap between the average free and reduced price lunch kid and other kid in the state. Now I put it in those terms because we have a long history in the U.S. of people saying well schools don't mean much the only thing that matters is family. This started with the Coleman report in the mid 1960s that said that schools weren't very important and it's continued now. What this says is schools have the power in terms of teacher quality to overcome these differences. They might not use it but by the estimates of the variations in quality they have the power. Let me give you another example. Let's look at Gary Indiana. Gary Indiana close to home here. Poor black kids all poor black kids in Gary Indiana the difference between a good teacher and a bad teacher was one year of knowledge per academic year. Let me say that slowly. A good teacher got a year and a half of growth in learning. A bad teacher got half a year of growth in learning. The difference depending upon what classroom you ended up in could be a full year of learning. So it doesn't take very many years of a good or a bad teacher to see that you've either that you've dramatically changed their life for good or bad depending on which side you get. The problem that many people have argued is that well unfortunately the quality of the teacher is basically unrelated to our measured characteristics. There's been a lot of time and this is what confused Coleman in the original Coleman report of looking at the characteristics of teachers and thinking that if the characteristics were not related to achievement then teachers then nothing about teachers mattered. But in fact teacher graduate degrees are essentially unrelated to performance. So if you walked in brought your teacher into school and the principal proudly introduced your teacher as a teacher who just has a master's who has a master's degree and is ready to teach your kid you should walk away knowing that that conveys no information about whether the teacher is particularly effective or not. Experience has an asterisk beside it because we do know that the first couple years two or three years students count and teachers learn their craft more in the first couple of years but the difference between a teacher with five years of experience and a teacher with 20 years of experience on average is nothing. There is no difference. And even this is a little bit harder because all states have different certification requirements but when we've looked at certification of teachers it doesn't really matter if you have a fully certified teacher or not. On average they're going to be equally as effective. Now that's not to say within each of these categories there's not wide variation in performance. So what does this all mean to me? Let me sort of complete the circle. If I take the estimates of the quality differences of our teachers and I go down to the bottom of this distribution where there are teachers that probably are best described as harming kids and I move from the bottom of the distribution up a little bit and say how much is that worth in terms of achievement and performance? Now I've given you two lines here. What I've said is what I've called the percentage that are deselected which is my attempt to be polite by saying fired or removed or helped to find another job. What percentage if I start at the bottom and how much gains do I get in performance? Now I've got two lines here because there's uncertainty in these estimates. Something that I've known that Brian has worked on and shown the uncertainty and so forth. But let's take the blue line which are the estimates that have a wider variation in teacher quality. And if I go out to the bottom 6% and say could I replace the bottom 6% now with a great teacher just with an average teacher and I look at bottom 6% and all of a sudden I'm at the level of Canada if I could replace the bottom 6% of our teachers according to these estimates of variation in teacher quality with an average teacher. If I go out 10% I'm at Finland or if I take less I don't know if we call it less optimistic estimates of less variation in teacher quality 10% would at least get me up to Canada by the lower plausible bound. So what's this say? Think about this, if we have a school with 30 teachers in it we're talking about the bottom 2 or 3 teachers in that school and replacing the bottom 2 or 3 teachers on average with an average teacher. Now I assert partly based upon Brian Jacobs estimates that principals know who's in this bottom category with fairly good confidence they probably know who's in the top category with confidence have trouble rank ordering everybody but you know who's in that category. So I would actually assert something more than that I would assert that the other teachers know who these bottom 2 or 3 are I've never talked to a teacher who hasn't claimed that she knows with certainty who the bottom 2 or 3 teachers are in the school I think they're the same as the principals would rank. I think parents know we have lots of parents in here and I suspect that each of you was ready to draw the line about some teacher in your kid's school at one point in time and was really upset about it I actually assert that the janitors know too that anybody walking into a school knows the bottom 2 or 3 teachers it's just that we don't have the will or mechanism to replace these bottom 2 with an average teacher where an average teacher is quite good an average teacher this is not a statement that all teachers are bad it's far from that an average teacher if we could get rid of this left-hand tail of the distribution could in fact act like a finished teacher or finished school without the drinking I don't know so the problem is that there's not in my book many simple answers that this takes hard work to get to this point the economist says it's all about incentives that it's changing the incentives that we don't have incentives to have higher performance today and that we need to change those one way or another incentives lie behind getting people motivated to get rid of the bottom 2 or 3 teachers and we have to have a mechanism to learn when things are working when they aren't which we don't really have now some states are trying to put it and we don't have it but let me let me ask some key questions does it matter what students know how well is the U.S. doing and what can we do to change things yes, not so well teachers that's my answer you can carry me off on your shoulders or you can ask questions to try to be sure that you want to carry me off there's 2 back there so you have to choose one yourself thank you you mentioned Canada is a country similar to the U.S. in culture and in income per capita but doing much better in terms of education also in terms of what they know but then you shortly after that you mentioned that U.S. has the best economic institutions financial institutions and I'm wondering considering the fact that Canadian banks and financial institutions are much stronger now than U.S. banks and financial institutions now in 2010 is it really true that the United States has the strongest financial institutions and second of all is there any causal relationship between Canadian level of know-how and they're much healthier financial institutions well I'm actually not prepared to speak about Canadian financial institutions per se historically their economy has not been as well run as we have in the sense that they've had traditionally much stronger union labour laws and a variety of other things that inhibit movements across firms and other financial institutions today part of the argument here that I'm making though is answer to your second question and that is that countries where the people know more in fact move toward better institutions over time that's part of the innovation that the U.S. has had in the past of working to get better economic institutions and that if countries have no more move in that direction it's really hard to document differences among countries and sort of claim that it's a single feature not with the small number of countries let me answer a question that you didn't ask but that's kind of related part of the issue when I put up all these countries is well what do the Canadian system do school system do that we don't do or what do the Finnish system do that we don't do and so forth I'm actually not convinced that that's something that we can learn all that much from because of the very large cultural differences and the makeups of the countries and their historic development and so forth there was a study by McKinsey and company run by Tony Blair's advisor Sir Michael Barber a couple years ago they tried to answer this question why do some countries do better on P's than others or the other international tests they had a whole bunch of things that I didn't believe that they could support that there were differences in one countries but the one thing that I think is that applies to high performing countries and that also explains where the U.S. is is there's a line sort of slightly buried in this report that says countries that have school systems that perform well do not let bad teachers stay in the classroom for very long and I think that that's the heart of the McKinsey study or the heart of the differences but it's hard to demonstrate that when you have a small number of countries and lots of things different you in some sense foreshadowed the question of this because it does seem that the implication of what you're saying is that this kind of is one crucial variable there are other countries that in many ways you would think are well Canada is an obvious example but you'd expect that your theory will write and I think many parents in the audience but as opposed to academics in the audience might be very sympathetic precisely because we wanted to bring the neck of some teacher because of whatever she was doing or he was doing but then it would suggest I did two things one was to suggest that if this is right these countries that are doing better are somehow finding out ways to get rid of teachers and then it might suggest trying to figure out what they do and part of the reason I'm making that point is because I'm fearful that many of the things that people who are making points similar to the ones you're making here in this country are posing are things that have potentially serious side effects for example if you put an emphasis on test scores the obvious thing that an economist would think of is that well what that will do is it will encourage teachers to teach the test and that's not what we want to have happen so the question is kind of if these schools are effectively either not recruiting for teachers or finding ways to get rid of them maybe because the teachers see the light and think of it as something better to do how are they doing there's lots of differences of opinion and basically country case studies that suggest well they're redoing this or that or so on the place where the debate is a little bit unsettled is whether they are through various mechanisms of teaming together or counseling teachers or so on pushing people out or are they taking the people they have and making them better and that's where the debate is really in the U.S. can we take our existing teachers and make them better and there's some so some of these case studies will describe the training that goes into teachers and the lesson planning that is prepared to help teachers get good lessons and so forth and the question is whether you can take a mediocre teacher and make her into a good teacher I actually come down on the former side but the evidence isn't entirely clear I come down on the side that it's for one of a better explanation all genetic that there's a gene there that says there's going to be a good teacher or not and the whole question is is discovering which gene different people have and then operating on that now other people come out differently in particular talk about providing better professional development and so forth which is the common alternative on this I must say I would heavily influenced by a recent study of the institute for education sciences that went into professional development and it had a a test of random assignment across schools of three different treatments one is whatever you're doing today the second was a very intensive this is all early reading instruction second grade reading instruction basically let me see if I got this right 48 hours of classroom instruction before the beginning of the year on intensively designed to improve reading instruction by the teachers and the third treatment was 48 hours of instruction before the beginning of the term plus 60 hours of coaching during the year which is two hours a week of coaching during the school year the results of this study was at the end of year one in fact teachers in both of those treatment groups that got the intensive instruction had changed their behavior in class unfortunately the performance of students was the same by the second year they reverted to their old teaching methods so this heavily influenced me frankly when you have serious study that took on there are two issues in professional development one is people argue yeah we do a lot of it but it's not relevant to the classroom and we don't do enough of it it's not intensive enough this was a study that took on both of those issues and came out pretty definitively now it's only second grade reading and there's lots of other things so I come out that it's all selection and that the countries that are doing well at it are better at selecting but not at all, why is that oh no but the incentives John that you want RG to have to get the right people into the classrooms and get people willing to in fact take on this kind of thing there's another part of this debate about whether we should have performance pay and what that means and the president has come out pretty strongly for performance pay I'm not sure what he has in mind when people say that it usually means well I'm going to line up everybody in this class in this room on teaching ability and decide pay and if I do that I'll get all of you to work harder that's the usual model I think that's the wrong model I mean I think teachers work very hard they work harder than I do but what you want to do is get the right selection of people coming in to the profession and secondly the deselections story I gave you is performance pay where I eliminate the bottom end and then pay everybody the same but they're essentially equivalent as paying everybody different amounts and trying to get people to self-select in and out so I would have said it to colored but I was wondering how you think what you're saying about teacher quality applies to intercity schools with a high percentage of kids of color and I would like to refer particularly to Detroit where I do play through the work the fourth grade math performance scores in Detroit are the lowest in the country and it doesn't matter almost which school you go to in Detroit the financial master of Detroit so I would say you sort of discounted what Coleman said about family being important the financial master of the Detroit schools Robert Bob said they have two rules now for the kids coming to school in Detroit one is that they must get a good night's sleep and two the parent must see the kid gets to school on time now I mean you could say what you want I mean about those being criteria but I mean that's the reality of the situation the environment you go around the streets of Detroit one third of the city is about to be told those I mean what do teachers do in a situation like that not with Michelle we used to have the same problem in Washington she does and so let me respond from the top first I don't disagree for a moment with you I'm going to actually respond from the bottom first I think that there's lots that we could do to improve the learning of kids that's not part of the schools we could in fact make sure they have a good night's sleep we could have them get them around time and there's lots of other things the problem there is that for the most part we're hesitant to intervene in what goes on in the household in the family and so I emphasize schools because that's what we can intervene with that's a public policy instrument secondly I gave you the response of schools in Gary Indiana which is a prior version of Detroit maybe not as bad but they're close they had the steel plants and so forth my measures of performance are what's the gain in learning that goes on in classrooms as opposed to are they at some level of proficiency are they at a high level and what they do and what those estimates of teacher effectiveness or quality have to do with the average gains in student achievement across classrooms some classrooms the teacher is getting half a year of learning some identical as far as we can tell in terms of family backgrounds and incomes and so forth the teachers are getting one and a half years of gains and that's what I want to relate on I would like to do more in operating other dimensions but I don't know how to do that effectively all of our attempts to make parents better teachers or better parents I think are have kind of failed public policy attempts to do that we have attempts to introduce preschool which is essentially trying to intervene with the family I think in our family at an earlier point in time we have mixed results on whether that works and in what circumstances or how to do that so that's why I emphasize teacher quality but I think there's a lot that can be done in Detroit but right now right now if you're in one of those Detroit schools and you're doing a good job that knowledge is masked we just see that the kids in the fourth grade have 2.8 years of reading proficiency as opposed to the gains that are made in some classrooms and that's what we have to change Is there evidence for interaction effects that is the measurement or the performance of the teacher depends in a quite systematic way for the colleagues in the immediate environment or the principal some of the policy prescriptions seem to go in that direction if you immerse someone in the midst of good teachers they will perform better than in the midst of poor teachers that evidence is hard to put together we don't know that for sure the reason why it's hard to put together is that the estimates I've been giving you there's a lot of machinery behind the screen you don't want to pull the screen aside too much but one of the things that's behind that screen is that most of these estimates are the most conservative you can get that come from just looking at differences within an individual school and the reason for that is that it's really hard in this business to separate out the choices made by parents of what school to go to and the principals in the school and so forth and so the attempt is to measure variations in quality just within an individual school now there's some other evidence and I've been working hard at it but it's hard to do that quality of principal matters and we have lots of anecdotal evidence about the principal with the baseball bat and so forth or the charismatic leader but there's stronger evidence that suggests that the principals probably make a difference but they're really hard research questions to get at in a systematic way so the evidence is not what one would want one back there yeah well I mean I would use some quantitative information if I had it like estimates of value added in statistical models that I have here but I would rely a lot more on evaluations by principals and serious evaluations by principals and peers if you could get it done and the problem is that part of this is breaking down a culture in which the unions want to deny any difference among teachers and so they don't want to in general participate in evaluation systems where I think it's in their interest to do so the work that Brian has done suggests compares value added measures of performance on statistical measures with the evaluations that are given of teachers by the principals and what it says to me is that you can roughly put people into a good bucket a bad bucket and an indescript middle but that's all that I think that you really need to use that information and so I could think of all kinds of systems I think that in all cases they have to be collectively bargained because we have existing union contracts and that you have to do this my answer was always I should say I'm a little bit wary about talking about this my answer was always what I would call Michelle Reed Michelle went into Washington DC and said look I'm going to dramatically increase your salaries but in order to get the dramatically increased salaries you have to accept some employment risk and the unions said not on your life over our dead bodies you're not going to do that no matter how large the salary is you can't have a system that runs like that and the two of them unless the Washington Post was different today than it was yesterday we talked about the horns on this Deborah so at the beginning of the talk you asked the question does it matter what students know and I think what we show this is that the difference between the kind of payment and how good the skills you get you get much better it's much better to be able to see what they're like so I think you're already given what I know you believe about the fact that given the craziness of economists it's not a bad argument it's what you're saying no no not what I'm saying in an environment where very few people appreciate the importance of teachers I appreciate the argument and it's extremely clearly made so we get up to the point where you and I start to part companies right at the end of the talk and yet I'm not really part of the company because I think what you're responding to is something rational teachers matter this much and you can see that removal of giving back teachers can have that kind of effect it's a rational argument so I'm asking sort of the next step if attainment was so poorly related to the things you were showing us at the beginning you could argue, I'd like you to comment on this you could argue that the current professional development basically unrelated meaning like attainment can you comment on the prospect that suppose we just say teachers who still have an interest in growth that is the growth of teachers who actually have a capacity to do this kind of things can you comment on what it would take to get to a place where we would have that in our disposal as well as in the policy yeah so you know well we've discussed the same issue in different contexts so I think this is a complete black hole that we have in fact pursued that policy and I am not going to comment we have a research we have followed a research agenda designed to identify characteristics of good teaching or your styles or what have you and I think that that has not proved to be a very fruitful research agenda I am willing to believe that there is knowledge to be had there I just think I'm too old to see it I mean that it's not going to get there in my lifetime in a oh it is well no Debra is asking no no it's sort of identifying characteristics of good teachers and maybe something that we could all do or teaching I'm sorry there is this distinction that I always blow and so except my you can substitute teaching any time I say teachers if you want and it's been fun to me and if it makes it clear please do that the can we identify some of the characteristics of good teaching that leads to higher performance with the hope that we could then take the University of Michigan School of Education and produce people that had more of that talent skills, characteristics whatever and I willing to believe that it may be possible I just don't see it happening because I've seen a lot of work that's pursued that issue and has been very unsuccessful now if you want me to go deeper I'll speculate on why that's the case and why I think it's the case is that we generally have both with the characteristics and the portions of teaching basically a linear model of the world in our head where if I have .2 of empathy for the kids and .1 of subject knowledge and .05 of Schumer and I add these all together and I can get a good teacher and my view is just very nonlinear that people with very different approaches and skills and knowledge based and so forth can produce the same outcome or people with the same characteristics produce very different outcomes because individuals make adaptations why is it that I never tell a joke when I give a talk well there's an obvious reason that I know that it's not going to work and the same way that all kinds of other things that I just view teaching is so much more complicated than our current analytical abilities that I don't think that we're going to make much headway on that now there are smarter people than I am that are more trained in this and I'm willing to give you a chance to in fact show that it's possible I'm just giving you my bet and if I had to bet I would bet on identifying genes as opposed to traits of teaching but that's that's my bet and I've identified it as my bet so and I also have a bet not with you but with Bob Gianta about exactly you know Debra and Bob Gianta and some other for that in which I have a bet of unknown dimensions but it's on exactly this whether in five years we'll be able to identify good teaching or not and I'm willing to take any money that Bob will give me in five years I just want to follow up and ask if we accept your estimate of 6-10% of deslection so can you just do a back of the envelope calculation of how many teachers that would be to would be deselected in the country because as I recall there's quite a few we have 3 million teachers so it's 300,000 yeah and where would the replacements come from and what kind of incentives and did you build incentives we have replacements into the model in your estimates of what the payoff is so what I don't know is the cost of doing that and you're right I should address that economists should address these things what I do know is that we have a huge reserve army of potential teachers I mean every year what do we do we produce three times as many teaching degrees as we have teacher openings and they go off someplace else but if I go beyond that which I would go beyond those that are trained in teaching to teach for America times n we have an infinite number not an infinite number we have a finite economy we have a huge number of people that would be willing to teach if they were allowed to which they aren't allowed to today they'd be willing to teach and who might be very good at it there's a lot of evidence all of the horse races between teach for America and what's the other program in New York City teaching fellows program and traditional teaching training says the horse race is a dead heat so my estimates let's be clear here my estimates are artificial in the following sense I'm saying that we can permanently get rid of the bottom 6 to 10 percent of performance and replace them with an average now that takes a lot more work because you have to keep buying better teachers than you have on average today well if you just randomly choose from the population each year you would have 6 percent more but I would couple it with traumatic expansion of the numbers and right now today each year we allow these bottom 6 to 10 percent to continue teaching as long as they want to teach and what I'm saying is we will not allow the bottom 6 to 10 percent teach as long as they want to teach so taking out the bottom 6 percent says lower the average years of treatment that they give kids of harm that they give kids by getting rid of them quicker and we'll still come out ahead but there's a whole series of things on cost side that you're absolutely correct that I have been avoiding about nothing about the supply function of good teachers it would be that actually I mean I have to call the people part of the that's great really nice talk thank you so much thank you