 reading the milestone, states are testing to determine whether or not reading in math is improved. And stated assessment tests generally focus on reading in math and so as a result other subjects also get squeezed out of being evaluated and because other subjects are not being evaluated with equal intensity, the theory is what you don't evaluate gets neglected, gets short chained. And there has been a study that's been done, and we'll talk about that study a little bit later, but a study that looked at how elementary school teachers nationwide are portioning their time each week. Now this is just elementary schools, it's not secondary schools, it's not middle schools or junior high schools, so it's not only elementary schools, but the theory is this is where the burden of no child left behind is falling mostly upon because it's really an emphasis on primary education. And basically for grades one through six, you can see reading is up, more attention is being given to reading. Math is actually down, science is down and history is down. One theory that a scholar at Brown University has predicated is that math doesn't take as much time to teach, you can't really use time as the singular metric for how well students are acquiring, because it's many times problem sets that are required as homework assignments or frequent quizzes and so forth, so it may be an outline that there's been success in math even though it's being taught less. So this is one criticism, one set of criticisms. There are some others, and these are implementation problems that also I think overlap into the field of evaluation. Federal law requires students to be tested annually. That is part of no child left behind. Again, states do the testing, but the federal government wants to see the scores and will use the scores as a measure for how much to support states in improving their educational efforts. Basically do well on the test scores and you'll get more support. Do poorly on them and you'll get less. Now if you're a state and you have to go through the process of testing, what's going to be your motive to improve the math scores, which means you're going to be testing frequently, you're going to be concentrating also your efforts and learning out of times, but it's up to states to devise the exam. One theory in Martin West at Brown University who's the same guy that did this natural study insists that because it's up to states to devise the testing process, the result is that critics, or at least are stating, and West interviewed people in various states around the country, that states are making their tests easier so it appears that students are doing better than they really are. What do you think? Is that plausible? Isn't there some general principle of separating implementers and the evaluators for reasons just like this? Yeah, that's exactly right. And essentially what West is asserting is that that's not being done. You don't have a firewall between the implementing of the program and the evaluation of the program. He proposes in fact that what ought to be done is it ought to be evaluated in a national way. And it ought to be evaluated independently of any agency. It shouldn't be done by the U.S. Department of Education. It should be done by some independent examination board, which presumably might be made up of universities, teaching institutions, and so forth. Some accrediting organizations perhaps. But that's exactly what, by its own count, Mississippi is tied for the best score in the country. Not to pick on Mississippi, but you know, the statistics would suggest that that's going to leave one to be a little bit suspicious. Mississippi is a state that historically has not, on a per capita basis, supported public education at a very high level. The legacy of Mississippi and other states in its region is that for a long, long, long time, public schools have been de facto segregated, if not de jure, meaning by law, segregated. And it's a little suspicious when all of a sudden you kind of rise to the top of the scale or rise to the top of your own anointed scale tied for the best score in the country. Particularly when you compare it to a test that is administered by the U.S. Department of Education, it's a test that has no binding authority. But it's simply a standardized test that's been devised by the U.S. Department of Education in Washington called the National Assessment of Educational Progress. And the state drops to 50th place. I think there are 50 states, right? So, okay. So, whoa, that's quite a difference. And a whopping 71 points lower in terms of the testing score. I'm assuming that it's probably like 100 points. So 71, so the score is 29 out of 100. That's usually a failing score on anything. Okay, enough on that. So, Dan at Google evidence does suggest that when time to teach increases, so might score. So that's a good thing. But basically here you have a problem with testing. Implementation versus evaluation, the separation of the two. We'll come back to that issue a little bit and talk about evaluation and its own policy process. But this is a real challenge. Questions about this? Questions or comments? Yeah, I mean I studied education policy this month. I think it's interesting that the pioneer of NCLB would be Diane Bradwich. Yeah. And then that was, she wrote a book in here later, how NCLB has caused more damage to students learning and kind of unraveled the effects of that. I mean it makes its educations very dependent on that swing. Sometimes this is going to be a standardized education process. There's more of this accountability than it kind of swings back to control and develop a little bit of standards. It's always a dilemma. It is a dilemma. And in fairness you point out correctly, there's a pendulum. You kind of go from one here to be an extreme to another extreme. And in fairness, and this gets us into the area of policy evaluation, this is a good place to approach this. A lot of it has to do with societal expectations, which is what both of them talked about in their analysis of policy failure. Oftentimes what we evaluate, and what we recommend as a result of our evaluation, is a reaction to a perceived failure of a status quo. Well we're not testing and standards seem to be going down. If so, fact though, if we test and set the bar really high, things will improve. It makes sense intuitively, so you go to that extreme. In fact of course then you generate all of these unintended and unanticipal consequences and then you're kind of back, maybe not back to where you started from, but you're in the same situation of not really having a change in improvement in the policy outcome, which is the goal of the evaluation to see whether or not an outcome is improved in some way, some goal is being achieved. Is there any other questions? It sort of seems like a cost containment argument that rather than looking at the structural way that we try to intervene on these problems, we just try to make a broad intervention, whereas if you've got changes of how to think in a very multi-level, you know, emergent curriculum sort of way, then they would have universal ways of thinking, but we have to change the way we do professional development at the same time. Yes, that's true. And it's a long... Yeah, I think it also, and we'll come back to that point in a little larger context, oftentimes when you have a failure like this or a less than satisfactory outcome, it's a term I'm going to use later on, but I'll introduce it now, it's a multi-mode. You have a failure in several aspects of the policy system. You have a failure jurisdictionally who controls what, who's responsible for the way. You have a failure in terms of defining the problem. Is the problem that students are not testing well or is it that they're not learning well? And are those two things necessarily equal and accountable in the same way, and testable and evaluatable in the same way? And then there may be a further failure in terms of expectations. Is it, in fact, reasonable to expect that educational outcomes will improve simply through providing more time in reading and providing more attention in math? Or are there deeper things that need to be done within the educational process to achieve those outcomes that may be more holistic? Maybe all of those things are true. Well, this last summer there was an attempt instead of reading the students to be testing, there was that L.A. Times published the teachers' ratings, and it was met with so much criticism, mostly I think it names things, but it just seems like no matter what you're trying to rate, there's a lot of criticism. Yeah, there's an old saying among attorneys that when you put somebody on trial for, say, a tort or liability, not a crime but a civil suit, what juries look at is not what went wrong, but who's to blame? We want names. Who's responsible for those? We want a flyer. We want to find them or we want to put some big tort liability on them. There's this notion of personal responsibility that becomes kind of a moral imperative. I think your notation about the L.A. Times series is a good example of that. Name names, it sort of sent the message that, well, you know, it must be their fault that the kids aren't learning. Is that really the right message and does that really solve the underlying problem or is it a nice way of, you know, getting attention? They're obviously doing it. Look, they're really poorly on their scores as teachers. Well, I think that's a huge problem too. I'm starting to actually close schools down because they're not meeting the school or the state requirements, and so it puts a lot of schools in jeopardy, and so it's not good for the students if their schools are shutting down and you have to relocate, but it's kind of hard. You can't blame your teachers in an aspect because, for example, in San Diego where my mom teaches, she has a lot of bilingual students who don't come in and don't have the support from home and since they got rid of bilingual education, they're not going to be able to do well as schools in San Francisco students who are native to the language. So there's a lot of problems with the way that they're going to be evaluated. We have to take into account other things other than just who's teaching and why are they teaching that. Yeah, exactly right. And how do you evaluate all of those things, and then how do you make improvements on all those other variables that may be outside the control of the schools, much less teachers? So on the other hand. Well, I guess just reflecting what is generally been said, it seems like a very modernist way to approach the complex problem, the idea of quantifying progress. Right. And in other places, well, this is done and racism is similar to it does not take into account critical thinking or even focus on other topics and other subjects that you can argue are just as important. Like even music, which involves several forms of learning styles. Math, history, physical education. Art. One subject I did well. The only one. That's my column. That's exactly right. I mean, again, it comes down to what we value, what we quantify. We'll come back to this a little while and talk about evaluations, but there's a lot of symbolic uses of terminology and evaluation. In the 1960s and 1970s, whenever we had a policy problem that we wanted to single-mindedly attack, we would say we're having a war on. Now, if you can use the metaphor of war, mobilization, self-sacrifice, discipline, focus, attention, it's great. It's one thing to make a war war. It's another thing to have a war on poverty, a war on racism, a war on urban blight. Those things don't quite fit. A couple other questions, comments. Well, I was thinking that it also seems like no child left behind is just going to exacerbate problems between, it's going to make good schools better and bad schools worse because of the way the funding is evaluated. It seems like it would make sense to fund schools that we're doing worse because they clearly need improvement. Yeah, and that exactly is one of the things that's embedded in Martin West's critique. Some of his writings, you see this, West argues that what you're doing, and this was maybe the intent, raised behind the law, is to actually create a kind of a competitive system among public schools. We know, of course, that many of the people that initially advocated no child left behind felt that public education had profoundly failed because it had somehow declined with its standards and expectations, and there was also a resonance between not just the testing idea but of supporting private schools and making it easier for parents to help create charter schools to divert resources from large public schools to specialized schools for certain populations, and so on and so forth, many which may be good policies, not criticizing them, but the fact is what it's doing is trying to transform the public education system into a kind of a market where schools will compete. Better? You get more. Yeah. It definitely plays in the social determinants of health because poverty taxes indicate the amount of money that goes to schools if you go to an amateur area and you have a right and your schools are just as better, so it perpetuates the poverty issue. Yeah, it may in fact do that unless you can actually change the way in which these metrics improve, not just by opinionary effect, but by actually bringing in regulations. I was just going to make a point that you made earlier that it was physically difficult to try to justify giving more money to schools that are doing employment without saying how that money would be used to actually improve what they're doing. I know that there's a lot of political opposition to just throwing money at problems, but additional funding could absolutely help. It's just that we need a clear and directed, yet flexible way. Yeah, that's an excellent point. Accountability. I mean, this is a thing that we'll come back to in a little while. We'll talk in detail about evaluation. Evaluation is based on an expectation that if you can identify a failure, you can then identify means of correcting the failure. This is one of the things that both Vincent Hart talked about and not perhaps surprisingly, as you read through that chapter, one of the things they point out is that public policy is more often right than they do succeed. They somehow fall short. Often because either they simply don't meet the expectation for one reason or another, or the expectation changes. It's a free-thing goal, okay? People would use the metaphor slippery, slow, but you have a certain goal. You institute a policy and then you say, no, no, wait. What we really want is better school. What we really want is for having the best schools rated by some independent whatever, whatever. And then you keep changing the goal and then what happens, sure enough, is that you fall into this trap of an unattainable goal and a non-accountable objective, an objective that can't be overcome. So let me wrap up and segue over to evaluation by suggesting these kind of summary points to chew on and think about in terms of policy implementation. No child left behind, I think, suggests this is a problem. Many other public policies also allude to this problem and have this problem embedded in them. Governments often fail to acknowledge the need to not to synthesize to combine in some way. What experts and policy analysts know or think they know of policy with local indigenous the distinction here being indigenous often is knowledge that's inhabited by certain populations that are definitely susceptible to the problem that may be outside of the mainstream minorities, for example, the resource. Or street level knowledge. This is a common problem in public policies. A constant obstacle. We're going to come back to a couple of examples today and on Thursday we'll talk about evaluation. Over-reliance on experts and this is a point that Stone makes that I think resonates strongly with what we've been talking about for the past 20 minutes or so. Oftentimes when you rely on experts you not only have programs that are structured in such a way that they ignore street level experts or become deaf if you will to the need for making modifications based on the ground results but you also can produce efforts that are absolutely counterproductive that end up intimidating or even trying to indoctrinate the public efforts to compel immigrants to abandon their traditional folk ways. Because Stone talks about that as being an example of how public policies often, in fact produce very counterproductive results by adopting policies and implementing them in ways that ignore the actual consequences on the ground. Or the safety of nuclear power. You must accept that we as experts have done all of the analysis and we know that it cannot fail and if it does fail there are backup systems on top of backup systems on top of backup systems and that's the end of it. Didn't use this example this term but one of the best examples of the failure of this hubris is this approach to ignoring street level knowledge and ignoring the public and over relying on experts happened some 30 years ago now Three Mile Island which is a nuclear power plant in Pennsylvania I don't even remember reading about this the power plant had a multiple failure in its safety systems and the power plant down the core went critical and there was no cooling water it was a very very dangerous situation the only thing that averted the disaster was that it had a huge concrete containment structure which did work contained most, not all but most of the radiation the federal government the nuclear regulatory commission said there was no danger however as a precaution all pregnant women in a 60 mile radius should probably get the you know what I mean now, what would your reaction be to that? I think leave two, okay? I suspect all of you would know you get the message this was an effort at trying to take risk information giving it a conservative estimation and trying to reassure by instilling confidence in the public it had the exact opposite effect fight severely essentially when the NRC said that which was a wise thing to say everybody got the message that these guys really don't have it under control and this is really a bad thing and we don't get out of it and that's when the governor of Pennsylvania Danny Richard Thornberg intervened and said, I'm taking charge because you guys obviously don't know what you're doing so we're going to evacuate everybody until it's necessary now and you get the picture two other points translating expert knowledge into information that's actually usable by local populations through sensitivity to local cultural practices norms and traditions is essential this is something that policy experts often don't do and experts in turn need to incorporate community perspectives into their professional judgments to gather all the information needed for making the decision so this is the blood who's done a lot of work in this area in regards to environmental policy and Frank Fisher who's done a lot of work on this in regards to science and technology and risks technology and so forth both argue citizen participation is important this is one of the reasons why you want we'll come back to this point next week we'll talk about future public policy why you want the public involved in policy decisions it's not just to get them to buy off on policy it's to get them to share information about what they think is really important that's how you make sure how the policies work at least in a democracy where we believe the public policies ought to be working on the public good community expectations about schools about local public health and nutrition we talked about that are also likewise similar examples of this problem questions about this does this make sense this is really a big, big problem Victoria one of the things that stands out to me is before when we were talking I think you mentioned one of the issues with the no child left behind is that it really stripped the evaluation of it was stripping any context from the broader social context these points bring in the context because people who are experiencing the education system have you are navigating these places every day within the context of them that kind of bridging of an expert in the local brings in the context you're exactly right, I think that's an excellent point and it is very important with this discussion because again it comes back to an expert we think of a policy expert from an evaluation standpoint we'll look at some failures we think of an expert in policy evaluation as somebody who is very studious who looks at a lot of aggregate data who interprets it who figures out the directionality and the intensity all of this kind of thing but it's decontextualized policies affect people they affect their lives in various ways a danger of a nuclear power plant nothing down it's not just a set of scientific risks that can be calculated and computed it's also people's lives and welfare thinking about it in very personal terms schools it's not just about scores that's my kid right it's a very good point and we'll not see other examples of that as well so let's turn to evaluation outcomes and evaluators it should be pretty clear by now that evaluation is often termed the final stage of the policy process we'll see an illustration of that in a few moments it is as we've talked about throughout the quarter and as Ray discusses the heart the firm evaluation is often viewed as dispassionate as detached as scientific it is usually neither of these things it's neither scientific nor detached it is agenda driven it is a part in other words of the policy process it is valuational almost every evaluation has a point of view something that it is directing its attention to because the evaluator or the people paying for the evaluation believe that that point of view is important and it is political so a couple of issues just to kind of open this up the range of organizations doing evaluation and the approaches they employ and secondly the divergent ways in which problems can be defined and evaluated both by policy makers and by analysts who are the evaluators well evaluators take numerous forms we've seen some of this already evaluators can be issue based they can be philosophy based you can pick your point of view do you want a liberal think tank do you want a conservative think tank do you want a vegetarian think tank pick your point of view they have different sources of funding, clients and products and evaluation has also become very much of a boutique industry particularly in the united states and in a lot of other developed countries essentially you can choose the kind of analysis you want we talked a little bit about this few weeks ago we talked about the role of the policy analyst you can analyze benefits and costs you can analyze risks you can analyze public support you can analyze even all of those things based on your goals your policy orientation your analytic approach the major products that you want have a stage in the process you want to evaluate let me say one other thing that's sort of embedded in this but not explicit you can choose the form that you want this evaluation to take do you want it to be a high minded academic treatise that's only read by a handful of high minded academics or do you want it to be in a newspaper or on the web or everyone will read maybe create a blog the way in which you market the evaluation is also a very important part of the process this is one of the challenges that we face in public field public policy feels like education where there's so much information out there some of which is not really rigorously reviewed in terms of evaluation that wherever it appears whether in a newspaper or a popular magazine or the internet often times it's the same attention as a study as something that may be more refined and more rigorous a few weeks ago I showed you a similar diagram outlining the entire policy process the inputs the agenda setting policy making phase that black box in which policies are formulated and implemented and then finally the outputs the laws, the rules, the regulations the programs that are created by governments well this big blue loop is feedback and that's really where the evaluation fits in the policy process indeed it is the last stage but bear in mind if you're following the logic of this diagram it's also the first stage it's a loop a closed loop perhaps but it's a loop on the left it's evaluation and policy impact process are policies meeting their goals and expectations if they are how can we make them even better or how can we change them in some other way if they're not how can we go back and revise or change the policy to make them succeed let me go through some of these points at the bottom of the screen what I want you to carry from this discussion and from thinking about Bolvins and TARP and other readings that you've looked at this quarter when it comes to evaluation is to realize that every stage of the policy process every stage features an interaction among competing interests just as there are competing interests that are seeking to create public policies as we saw some government analysis some time ago bear in mind that there are competing interests who are just waiting to evaluate policies and they will evaluate them in the virgin ways because they have different approaches and philosophies and expectations all of which reflect those interests this also explains why at some level all public policies could be seen as failures because if you stop and think about it there are no outcomes in any policy that would ever be viewed as favorable by everyone think about the recent debate over the healthcare issue that we talked about a couple of weeks ago if you are opposed to the notion of government providing an insurance guaranteed set of insurance coverage for the uninsured do you think that is a bad idea of public policy to be engaged in if you think that creates too thick an approach to the role of government would any evaluation of the effectiveness of that program please you probably not except what an evaluation would say policy abjectly failed and thus we need to do what trash it get rid of, end it, terminate it no outcomes will ever be viewed as favorable failure is in the eyes of the beholden but let me complicate matters just a bit think about this one even a success in some ways can be a failure assume that if a policy outcome is viewed as widely successful will we want to leave the issue there this is great you know the government has been spending billions of dollars over the past 45 years through the national institutes of cancer and sure enough what's happened to cancers what about deaths from cancers mortality we're living longer actually mortalities are going down now we're finding cancer sooner more of them are proving to be treatable or operable we know the causes of cancer so what should we do we're spending lots and lots of money on cancer research and other forms of medical research do we say it's fine leave it alone don't break it or what spend more spend more and don't just spend more the same model and do what if it's work for cancer replicate do the same thing for every major disease create a national account that will direct money competitively to science to universities to medical centers to do research and to develop treatments that will get into the marketplace and get into the health care industry and solve these problems now is that a success or a failure or is it a little bit of both who you think depends on your perspective depends on your perspective you're sick if you consider if you consider this kind of a Darwinian system by the fact that the policy has perpetuated itself I mean if you take that point it doesn't really work the same for all diseases because the health care industry is not going to support research for diseases that affect people that can't pay human amounts for treatment cancer affects older and usually wealthier people that's it we're going to pay to do the research to treat the illness the assumption being that once the treatment is developed somebody is going to pay for it either through insurance or through their own means that may work for cancer it may not work for AIDS are we spending as much on AIDS are we spending as much for case acts but I know what case acts is are we spending as much for no there is a definite selection process it is somewhat Darwinian but I would sort of resist using a Darwinian analogy here and simply say that there is a set of choices that are power based and it really do resonate with who speaks the loudest and who gets the most attention and resources all programs that are successful fall short they fall short in some way national parks are wonderful let's have more it's not enough to have national parks in rural isolated beautifully scenic areas let's have national parks in cities so that they'll be closer to people being able to get to them it doesn't really raise the question whether or not that's the best means of protecting park lands is to create national parks maybe it's better that they be state parks or city parks it also doesn't answer the question as to how you then prioritize the allocation of the resources for this good it's good but does every park deserve equal protection and will there be a steady stream of resources to support what we want to call a national park intuitively you say of course you want national parks but wait a minute is that the best model for protecting public lands or even creating or establishing public lands cancer research public works programs way how do you see it personally about recognizing that clinical research they surrender to the notion that things are place based undermining all of the research the notion that things are generalizable so that kind of protects I think this this dichotomy where you have like the cancer research I mean clinicians are a thing to do the scientific method and the underpinning is that it's generalizable so what do you change based on the street level bureaucracy or whatever so I don't know how to reconcile okay so let me give you one this isn't going to answer it either but it's one possible direction toward an answer because this is in fact exactly how critics speak to the kind of argument you're speaking to it's hard to argue against cancer research it's very difficult particularly since it is a ubiquitous set of it's not one disease it's a set of diseases that affects different populations has response to different forms of treatment has different both physical causes and environmental parameters okay so how can you argue against doing research to treat an illness that affects a lot of people it makes you sound cruel and inhumane if you say that but if you start to think about it if we know that many cancers are reducible to lifestyle choices if we know that certain kinds of lung cancers resonates strongly with smoking or working around asbestos how many of you work around that just check working around asbestos or working around certain toxic hazards in the environment would it not perhaps be a better public policy choice to do what to prevent those activities to avert them to protect against them to regulate them eventually what you would do is then bring down the rate of cancers that are quite preventable and you might then have more resources for TASACs and other diseases and ailments that doesn't quite answer your question general vicinity and it's not a question of do we not care about the problem of course we care about the problem the point is what evaluation research that is the study of good in that evaluation should teach us to do is to accurately define the problem and the best avenues for attacking it some problems clearly have to be attacked by research into treatment in addition some can best be attacked by prevention of versions circumvention regulation I think it's hard when you're working in evaluation you are from personal experience you're isolated from what maybe the actual problem is so your immediate problem that you're evaluating is maybe not the bigger problem that we're trying to address that's right and you're being tasked to evaluate this, yeah, that's exactly right I think just in practice how do we negotiate? how do we negotiate these avenues and in reality what we normally do with health programs is we expect Congress to evaluate and make that combination of assessments when they fund programs. I'll use the example of the National Cancer Institute was one of the very first national health institutes established by the federal government, it was back in the late 1940s there have since been a multiplicative number of national film and film institutes including one for environmental health national institutes do these institutes talk to one enough it would stand to reason that they ought to in order to coordinate research dollars and attack the problem what do you think? they don't talk to one another they work in stove pipes they're not the only ones by the way most federal agencies work in stove pipes, most state most departments that you see I work in stove pipes I can say that because it's true I see it every day as a department chair we wish it were otherwise but unfortunately we tend to work in our own stove pipes it's up to Congress to say you know what we're spending all this money on cancer research maybe we ought to shift some of these funds to environmental health research so we could prevent some that's what committee staffs and Congress do this is one of the points that Radden makes some time ago that this is sort of where the sausage gets made is that the best way to make this call? is that the best way to do the coordination in terms of policy analysis? probably not because it's going to be very interest driven what do you mean you're going to cut some of the national cancer institutes to steer some of the money to the national institutes no we have to spend money on growth not one or the other or shift or refocus the attention or evaluate policy outcomes in a broader way each program wants to competitively evaluate itself to put what it's spending its money on in the best way or to warrant additional support and additional funding before you get too pessimistic you have every right to be pessimistic you want to talk a little bit about policy evaluation from an overview standpoint so it obviously is an interest laden component of the policy process and if that's true then how is it conducted and how do we determine whether something is going to be viewed as a success or a failure and obviously the answer is subject to divergent perspectives so for a few minutes on time as a perspective in policy evaluation this is one that Volvins and Tarp devote a lot of attention to but I think it's a very important one we're actually going to be looking at some examples and illustrations of this a little bit the data for evaluation let's get into some very basic methodological questions some of this picks up also on the past few minutes short versus long what's better do you want longitudinal data or do you want a one year snapshot well if you want accurate projections and interpretations you want a longer term view the problem however ironically as Volvins and Tarp point out is that policy makers want what they want the immediate results we're spending several billion dollars on this program that was a year ago what have you got to show for it how am I going to go to my voters in my district and say we ought to support this program because it looks like you feel really good about the data and maybe in eight or nine years we're going to address that problem satisfactorily you're going to have to give me something better to go on short term actually is the more common even the longer term will be better but then there's a flip side to that the longer the longer the time period over which you evaluate and assess a program the more likely it is that the underlying public support and values that favor that program may do what may change may evolve support may wax and wane we talked about electoral cycles public opinion cycles some time ago and come back to that point just a little while with an example if a policy regime by regime we mean an array of laws and rules and regulations and programs to institute some policy outcome persists for a long period of time public attitudes toward the policy are likely to change and with that change sometimes ironically due to the success of the program you promised you could accomplish your goal and sure enough you did now I ask you why do we have to have that program and there's a time paradox if a policy is successful it is likely to be continued beyond its original anticipated duration I mentioned this example last week I want to allude to it again back in the early 1970s when the federal government was running surpluses that means that they were taking in their money and they could find a way to spend what a nice problem how many of you have that problem by the way with your checkbooks never alright so we began a program called block grants basically blocks of money that were given to states to spend however they wanted there were no strings attached the only strings attached were that you could not use the money to create any program that would discriminate against any categorical group outside of that you could spend the money for whatever you wanted it to be it was greatly flexible in terms of what you could spend the program money on it was extremely popular because the states had extra money that they didn't have to raise from their own taxes and sure enough the money was generally spent on things that were politically popular and most embraced by large numbers of people in states quite high ways like public works of the day energy price supports bovins tartlynch in this one originally back in the 1950s the united states government created a lot of price supports for fossil fuels basically these price supports were guaranteeing a minimum profit to energy explorers who found new sources of oil, coal, natural gas, fossil fuel even uranium when they thought nuclear power was going to grow and grow and grow the idea was to create a program that would develop incentives to exploit these resources and guaranteeing good supply of energy they were supposed to be for fixed amounts of time usually five years or so then of course they became renewed because they proved to be very popular the problem is over time it increases the likelihood of failure what kind of failure well in the case of block grants the failure was what the states had to reconcile you're very very close that's a very rational outcome what you just discussed here's what actually happened the surplus ended the programs were so popular that the categorical block grants continued despite the absence of a surplus and the appropriated funds that added to the deficit then, after about a half a dozen years they were eliminated and instead put into a series of specialized categorical grants that you would have to apply for so instead of just a block grant the funds that would have been available would put into highway trust so that they could go to these projects it would have been rational to just end but that would have been politically suicidal for congress to do so kept the program going at least for about a half dozen years finally ended under Jimmy Carter late Jimmy Carter's presidency he didn't get re-elected energy price supports how is this an example of the time paradox did energy price supports work did they support energy exploration did they make it worthwhile for energy companies to take risks in finding new sources of petroleum and coal and natural gas what do you think artificially it did artificially it did because if you subsidize it but realistically it's still not so what happens if you take away the price of work which happened in the 1980s by the way then they're no longer viable and what happened to energy prices they go up of course there's plentiful energy but you just gotta stop eating so that you can afford to purchase it two things happen one is the prices skyrocketed because basically what the price supports did was artificially attenuate inflation a certain amount of inflation and a resource is natural when you hold it back for an indefinite period of time you don't eliminate it you only hold it back for a certain period of time there had to be a natural market adjustment and energy prices went up dramatically in the 1980s yes partly due to energy embargoes but mostly because of the removal of these price supports so that was one second thing and a lot of research resources for the future which is a think tank that Washington has done work on this argues that one of the things the price supports also did which was actually successful was it provided a means of lessening the risk upon energy companies who chose to explore four sources of energy say in the Middle East or in Latin America or in Sub-Saharan Africa Nigeria all of which were good in terms of expanding the base that you would have for energy but what's the unintended consequence that would be putting it very nicely yes not very friendly the risk of an unstable energy supply that could be vulnerable to being cut off or reduced for seeing yet another energy price shock in the past couple of weeks and simply stated and if you look at the data on this example data the amount of energy that the United States imports has over time increased the amount of energy that's actually domestically produced has decreased proportion reinforcing the vulnerability so you can have a success you can have a failure and let me add a third one you can have a successful failure or you can have a failed success make your pick this is in fact what often happens with the time here questions on this again you can't foresee or oftentimes it's difficult to foresee some other spatial and perceptual failures and then I want to actually spend a few minutes looking at a real policy that shows us these paradoxes I'm not going to spend a lot of time talking on this although we will come back to some of these a little later on on Thursday but these are examples that are cited in boldness into heart and also in other readings that you see in this quarter facility siting issues often are public policies that are frequently cited when we talk about policy failures and policy evaluation these are acronyms that many of you know from other context what's a NIMBY what's in the abbey not in anybody's backyard I'm not against nuclear power in my backyard I'm against it anywhere and a LULU there you go energy issues, waste management homes and facilities for special populations mentally challenged or ex-convicts these are also locally unwanted land uses these are policy programs that in effect fail before they fail they usually don't even get past the agenda setting stage because they are seen as issues that are so controversial and so likely to generate consequences that are unwanted and undesirable and in the wrong headed view of many of us this is the point that both of us are at may in the wrong headed view if we simply didn't build these things we wouldn't have a problem which is ridiculous you have a problem there has to be in a humane society some way of providing homes for mentally challenged people halfway houses for people out of prison to say that if we didn't have the facility we wouldn't have the potential for crime in our community or my property values might decline a little bit maybe a valid argument but it's a short cited argument you still have the societal problem of how do you home house populations how do you provide them with services and how do you address underlying problems that may have led to the need for these facilities to begin with simply saying this is a locally unwanted land use and put it all on the urban planner and say it's your job to figure it out us policy makers are too tired to figure out what to do with crime so it's up to you you figure out how to prevent that facility from being cited is wrong jurisdictional extension is an issue federalizing the state policy block grants would be a good example of that or internationalizing a national program taking a problem like trade for example and putting it up into an international jurisdiction and it's harder to evaluate and finally social expectation issues I want to go through these very quickly of the use of illustrations on your name when a program and its cultural values are out of sync civil rights programs for decades took this form public housing programs we will look at public housing as a specific case study of Thursday an interesting case public transit is the failure the fact that we provide public housing is the failure that we provide public transit or is the failure and the reason these programs often do not manage to become self sustaining or popular is because they are out of sync with the dominant values that we have as a society car equal freedom talked about that issue last week context of program funding limitation it's also a problem in terms of evaluation well public transit is great but it doesn't earn a profit by not only does it not earn a profit it doesn't really sustain itself it's cost has to be subsidized well I've got news for you every form of transportation is subsidized we should understand that's true of pilots by animals talked about the trust program last week and of course public perception are on anticipated consequences again land use decisions housing transportation many environmental decisions and a final observation before we go to an example think about this it's easy to say this policy failed because or it's easy to say the cause of the failure was but in fact we would be much too simplistic if we said that failures in quotes are interconnected and systemic they are failures not only of the outcome they are failures of the process by which they became policies we're going to see some examples of this the way you make the policy can often determine whether or not it's going to be a success or failure so there are political failures not just failures in programs questions one example I think it's a facility citing the issue that's under court review in Florida that permitted sex offenders in Miami from living within 2500 feet anywhere the children gather but what it actually led to was eliminating all sources of housing even for people who have already served their time and so it created a population of sex offenders that had to live under a specific underpass in Miami since it was the only area that was not within 2500 feet and so it's being reviewed as to whether you can continue to punish people after they've already served their time in prison and whether this is discrimination exactly and that is very interesting case too because as you know in the accounts of that case that I recall reading at least one account I think it was in the New York Times they were talking about that case is also exemplifying the further in a sense the further the consequence you're doing exactly what you don't want to do which is you don't want to take a population and sort of concentrate them in one area you don't want to concentrate poverty you don't want to concentrate people that have various problems of one sort or another led to criminality it is counter to things that we do know in good social science about how you don't solve problems how you don't address them well not by chance let me use an example that may not immediately come to your mind as a public policy any of you see the movie Apollo 14 some years ago great movie with Tom Hanks it's a true story by the way on the left are some pictures of that was me on my side wish this was Apollo 11 which was the first human landing on another world it's still quite unbelievable to actually have a national do something like this but we did and there's a footprint there's Neil Armstrong the guy in the ladder is Buzz Aldrin by the way lives in Laguna beach he's a retired astronaut sometimes comes to UCI on a desktop it's a great individual great stories to tell needless to say first landing on the moon that happened in July of 1969 on the right is a Saturn 5 rocket that lifted the Apollo 13 mission the one that was made into a movie which was an aborted mission it was a mission that failed it was to take astronauts once again to the moon to land them and bring them back to Earth but it was really a successful failure because they got back it's interesting one of the parts of that movie here you have all this high technology and they got back because of duct tape co-danger plastic bags it was really identified with that anyway that was the Apollo 13 rocket the very top the actual spacecraft of the land on the moon and the service module that carried the craft that's what exploded in the flight and by the way, if you ever go to the Smithsonian Air and Space Museum in Washington you can actually see this, this burnt capsule that's what brought the first astronauts back from the moon about as much room as a telephone booth if you remember what telephone booths were and the computer that powered that spacecraft was less sophisticated than it was pretty exciting program what's this program all about well, it's a good object lesson in policy evaluation the goal was to land an American on the moon and bring him or her back safely by the end of the decade President Kennedy in 1961 articulated this as a national goal this was a very optimistic era can do we can conquer everything it's all a question of just applying ourselves to the problem this was the Cold War so of course it was not just a scientific mission it was a political goal we gotta do this because the Soviet Union is gonna do it if we don't it succeeded by the end of the decade $10 million sound like a lot of money but in the 1960s that was real money and it was divided into three sub-programs which some of your parents probably may have some books or magazines old newspaper clippings laying around showing this these missions were called Mercury, Gemini and Apollo and each of them was to devise different spacecraft different missions and different technical feats all of which were necessary to prove that you could actually land somebody on the moon and bring them home Mercury was a one person spacecraft and they just showed that people could survive in space Gemini was a two person spacecraft and the idea was to show that you could dock a rover in space and do all the things that you'd have to do to eventually land on the moon and then Apollo was actually to land on the moon it was run by the National Aeronautics and Space Administration which came into existence only three years before this goal was articulated NASA was formed in 1958 and it was one of the most popular cold war agencies in the United States it was an agency whose aim was partly scientific it sold its lavish and excretive budget as being something that would help American science and technology and also political it would restore national prestige and of course land people on the moon some of you might remember another movie based on a book called the Right Stuff The Astronauts the people that participated in this program of course one person Tom Wolfe the author of that book once said that we don't have cowboys anymore so this is it these are the guys but think about this other benefits this was a program that had numerous benefits and right from the outset we knew these benefits would be built into it huge economic multiplier effects California where much of the program was based Florida and other Sun Belt states it was mostly in the Sun Belt that the money was spent for every dollar of public expenditure there was a minimum of five to seven dollars in that community in other investments other jobs other spending programs and we don't have a lot of time we'll pick up on this on Thursday but huge almost incalculable spin-offs telecommunications cell phones the technology for cell phones actually came directly out of the space program to be able to communicate great distances very small electronic devices personal computers and digital technology directly came out of the space program material science the spacecraft when they re-entered the atmosphere had to endure tremendous temperatures the same basic ceramic technologies have gone into things as varied as cookware and counter-tops we don't know what else aerospace biometrics being able to monitor medical activity in the human body and of course education University of California is a land-grant university which means we get money from the federal government for farms agriculture we're also a sea-grant university which means we get money to study oceanography and there is actually a program called the space-grant program we get money to study other worlds that's money that was spent out of the space program huge success huge success I think I'd like to comment on I mean we really shouldn't underplay the I mean you know what are you talking about the cultural success of it I mean I personally my girlfriend's father who opened Scotland and saw what was happening and wanted to be a part of it and everyone looked at him and thought that he was good and now he's he came over here to study he's now working in space part it's hard to under you're exactly right and that's the point I want to leave you with we're going to pick up on this on Thursday but that's the key point this was a program that was so popular it was impossible to envision how anyone wouldn't want to continue the program but we did why didn't we save the suspense until Thursday