 My name is Kevin Kerry. I direct the education policy program here at New America and we are very pleased to have this event focused on the new book by one of our New America Fellows on your Cominets, the test why our schools are obsessed with standardized testing but you don't have to be. In addition to its various programs, New America has a very robust fellows program every year. A lot of fantastic books come out and we're really pleased that this could be one of them. We're also joined and very happy to have Maury Sykes with us today who's the Executive Director of Early Childhood Development at the University of District of Columbia here in Washington, DC to also talk about some of these issues. So I know everyone has come here to listen to Anya talk about her book and so we will we'll start with that directly. So in reading the book and I think if this were on John Stuart I would do this. The camera would pan in for a moment. I don't know if we can do that out there. So you start your book in the introduction and I'm gonna read from the second paragraph which says as an education writer for the past 12 years and as a parent talking to other parents I've seen how high-stakes standardized tests are stunting children's spirits, adding stress to family life, demoralizing teachers, undermining schools, paralyzing the education debate, and gutting our country's future competitiveness. So that sounds pretty terrible. We would love to know how you came to those conclusions and what brought you to write this book. Thank you so much Kevin and thank you to the New America Foundation for supporting my work and for hosting me here in this wonderful venue and also for all of your thought partnership over the years it's been really great. So the reason you know that that sounds like a damning bill against standardized testing and I did not set out to write a book about standardized testing. I'd written a previous book DIYU which was about innovation and higher education and transformation and technology and economic change and globalization and all these forces moving within the higher ed world and I wanted to write a very similar book about K-12 education. I had a child and I was interested in K-12 and I had heard of ideas of things that were going on in K-12 and pre-K-12 that I thought were really fascinating about social emotional learning, about technology use, about homeschooling, unschooling, about various types of uses of connected learning and educators getting connected and I wrote a proposal about these things and my agent said you know there seems like there's something missing from the center of this all of these movements all these ideas are very much at the at the edges of public education. Why is that? And I said well because of standardized testing changes and updates in education that we are looking for that we want that we need public schools don't seem to be able to incorporate those things because of so many resources being pushed towards standardized testing because of schools being held accountable for standardized tests and the outcomes of standardized tests and he said well you kind of have to write about the gorilla in the room and not just write about you know the nice party that's going on so I said okay this is what the book's about it's about standardized testing and I didn't know much about it at the time but through my research what I really came to realize was that there was a really really deep you know the issue at the heart of this book is sort of America's core moral issue which is we want to have a broadly open and competitive society with fair level playing ground where people of various backgrounds can come to compete and to succeed based on their merit we are all in for meritocracy we're all in for competition but we want that competition to be fair and that's kind of America's unique contribution in order for there to be a fair competition though you really need to have fair marking point posts you need to be really really correct about the information that you're using to judge people against each other and when I came to kind of see is that standardized tests are not that mark they've never been that mark the more we try to make them an instrument of increasing equality they're more they're going to fail us and the more they're going to fail the kids that really need our most of our help and most of our support and you know that your book is pretty comprehensive in looking at the full dimension of critiques that have been leveled against standardized tests and different ways that they can in our can be in our problematic can you distinguish among some of those major dimensions because I think yeah you know partly you talk about the limitations of the test themselves as instruments of revealing knowledge but then a quite a bit is about the use of them their implementation for school accountability for teacher evaluation student promotion etc etc and those seem like important distinctions yeah that's a really important distinction thank you so so first of we're going to talk about you know standardized test themselves we're going to say that they're they're in a tradition of psychometrics and intelligence testing which dates back to the 19th century which is based on some pretty foundational social science techniques but also has always been involved in it with an ideology that is a eugenic a eugenicist ideology the people that came up with psychometrics were by and large eugenicists and so there's limitations in the conception of intelligence as a fixed unitary hereditary quality and even when we were migrated over into what is called achievement tests which are attempting to measure up not aptitude but you know not math and reading knowledge at a particular point in time the way that the tests are designed and normed and referenced still bears this lineage that comes from the intelligence tests that came before them so the tests as instruments have limitations they are technical limitations of the tests because they are created by companies you know they have they are fallible there are mistakes in the tests often you know that people sort of criticize individual items on the tests and they're always open to debate we don't have transparency about the items on the test truth and testing laws and that debate goes back to the 1970s and Ralph Nader talking about we can't really examine test items because there are companies proprietary information and so there isn't a robust public debate about the value of particular test items so that's the tests as a as an instrument the tests when you attach high stake to them become a totally different animal because while you can argue reasonable people can argue about the value of a particular math or sight reading score let's say for a teacher doing diagnostics or for a school that wants to beef up its program in one subject or another once you make and punish mostly punish there are some carrots in the no child left behind law but mostly they're sticks when you punish schools with punished teachers and districts and states for the results of these tests when you attach so many different outcomes to the results of a specific individual test that's when you really start to have problems and that's what that's what we're kind of seeing here right now is a kind of popular revolt against the idea of high stakes testing. Are you one of the responses that people bring up around the critiques of the quality of tests is well that's true but we're now in the process of moving toward a new set of exams aligned with the Common Core that are going to be put in place for the first time at the end of this school year so right upon us the product of three hundred million dollars in federal investment right you say this is going to make things worse in the book why is that I mean one of the ironies of the last few years of federal policy is while you know the the Obama administration has been pushing for attaching more consequences to the tests specifically in terms of teacher accountability ratings being tied to test outcomes at the same time they've been saying that current tests are not very good and funding this enormous effort for the Common Core and Common Core aligned testing as you say you know I talked to some of the folks working on making those tests I talked to people that have independently reviewed them and what they've basically told me is that there's not a really huge departure in terms of a kind of test with the new Common Core tests they're more difficult there's going to be a major score drop in the next few years and the score drop is why I say that this causes more problems because more failing schools more failing students more panic if you see that the recent case of York Pennsylvania the entire school district being handed over to charters on the basis of test scores so the more you have dropping when you have dropping test scores you have a reason motivation for all kinds of actions why I say that you know people who independently reviewed them basically say look when you have a test that costs less than $30 a student to administer it's not going to test higher order thinking it's not going to have a lot of writing on it it's not going to have a lot of kids writing out equations it's not going to be graded by a credentialed expert you just can't you can't buy that for that kind of money and so without massively changing the format of the test or some huge leap forward in technology that isn't there yet you can't test higher order thinking with it for the test that costs that much Maurice would love to hear your thoughts about the book and also your kind of broader perspective on standardized testing as an expert on early education among other things sure I mean I think that I read the book it's a great book it does bring up you know there's I was recently asked to sit on a panel and the young lady who called me said that we like you to sit on this panel because you've been in the early childhood space for quite some time and I said to her you mean I'm old right and so I've been in this space for quite some time this psychometric space and you know and even the chronology when you write in the book I mean this really for those who've been around for a while Jerry this really started with Sputnik you know it really did start with Sputnik and where we're not first in space and so this has been an historical then we went to minimal competency and we had the Texarkana scandal because the teachers cheated this is in the 70s and so we're always trying to get better and I guess I'm wondering and I have lived in the early childhood space but I was definitely superintendent for the DC public school before so I know this pre-K to 12 arena but what is it we're trying to do what do you think I mean what in the hell are we trying to do I mean what do you think I mean what is the purpose of all this I mean I read about children getting tummy aches and crying and everything if you've taught children they always have tummy aches and crying they don't want to go outside they don't want to do the math lesson so that that was not here's one theory is that what we're trying to do with education and reform is to square a circle we want to eradicate poverty without paying for it and so somehow if we invest in education enough to increase opportunity I mean if you read the book no excuses right Heritage Foundation 2001 right yeah so I mean I called that guy up and I said okay you wrote this book that said there's there's seven high-performing schools and these in high-poverty areas that are succeeding and if every school what does what these schools do every school with high-poverty students will succeed the outlier will become the norm and that is a statistical imaginary idea the outlier doesn't become the norm because the outlier in the norm are not the same so you know the whole no excuses movement from from then until now has been based on data you know they're taking data as their basis for how they're making decisions how they're kind of moving toward but they're not necessarily I don't I haven't seen yet a compelling explanation for how that actually happens so you know in terms of so some of this comes out of our vision of the child you know how we see children and their development and so there is an assumption in this obsession yeah which I'm telling you this obsession one of the reasons I don't worry about this kind of stuff because the pendulum swings back and forth when the test scores go down on the park and the other there we're going to find you know we were going to be the 2000 we were going to be the best in the world you know just like Lake will be gone and then we were to leave no child behind it every time we leverage it around high-stake testing it fails and one of my big concerns well there's two dimensions is so one I mean what do you think the vision of children is based on this obsession with testing what what do you what do you think people psychometric you know the government and policy what do they think what they know about children first of all well I mean one surprising reality that I learned was that assessment as a discipline doesn't live in the same realms as cognitive psychology developmental psychology or little learning scientists sciences they're not they they can tell you what you know and what you don't know based on their construct of the test and what they built they can't tell you how to learn what you don't know or or indeed how likely you are to be able to learn what you don't know right no you're absolutely right and so if we believe that all children can learn and develop at the same level and that there are periods in their development when we can check in and they are just wonderful you know to me suggest that we don't understand child development we don't understand human growth and development we and our vision of children is that we can assess them and move in sort of sort of like an assembly line mentality in terms of looking at children's growth so one of the things I would offer you know is to consider multiple ways of measuring intelligence I mean Howard Gardner or any other people who are doing neuroscientists and then the other thing that didn't come through in here which was not your purpose in writing the book but it's something that is of great concern to me and that is the achievement gap you know that when when you create high-stake testing and you do not make allowances for children from under-resourced communities I mean the assumption about testing is that everyone has the same thing I do feel like I talk about the achievement gap and what I kind of came to understand about the achievement gap in the book was that I I thought I mean I call it a tautology masquerading as a problem you know when a study shows that 85% of the variation in test scores of a group of students in North Carolina 85% is predicted by their family income why do you even need to give the test if you if it is so closely correlated and it has like the SAT is also extremely closely correlated correlated with family background and income why do you even need to give an achievement test to define what's happening to these children we already know what the issue is and once you level if you manage to level that resource issue and you're still dealing with natural variations and inclination and interest and that is a job of the educator but I would argue that that the achievement gap is a job that's being left at the door of the educators so what would you what would you think about the national movement to eradicate the achievement gap I think as long as we call it the achievement gap it's it's not necessarily going to help we call the opportunity gap there's a gap that is real and persistent I mean to kind of get back to your question about what are people thinking you know why is this happening I mean there's a line that I think it's actually it's a really good line in the book but can be taken a couple of different ways where you talk about and I'm not going to quote it from memory as well as it's pretty good to somebody the scent that as things become more and more standardized and they are becoming more standardized we're going from 50 different you know we're going from thousands of standards to 50 standards to two or three sets of standards now that's been the course of the last 30 years and thus the tests are standardizing along with them and you say just you know inevitably this has the effect of eradicating difference right I think that's the line but but I you know I think that the the policymakers mindset is that it's to eradicate the difference of ignorance that in other words that you have that you have vast differences in outcomes among lots and lots of people and that those outcomes are mostly a function of a combination of pervasive societal inequality that manifests both in what happens to students in school but also to what out of school but also to what happens to them in school that you can and you talk about this resource disparities teacher disparities lots of disparities and so that the foundation for some kind of pro education equality agenda is knowledge of the nature of the of the inequality and that that should be expressed in terms of learning and so you find the best measurements available to understand that inequality and then you you move from there so so is I mean putting the the limitations of the measurements aside for a second because your book actually in the second half I read is quite optimistic about how we can know more right is that an invalid sort of public policy theory to sort of start with the information and then and then create policy around it well I mean it depends on political what what your political background is and what you're most afraid of right you most afraid of government intervention to schools are you most afraid of what schools will do or localities will do when left totally on their own is there a distinction between government and schools and so public schools I think that the debate around the common core that's playing out suggests that families perceive a very big difference between the jurisdiction of their local public schools over their child and the federal government's jurisdiction over their child's education whether that's correct or not this is a very locally controlled and diverse school system right in terms of standardization I mean to me to my the standards the standards are not that meaningful without the tests and the accountability framework that underlies them you know the standards could be stand on their own to be taken or left without the necessity that schools test to the standards and punish for failing the standards and to the extent that we're going to hold schools accountable these specific learnings these specific types of learning and you know not a curriculum but a set of types of learning we better be pretty sure that that is exactly and only what students need to know and I haven't seen the data that would suggest that we didn't validate the common core right we didn't test the common core before it was rolled out we didn't pilot it and this is in point out by many many critics so whether I believe in theory in the unitary set of standards is kind of secondary to the actual standards that we have. So you the book is kind of doesn't feature a lot of robust defenders of standardized testing not because I don't think you were looking for them because in the contemporary dialogue I don't think they were defined. You know I actually got a chance to talk to Sandy Kress after the book ended unfortunately he said that he would take take it to me this maker the the slight increase in achievement and the slight narrowing gap associated for African Americans which has been seen over the last 12 years so that should be on the record no that's certainly when you come to the end of that line I think you'll find Sandy standing there you know so but you know at the same time and you talk about how in different districts there are like a lot of examples of parents parents turning starting to opt out and probably it's probably worth spending a little time on the opt out movements at some point but nonetheless like day to day every state in America and every major district either and the national government is sort of objectively pro-standardized testing they're all doing it there's no you can't find a district of any size that says no we don't want to do it so why is that do you think I mean is it is it mass delusion is it lack of something better like what is going on how many people in this room remember what they got in their SATs okay and has that information has that information useful to you other than the guy who teaches it so numbers are super powerful numbers that say something about our own ability are unbelievably powerful and to the extent that we want every parent wants that information about their kid and every parent wants that information about their kid's school every parent wants that information about their kid's teacher every school wants that information about their kids and about their and all down the line our thirst for metrics and data is not and that's one of the reasons like the arguments in this book don't satisfy some of the most extreme opt outers because who want to get rid of everything except for teacher-created assessments because I acknowledge the point that we've gotten to this place because everybody wants data and information and the only question I'm asking is you know your metrics are only as good as your underlying data and your analytics are only as good as your underlying data so let's really peel back the curtain and see what we really have here not to deny the power of that because I think it's very powerful but you know um think and part of sort of the historical trail journey we've been on is the resistance to a national curriculum and a national assessment but the truth of the matter is that's where we that's where we are not even where we're going and I think that so there is a body that wants that and again I go back to it's a body who does not understand human development and so I just wonder if if in your research and thinking about this book and of course people who opt out and you know opting out is an issue but people leave when they don't feel that they're getting what they want because the truth of the matter is in some instances in some communities opting out for children from under resource communities has made a difference for them and their parents for the first time believe they have a choice but I just wonder if there is any talk as you talk you know interacted with people that people are just saying that we have to have a different way of looking at this absolutely there is I mean I see leaders all across the country from Joshua star and Montgomery County to the core group of school districts in California you know various charter school network leaders that perform at standards consortium all across New England and New York in New York as well Julian Vasquez-Highleg and Linda Donaham and their multiple measures community accountability approaches the idea of bringing in I mean at its very baseline let's remove the single point the thread from which we hang all accountability and add other measures into our accountability determinations and not make any decisions based on one test that fundamental idea I feel like is gained a lot of traction there are multiple forms of assessment and there's not just one multiple forms of assessment multiple approaches to assessment multiple factors other than just assessments in accountability so all of these things and I think you know the opening up of public pre-k is and the unfolding of pre-k into the public school system I think is a really interesting opportunity for this because I mean on one hand people say oh we're going to have kindergarten assessments and we're going to have four-year-old assessments and your standardized tests will reach into the cradle that is one that's one but on but there's also a side of well if you want to do accountability for a pre-k everybody knows that pre-k students have social emotional needs and a good pre-k teacher is one who addresses those needs it doesn't matter if all the kids know their letters if they're you know hitting and biting each other and peeing on the ground right so how do we assess schools right but I mean you know so how do we assess schools and hold them accountable within a public framework without just testing the kids because clearly that's not enough in pre-k but the way has we now states have kindergarten entrance assessments that you know and so there's also pre-k assessments and so that this pushing down and this obsession your words this obsession with testing because people don't know about three and four years and one day they know something next day they don't know something but so like you can't really assess young children with any reliability but this is I mean the big concern in the field is the pushing down of the well there's a curriculum but there's also this testing right that and I don't I don't see this ground I mean people complain but I want action want people to do something about it I understand that definitely so the the second part of your book the first part of the book is again this you know very comprehensive critique of standardized testing as we know it now and and how it's used the second part of your book goes into a lot of detail and this is I think this was kind of the biggest learning experience for me probably because I've kind of like lived in the first half of those debates like for a while yeah about a lot of possibilities for how assessment I won't even use the word testing because I think that's too narrow but how assessment measurement could be better and you you talk about robots monkeys butterflies and unicorns but tell us what you mean right so um right so in order for assessments to get better first of all people have to be motivated for them to get better and it varies points along for example and in the big test um Nick layman's great book so sorry um in his great book about the history of the SAT you know they experimented with all kinds of different tests including Myers-Briggs version maybe even some Rorschach blots but in the end it was all abandoned because it was too expensive um so the question is how could we have these kind of better tests knowing that what we know now about what makes people successful um and the the robot approach is you know what I called in the my recent ampere post self assessment and this basic idea is you know businesses you to shut down at the end of the year for three weeks just to take stock manually right with a price gun and that is what schools do they shut down for eight partial testing days right at the end of term to take stock what if we could do a just in time accounting of what students know on a daily basis and what we'd also learn from that um is not just the math and reading information one score in one day but we would learn a pattern of how students approach difficult problems how they approach easy problems do they give up easily um you know are they diligent to try and stuff before they get it so within the data and this is when you get into really you know people that are interested in data are human people too and they're interested in the human sides of data more so than what a single score can tell you so um I mean I'm interested in that possibility that's to bracket that out that is a big business possibility that something in Pearson's really excited about ETS is really excited about so if you're nervous about the potential of for-profit companies controlling assessment you should be nervous about that too but that that is one thing um and I really do think it's different I think it's different from a static math or reading score I think it gets rid of test anxiety I think it um I mean potentially could uh you know uh change how we think about I mean all these they there's a lot of talk these are about growth measures which is basically comparing the score in one test to the score in another test but tiny summing up every single day can do that in a different way um the the monkey guys right this is the idea of using low stakes data gathering in the form of surveys which is what we have to talk about social emotional and and and not just I mean social emotional skills is really the wrong way of talking about it we can talk about it social and emotional climate so we can talk about um the behavioral records in the school the discipline numbers we can talk about um you know for example there's a really awesome study about teachers reports of eighth graders behavior problems uh predicted their high school graduation rates really robustly so if you ask someone's seventh grade teacher or eighth grade teacher if they get in fights you know if they're going to graduate high school so gathering that kind of information summing it up with these surveys which groups like Gallup are making available that are and in California then this is a healthy kids survey which is pretty big and asking kids really simple questions like I feel excited to come to school every day I am the best friend at school these these data points when summed up again you can't attach high stakes to that because you can't attach high stakes to a kid saying they're happy right um because the teacher stands over them and says you are happy right down um but as an information gathering purpose and as a basis for decision making on a school level this is promising um the the butterfly people are the um kind of core of people that say that each child is an individual um and testing and learning should be one in the same that we should assess through authentic tasks that are performance-based portfolio-based presentation-based research projects group projects assess the kind of skills that students really need in the workplace have them you know be able to speak publicly and to talk about their work and to combine disciplines as we all do in our lives a 12 year old kid in Kentucky I talked to you know I'd done one of these he said let's get real no one's going to be bubbling answers at their jobs um you know in this school I know that what I'm doing really matters it's a beautiful idea the schools that are organized this way seem like the ones that I visited seem like really great schools the students seem really great um it is very hard to codify this stuff um Kentucky in fact included portfolios in their state accountability ratio uh uh system 20 years ago it was dropped principles and teachers can pay like complain that was too difficult the idea of imposing this on their national basis um seems kind of tough but it's still a really promising approach finally unicorn unicorn I say because it may be mythical and this is idea could be somehow combined a technologically approach that is very data intensive as that with an approach that has kids performing authentic tasks um like a performance-based approach um and so these game-based assessments that I talked about glass lab where basically the kid is playing the game the game presents them with opportunities to make decisions or to critique critique their own work or to model a system and the game is sort of using um you know complicated analytics to make extrapolations and gather evidence about the kid's thinking process and so what you get at the end is not like a score per se but what you get is a profile of a kid that says this kid has a second-order understanding of systems thinking and of using evidence to make an argument so I was intrigued with your unicorn people yeah right they're fascinating and I was curious have you seen that both and any place around you know as you did your research where there are places where people were pursuing that well yeah I mean that's the glass lab and the the AAA lab I mean these are tiny tiny tiny experiments um glass lab is available they're making another game available um which is teaching kids rhetoric and argument structure um so but then there's also I mean within the broader realm of serious gaming not so much thinking about assessments but thinking about what does game play teach kids and what can we learn about kids having them play games um I think you know that's that the interest in that area is only going to grow right I mean they're using it in teacher preparation programs right yeah well simulations have I mean simulations have been around for a really long time right and they are not speculative they're proven in terms of learning a thing by practicing it in a low stakes way so at the near the end of your book you have a pardon me um a little scene where your daughter Lulu who is how old now Lulu she's three three and so in your neighborhood in Brooklyn um there's a for-profit chain of essentially drill and kill test prep for three and four year olds yeah so like as the two year olds two year olds so as the parent of a four year old I was just terrified yeah but that that even existed and revising my like aspirations to live the Brooklyn Yuppie lifestyle yeah so I wouldn't do anything or stuff like that um but so but you know you kind of also acknowledge that um even as a you know you as a parent or anybody feels a kind of almost a visceral reaction to that there's also a strong desire to know and um both um inherently and also because you understand that the number will matter some somewhere sometime you know whether it's and probably not until probably before they turn 17 and have to take the SAT or whatever we use then so and so uh what should the parents in the audience or watching online or whoever like how should they think about this what should they do the last chapter of the book is a step-by-step approach like I take it as um here are things you can do if you want to change a testing landscape here that here's how we topped out if you decide to opt out most parents of children of public schools are not going to opt them out of every single test especially the tests that have high stakes for that student like to get them into a competitive middle school to get them into a competitive high school or a gifted kindergarten um most parents I know if they're committed to public school are going to try to get their kids to do well in those tests and so what I advocate what what sort of neatly you know I feel like the more you understand about how these tests work a the less afraid of them you are and b the better you're going to do with them because really the thing the limitation of standardized tests is that they all test the same thing they all test your analytic skills your ability to eliminate choices and make a guess and stay cool in different situations your ability to think structurally about a test where am I in this section what kind of test is this does this test have one-third easy questions one-third medium and one-third hard or does it have 50 percent hard and 25 percent medium 25 percent easy that's going to make a difference on what I guess on that on this particular problem so the limitation of these tests is also their strength if you are a guy like Balabalaxander the testing wizard in Florida who can get NCAA you know people to meet their NCAA requirements even if they never passed a class because he teaches them how to think strategically about how the tests work and I think you know I don't see any harm in that as long as you are simultaneously working in other ways potentially to you know if you want to help whatever it is that you believe is right the right thing to do I think that having more kids understand how tests work particularly kids who have other disadvantages as long as the system is there to be game do we might as well game in people's favor and so think a little bit about kind of how these two sets of issues how we use tests and what the best tests can be might converge in the future so what fascinates me about this conversation is that I do see that for people who are open-minded that science is leading them back gently back toward a more holistic concept of what a human being is and what human development is and what I you know what I mean by that is you know if you take like Paul Tufts book right Paul Tufts book is about the most hardcore data-driven people who thought only test scores mattered looked at the long-term data which was outcomes for their kids and found it's not only test scores that matter there's something else that's missing here and we don't know what that is now when you rush into if you rush in to slap another test on that maybe you're really missing the point but the idea that science can lead us back toward a consideration that you know zero to three is really really important and that means that oh parents are really really important we must be working with parents and making home visits and figuring out what that is supposed to be brain science tells us that we need to feed kids so like the more you if you really look at what science is telling us and don't rely on old science I think you can get a better vision a better test and also I mean a better better outcomes potentially so I mean do you think that will embolden the next generation sandy crust to say well okay we've you know we've now the numbers are good enough and so full speed ahead with trying to implement some kind of robust policies where this stream of information is acted upon you know governmentally administratively et cetera you know teacher evaluations you name it well modern science is not positivistic and and good scientists keep an open mind to the possibility that they might be wrong about things and I mean this good I mean in the first chapter I lay out the issue with what we have now that we call a data-driven system which is that we attach when you attach an outcome to a measure it's no longer a good measure so I think part of the solution may lie in separating out the testing and the consequences or at least combining them in a different way I'm not sure exactly what that's going to be but we there's a clear structural problem with tying tests to consequences in the way that we're doing it now don't you also I mean I was interested in your pursuing sort of the non-cognitive and so there is this growing body of consideration of non-cognitive and you know a grit tenacity and perseverance and in Paul's book as well and so I mean I'm almost thinking that that is going to be the fallback position for people once the testing doesn't you know I think that people are going to raise it tonight and I I welcome it right I think people are going to raise that as well that really wasn't important because again historically every time we've gotten ourselves in a jam we moved on to something else right I mean well and the other thing I just want to mention is that you also did think about schools as ecosystems and teachers are very very important in this conversation and teachers objections to standardized testing I think are really important to listen to because a fundamental objection that I hear is that a professional is not a person who has an external that is that is um that is substantially committed to an external review on their performance a professional is a person who is open to feedback from their peers from their supervisors and from the people that they serve but there isn't an external person coming in to tell me whether I'm a good npr journalist based on a standardized instrument and so teachers who say that they feel demoralized or say that their moralism is really low because fundamentally they've been removed from the process of deciding whether or not they're doing a good job and that doesn't make for a great environment for for kids because you know if if somebody's autonomy is in root there how are they going to teach a kid about autonomy or how are they going to teach about anything that they need to know or how are they going to be pleasant when they teach um thinking a little more about the policy dimensions about this there's a section in the book around this idea of holding states and districts accountable for resources and you talk there's you relate a conversation you had with a a weary policy think-tank person at a Gates Foundation funded cocktail party party I think that was me am I right about that yes so for the record I was mostly weary because my flight had been canceled and I just had to get off the phone with my wife saying I'm not going to be home for 48 hours so you have to take care of our three-year-old for 24 days so but I mean it's it's a point taken and I think you know one of the kind of meta-critiques of the testing regime and the idea of holding mostly schools so we don't even really hold districts accountable for test scores we mostly hold because they're there you can't really move them we mostly hold schools accountable is that in the case of New Orleans uh special circumstances but yeah um I think that's right uh it you know is that uh you know any rational you know concept of how the system works is a combination of inputs and processes um and that if you just ignore the inputs that it's going to break because you're essentially holding people accountable for things that they're not responsible and you know even I mean some of the maybe more extreme critiques which I don't really share but I understand you know stand them is that this is actually a way to ignore the resource issue but I think you know the flip side is and I think this was my response at the time was you know you're really talking about like the nature of American government federalism and so we have education as as this core element of our of our society and our obligations to our citizens and our students that is more so than anything of similar size decentralized and de-federalized and always has been and so the you know the testing agenda has been in lockstep with an agenda toward more federalization and centralization of education and most you know most of those steps have been about equity right you know I mean most of the I mean you know in the middle of the 20th century that federal governments abiding concern around education was providing racial justice and very much renegotiating the power relationships between itself and the states and there was all kinds of resistance to that and so so you know I think the you know a protesting person might say look we need evidence to sort of show how bad things are if we're going to mount a political case for the kind of the kind of resource equity that doesn't exist right now so somebody could certainly say that I mean I think it's also the case that after you know in the wake of 13 years of hardcore accountability test base accountability our schools are more segregated and that it's also the case that charter schools which are in many I mean the concept of school choice is centered around this idea of we're going to test all the kids report out the test scores and then parents will have a basis on which to make a more informed decision about their children's education the charter school movement is also charter schools are also more concentrated racially and they're also more concentrated in terms of poverty so if tests go in lock step with an equity agenda I guess the question that I would ask is what are they doing for that equity agenda are they really are they improving it so I think so I think there it's probably there's a bunch of different ways to answer that I think I mean and you can interpret the national education data in different ways I mean the the achievement gap has not been closing over the last 15 years but it has not been closing in an environment of overall slight but real improvement right so so I mean it's it's not been closing not because privileged people are doing better and non-privileged people are doing worse they're actually both doing better but the distance between them is not shrinking you can think about that in different ways I mean I mean it's not and that happened during a time when overall American equality got substantially worse right so it wasn't as if the underlying equality trends were kept constant and then we really focused on closing the educational part of it they were getting worse and I mean I think anyone would recognize those two things are are interrelated I mean the purpose of Brown versus one of the board of education wasn't to close something called the achievement gap it was to desegregate schools and so right but I mean but separate it would equal was seen as profoundly unfair from an educational standpoint right so yes it was um well I mean so but I'm saying if if the point of that if the point of that equity movement was to desegregate schools not just because it's nice to have schools that are racially balanced but because it is an it's an unequal educational environment if they're not the point that I'm making is if we don't desegregate schools if we actually are resegregating schools then how are we achieving equity agenda even if everybody's doing a little bit better I mean we haven't resegregated schools schools have become resegregated because the legal environment and political environment doesn't really there are limitations to how it permits us to move around where you go to school so so you know again I think it's kind of well and why are people more segregated in their neighborhoods you don't know let's start with redlining and no I mean well I mean I'll give I'll give one potential explanation one potential explanation is that um there is a polarization of real estate values there's a real estate intensification right really high um uh mortgage really really high real estate prices part of that driven according to Elizabeth Warren in her book the two-income trap which came out way before the mortgage crisis um families looking for houses in good school districts huge huge differences between high test score areas and lower score areas issues so the bulk of the middle class in my opinion was developed with the gi bill because people could go to school and they could buy property there was a practice of redlining which meant you could only buy in certain areas if you were a person of color real estate increases and that's how you build wealth and so if I force you to live in a low valued real estate environment you're never going to build the wealth to move up you will so I mean that that is really the history of why people are in the neighborhoods they're in and now I mean it's off the chart I mean these are complicated questions I was so like a month or so ago I was interviewing Josh Starr who you mentioned from Montgomery County who's been a very a national testing critic but also not an absolutist by any means I mean I put it right to him I said if you were in charge how many tests would there be and his answer was fewer but not none you know there would still be benchmarks at different points I mean he was an accountability person but the other thing we ended up talking about and we will get back to testing in a moment but I think this is interesting Montgomery County is extraordinarily diverse school district one of the most sort of unusual school districts in America in a lot of ways and becoming more divided mostly because of the underlying economic and demographic trends and I said you know they don't really have a school choice environment in Montgomery County and I asked him should there be one and he said look some people some people absolutely want the ability to send their kids to different parts of the county some people really just want good schools in the communities they have and aren't really interested in that and and that's you know that's been kind of the push and pull of desegregation and and a lot of these things but we are here to talk about tests so a couple of things I want to give you one more chance to talk about anything in the book that we haven't had a chance to talk about yes well then go to questions from the audience we also have people watching online because this is 2015 just tweet it on you directly and cut me and the whole middle man out and you can just talk to her and she'll talk back to you I well I guess one I mean actually I just want to hear from the Q&A I want to hear okay let's go right to questions from the audience um down up front hi uh my name is Dave Price 30 years in poor urban areas as a teacher for as a consultant based in DC going around uh an observation I'll get to my question as quickly as I can Anna uh not only is your book valuable it's uh brilliant and the reason I say that it agrees with everything I think so that's why and I have that on 99.9746321% so we have it there I just want to speak for a second as a grandfather make the matter more real you have Lulu we have Audrey uh Audrey entered first grade this year in Dunwoody a very wealthy section of Atlanta and in her first two and a half weeks had four days of standardized testing wow that's America today now my question is this and you've really done excellently there's one place that I have a concern and again personalize it my college roommate is was high ranking in McGraw Hill but that wasn't good enough he moved to Pearson and I think he may be the antichrist I'm not sure but that's that's something else uh the point is there's so much money involved in this okay as you did your research as you go around um and again not not just to preface I know because I follow his career uh we give the test the kids don't do well so we give the prep but we don't give the prep real well because we realize there's not not enough time so then we design the school books and so we really have total control once I bill gates the other we're going to make the tests on computers great bill who makes the computers so how much do you think and a if it is involved what can we do about that the other things we can do thank you sure thank you so much um you know this is America and capitalism is very important uh no um but uh I guess what I'm trying to say is I mean I I looked at that angle and I looked at the business of testing and there's a lot to be said about the business of testing the profitability of testing and as you mentioned um and I mean many public statements make clear from Bill Gates Arnie Duncan and others that the common core from a certain set of stakeholders point of view represents the consolidation of a marketplace of tests materials assessments and machines to to put them on um you know that's just how they look at it because they're you know when you have a platform these things are interoperable and therefore sell to one sell to all um you know this is a business opportunity I think you have to remember I mean Pearson's the largest English language publisher but publishing is not a great business to be in um and so as much as they are the 500 pound gorilla in terms of education they're not such a scary enormous or thriving company even from the point of view of American corporations the involvement of the technology industry in education um is is a different story because it's a much larger and more powerful sector um and um and these interests are important I mean I think looking I think a case to watch is the one is the one of the iPads in Los Angeles um and the FBI investigation around that because it appears that although I mean the deal is an iPad deal it seems to be that it's Pearson that's under investigation not yet hasn't yet been announced exactly what the purpose of the investigation is but KPCC released these emails between Pearson and the office of the DOE LAUSD uh talking about the contract and or we have to make sure to get you guys in and get this bid in um and that kind of stuff so um yeah I mean there's definitely a major um consumer watchdog role to take uh but to me I guess my bias is that the politics comes first um yes ma'am in the back and I'll let you decide what to take Twitter yeah I'm just seeing what's the comments and what's the question my name's Roberta Stanley and I work for the state of Michigan Department of Education for a number of years and on the hill um my observation is that the governor is being more involved in education after they solved welfare reform in 1994 they were looking for simplistic ways and the businesses could they could access governors more readily than state departments of ed because of the bidding process would you comment on the governor's involvement in their their tendency for reformitis and wanting quick fixes um I mean I mean I can't comment too specifically about that but I guess I make the observation in the book that um test scores are different from other kinds of data let's say jobs numbers or environmental numbers because they are so much more open to adjustment and if you look at the way that for example um Mayor Bloomberg in New York City ran on test scores that were then readjusted and fell precipitously um between one term and the next um you know there's not many other numbers that you can do that with um and so I you do see that test scores of type of numbers that are embraced tend to be embraced by political leaders either as a banner of what's working or as a banner of what's not working yeah in the back thanks just following on from that um something that I'm aware of is the use of of tests that may intentionally kind of target a collection of schools so when we think about you know what is the use of tests what is the functional use of tests the political use of tests is obviously very important so thinking about um you know why is it so difficult to achieve you know better controls on tests you know why aren't we looking at at um income as much and then you say well who's advantaging from the current kind of focus on scores bereft of those kinds of controls I mean isn't that important and could you rephrase the question okay um if a primary use of the tests is political change possibly increased charter schools privatization some of these things where you know that there are you know private parties that are benefiting um you know is that is that something we should be focusing on more as a a problem because you know if we're not really looking at whether tests support educational change within schools that really mostly support a change of administration to a private entity you know what does that mean I mean I guess what I can say about that is where you see parents organizing and educators organizing against standardized tests they definitely see it as part of a broader agenda which is an anti privatization anti what they call market-based reform in pro-local control and pro-increased resources for schools and often per teachers union anti charter schools so definitely people see these issues as all being connected and it's not just about tests my name is Andrew refrain and I'm a professional tutor with prep matters as I indicated before academically and for tests my background is in science but first I want to say thank you for mentioning like if you follow the science then you actually get to perhaps more worthy reasons for your your actions my first point is on the test in general I think we can agree that the purpose of the test is two-fold to both reliably indicate a measurement of the student performance and then also hopefully to accurately measure that performance and predict their their future outcome now reliability I think we all agree also that the SAT and ETS do that very very well even if it's a bit forced with their use of the curve and and how they shift the test we're now coming up on the the next of many changes in the SAT whereas the ACT kind of just hung out but then the accuracy of the of the test I think you were you were mentioning how testing tricks and test prep and everything I think that's a valid point to say are we really actually aren't we calling to question by the very nature of there are tricks to the psychology of testing that we can teach students aren't we calling into question the validity of the test and its ability to do something accurately so any comments on that about how we can actually address the question is this even the way we should be going about doing this and then my second point if you had time I also appreciated Mr. Carey's comment regarding using the test data as grounds for political argument and policy reform I think there we may want to look beyond just the test itself and ask why are we testing what's the point of education is it in the end basically trying to get a population to be economically prosperous and beyond that happy some nations the world actually go by a happiness index rather than the GDP and if that's the case should we not be tying our education and therefore our testing and benchmarks to the shifting economy that we now see beyond just the technology use for testing tools but the fact that why is CS computer science not a major component of every education at this point yeah no no there's a really interesting point I mean I think when it comes to college admissions testing on the one hand you have about 800 colleges opting out of using standardized tests nice for admissions which is a really interesting phenomenon because we I mean the report is that most elite colleges are relying that almost exclusively as a first pass way of eliminating people and the questions about the fairness are just as you say it also your question also gives me a chance to mention the research of Robert Sternberg which I was really fascinated to learn about and he's had a series of projects over the years the rainbow project was funded by the college board and his idea was can you he sort of segments intelligence into analytic intelligence practical intelligence and creative intelligence he later added a fourth leg on the stool which was wisdom and that's sort of an integration of kind of empathy and long-term thinking a really beautiful sort of theory of intelligence and he but that what's really interesting about it he created instruments to test each of these components he managed to use these instruments on very large numbers of students first in the in the rainbow program and then at Tufts University in the admissions office where he was able to almost eliminate ethnic group differences in consideration because of the tendency of different ethnic groups to have different strengths distributed among those three areas and the practical and creative intelligence items that he created he first tried to create a machine scoreable version of the test and he said no matter what you do in a machine scoreable test you're only testing analytic intelligence but we had later versions of the test which required sort of a structured rubric grading process where in the creative one an item would be like come up with a caption for this cartoon a practical intelligence one would be you know you're having a problem with a with a classmate what do you say to your teacher you know solve a real-world problem using real-world skills and you know these instruments have been in use in various places he's now developing a grad student admission instrument with the with the right resources and even ETS is working on instruments that consider more factors and therefore give you perhaps a broader picture of what's going to make someone successful. Hi I haven't read your book yet I'm sorry I'm Louie McNabb with the National Council of Teachers of English but do you address the impact on the arts and athletics and how they have been reduced in so many schools because of the standardized testing? I do yeah I talk a lot about the curriculum and a later report that I was able to do for NPR talked about specifically in high schools where the graduation test requirements have been implemented which has been done in more than half of the states because with those graduation it's really important for the kids to get a diploma so what they end up doing is there's so many retakes of the tests they just get administered again and again and again and in some districts it's like 40% of a specific month of the year and when some kids are redoing the retakes other kids have to just watch movies because the kids that already passed the test if they keep studying math they're going to be way too far ahead of the kids that haven't passed the test so I mean I have you know I talked to some people in Florida people in Las Vegas who said it says really our school doesn't run it doesn't run in April it doesn't run in February whatever the months are where they're giving these high school exit exams. You do mention some of the European countries to whom we're compared unfavorably from a testing mania standpoint do you have an enormous one enormously high stakes end of school test and I wasn't quite sure reading it whether you thought that that would be something to go toward or run away from or well my colleague Croy Turner talked about this in the great MPR piece last fall where yeah like just one test with really high stakes is that really a better system but I think you know it's part of a totally different philosophy and that's what we have to like look at it as because these tests aren't being used to punish schools or teachers they are definitely very high stakes and obviously in many of these countries in Europe they're much more comfortable with a tracking approach to deciding children's futures based on you know a few weeks but you know on the Asian on the Asian system there are definitely the centers talking about the stress that these tests cause and how they're really trying to ration out a scarce commodity of university education and is that really the best way to do it but what I mean what I'm intrigued by in the UK in particular is they just don't see the point of giving cheap multiple choice tests in any way because they think that if you give high quality and of course tests the teachers have to teach for long-term retention and they have to teach deep understanding of a subject because you're going to have to demonstrate it after nine months of study with one kind of summative formative summative assessment and so as a pedagogical from a pedagogical standpoint it doesn't seem to me to be a terrible theory and then the other part of the picture of accountability in UK schools is school inspections which I find really fascinating as a tool and simple I mean the idea at least is simple right I always have this sense of the like Mary Poppin shows up at your school yes exactly you know white glove exactly yeah um sorry questions yes yeah right there hi I'm sore wonder I work at character.org you mentioned how many teachers are becoming demoralized and I'm curious what advice you have for teachers who see no end in sight with standardized testing and can't really opt out without maybe leaving the profession um I my hat is off to you it's a very difficult job I think you know I see teachers who do manage to be creative and and work around the requirements just as people do in all kinds of constraints situations and then you see teachers like those in Chicago and those in Seattle who decide that this is a workplace issue and a union issue and that they should refuse to administer the tests so there is a range of responses. Thanks Kevin my name is Tom talk I'm with the Carnegie Foundation. Kevin has pushed gently I think a couple of times against your seeming concern about accountability I don't want to say opposition but maybe it is um but it's also the case as Kevin has as implied that in many public schools black and brown students are treated differently the expectations that educators have for them are lower they indeed have fewer resources and the like but um the notion behind accountability test-based or otherwise and I think many people would agree with your assessment of the limitations of standardized testing many people have you know there have been a number of sort of analyses of that sort but that notwithstanding the the notion of accountability is designed to try as Kevin suggested to provide equity for students who for whom expectations are too low who are not being treated properly uh by by the system so what is your your sense of of uh what what is your view of accountability um I too believe that equity is a good thing and that students should not be treated differently based on the circumstances of their birth um I think you know the assumption the base a basic assumption of good intentions on behalf of anyone engaged in this debate is that that is the baseline that we're coming with and the question is what policies best carry out that principle um I think for a long it's a very the history of it to me describes a very interesting turn because what I see is that these tests were created by people as instruments of supremacy that's how they were created and they were created founded on an ideology of supremacy um I mean this this it's there and it carries from Galton in the 19th century all the way through the bell curve um at some point the instrument created by people as an instrument of supremacy became championed as an instrument of equity and the idea that we would diagnose people um you know in in in the book the the notes use his book he says diagnosis is not discrimination um and I find that to be a curious change that that that analysis is accurate um how do we solve the problem um I'm gonna be the first one to tell you that that that is bigger than the people in this room that's a very big problem it's not just an education problem um and I want uh I want our education system to be focused around it but I don't think it's just going to take our education system to solve it sure so one last thought on that so how do we want to make the perfect the enemy of the good if you will I mean I think Kevin was implying this earlier too that we we all ignore many most people acknowledge the impact of poverty on student achievement for those uh kids of color in many instances but also uh kids who are uh disadvantaged generally right uh what do we do uh do we wait until poverty is is eradicated or do we take steps in schools to uh address the achievement gaps I just don't I don't know that I agree with the term achievement gap I see an opportunity well no this is no I don't see I don't know that I agree with the disparity performance I agree with opportunity gap because what I see is a difference that is described and predicted by the resources that a student has in their and what they arrive with which leads to the achievement gap which leads to a gap in achievement yes but some of the differences are manifested in the schools themselves so it's not it's not as if every school is the same and then the resources vary and we get variable outcomes well every school is not the same and I mean I think um you know I would argue from my experience spending time in in certain schools that once the average income of a student or let's say the percentage reduced lunch goes above a certain amount in that school it's no longer an educational institution primarily because the job of that school becomes to house feed and close students and connect them to social services and to do so many different things that have to be done regardless of their experience in fact their job I think that educators use that oftentimes as something an excuse the pedagogy of poverty of lowballing what you give to children from certain socio and contributes to the achievement gap and a good urban school should look like a good suburban progressive school and what we do is we dumb down the curriculum but do you think the senator testing is stops us from doing that well no but you know I've always taught in poverty schools I mean every school I've taught in over 75 percent of children were free and reduced like but that's not the issue the issue is to teach those children and teach them like you were teaching your own children teach them like you teach middle class and upper middle class and that's what the problem is it is this pedagogy of poverty of these important children I don't have to do much and I've been in too many teachers rooms you know I know the conversation you're an outlier if you are progressive and acting on behalf of children hi Melissa Finkel I'm with Maryland Department of Education and a couple of things that you said tie into my question you mentioned schools in England you mentioned summative tests and then I heard you start to talk about formative and formative assessment is what we work on so I was wondering if in your research in your discussions with schools they talked about assessment for learning or formative assessment as a way to drive student learning and not really focus on the summative at the end say that again I'm sorry I was just wondering if formative assessment or assessment for learning came up in your conversations with schools and if you see that as as a practice we're hoping that it's a practice that's spreading more across the country obviously in Maryland but a number of other states that we work with as well including Michigan and I was just wondering if that's something that you saw at all a discussion of self assessment really student empowerment and feedback in the formative assessment process yes absolutely I mean it's I feel like formative assessment is is part and parcel of good teaching that's what I learned in that it's that good teachers have always seen assessment as an integral part to what they do and part of the reason they object to standardized assessments is that they're it's taking away from the practice that they already have in their classroom of creating assessments and helping helping students to develop their own assessments as well their own process of self-assessment and you know I talked to educators lamenting the fact that they work on individualized assessment plans with students who may have whether they have IEPs or whether they're they're gifted or whether they're mainstream and they have individualized goals for them which the standardized tests don't respect or don't include or ignore and therefore there's a mismatch between what the school's being held accountable for and what the students and the family and the teacher are interested in having that student do and achieve. Hi my name is Sheena Cook I'm a program associate here in the Early Childhood Initiative and you all mentioned kindergarten entry assessments and that's something that I'm interested in. Nacey had released some guidance around it and said that it was supposed to be embedded in the actual curriculum so it wasn't an assessment a kindergarten assessment in the traditional sense but it was something that was going to be observed over time seeing students how they performed in different authentic play situations so something that it's one of the solutions you provided. Some states have taken that so that was a federal policy it's come down to the to the local level and I can give you an example of mountain Montgomery County in the Washington Post recently they described the assessment and it was taking a student outside of the classroom for an hour one on one so the the main the lead teacher is not in the classroom and they're working with the student on this assessment in an inauthentic way for one hour so the policy didn't trickle down the way that that it was intended so my which I mean happens often so what I what I'm asking is so some of your proposed solutions seem at this point I know Maurice has had a long experience and has seen the pendulum shift but me as a formerly young teacher now working here it's really hard to see in the foreseeable solution future these types of authentic assessment solutions so if you could just describe some of like throw out maybe some of your vision of how that could be put into a policy format. Sure so so low quality assessments got really popular for a really good reason which is that they are machine squarable they are industrial mass produced you can drop off packets at a school and then pick them up later and the whole system is very kind of easy to use and any other kind of test is probably going to be harder to implement and more expensive and take more resources and more training than the kind of tests we're using now um on the other hand when you talk to a performance assessment school that is doing things the way they've been doing them for 20 years and you know they don't use regents or high school standardized state standardized tests and I asked them how much more money do you spend and they said we definitely spend less money because our teacher this is part of our teacher's professional development we don't spend money on outside tests outside prep the money is internal and it goes towards training it goes towards moderation studies where schools get together with each other and compare instruments of assessment and all the infrastructure and I think there's increasing attention these days it sounds like to me there's increasing attention on the importance of teacher professional development and continuous development and so if you look upon teachers as a resource in implementing and creating assessments that's where you find the ability to adopt the kinds of tests that you that you want it's more of an HR play than it is a technology play I just you know I want to move from antiquity to recent history um the congress because it invested so much in head start had a national reporting system where head start children were to be assessed and given the lack of reliability of assessing young children the whole program collapsed they don't test there's no longer a national reporting system for head start so I mean that's recent um and because I get back to development you kids are incredible you know young children are incredible but you know even this kindergarten um entrance and and certainly Maryland has been doing this you know Roth is a good friend of mine and colleagues so they have been doing it in a reasonable way and now they're in a consortium with Michigan and other places but I mean there there's a reasonableness to we're not saying we shouldn't get the information about the children right it is how we collect the information with the children well and isn't it the case that in many places the accountability accountability the accountability system for pre-k is also based around I mean robust indicators of high quality pre-k include the education levels and the pay of the teachers yeah I don't want to go into my uh about over promising and under delivering we are not clear about what those in the yes but it is true it commonly we say teacher preparation hi thanks congratulations on the book um I'm Beth Glenn with the Education Justice Network and I wanted to circle back to the um the the robot idea that you uh brought up and um the idea of the civil rights concerns that that Kevin raised and I was wondering if you saw a system in reporting for the book that includes annual determinations of progress that would allow for the kind of disaggregated data for tracking civil rights concerns that like Kevin raised um that puts that into play in a system without the punitive's high stakes testing um that is going on now I mean we we don't yet have this application these applications in place this is pretty theoretical at this point but I mean I don't see why not I guess would be my answer uh example that you talked about with performance sorry the Kentucky example you talked about with performance based assessment you know feeding that into a system where you could give parents information about how their child's learning is going in a quote unquote kind of real-time sense did that get developed when Kentucky did their experiment 20 years ago before it got dropped or did they not get that far um Howard portfolios I don't know how portfolios really figured into the reporting the accountability reporting I know there was part of the picture but I don't know specifically sorry um I do know that in the school in in the um in the Bate independent school district in Kentucky right now where they tried this performance based um assessment experiment they also used the the um ACT and the compass test at the middle school level and their intention was to track high school outcomes as well um to sort of create a bigger a bigger picture idea of how students were doing um yes hi um my name is Ashley Parker I'm uh with Hager Sharp which is a communications firm we work with a number of education clients and so I have two parts to my question the first is that you know when you're looking at assessments and your work for the book like where does some of these low stakes quote unquote tests like nape for example national assessment educational progress come into play and also in your work with speaking with you know sort of folks from the opt-out movement if you saw sort of you know talking about like parents and also students like how their knowledge of the tests and what they are if you saw sort of within those groups if there was knowledge to discern the difference between tests like nape and then some of these high stakes annual tests if there if there was sort of that understanding um survey data tells us uh and interviews my experience tells us that parents have no idea what their kids are thinking or why or what it all means and which test is important for the student which test is important for the school um I talked to the nwa folks um and I was impressed with what they developed and I I see that when you remove stakes from a test you have a chance to extract a lot more value in terms of student learning however where we see nwa being put into place often as a benchmark test for states and districts and schools to improve their performance on the state tests it's just one more test and so from the kids point of view experience wise it's not a great experience to take that test um anyone um twitter that you want to respond to um I it would be hard to but someone's let's see someone says um only 140 characters yes I know right yeah um right yeah so what is a school what is a good school district that's one question we'll end with an easy one yeah um well if we didn't measure good school districts by test scores how would we measure them and that is sort of the question that we're kind of in in in right now and this is the question about equity it's a question about my child do I stay in this part of brooklyn or move to a different part of brooklyn um what do we really need to know about our kids schools and how are we going to find that out um and I think that's probably the next book well we will look forward to that book everyone please join me in thanking marisa and ania for coming today