 Today's event is jointly hosted by ISR Survey Research Center and is also co-sponsored by two of the board schools research centers, the Education Policy Initiative and the Center for Public Policy in Diverse Societies. And I'd like to thank those students who are working together and to actively bring today's round table together, especially Robin Jacob and Ryan Jacob, Robin at ISR, and Ryan, of course, who's a member of the forceful faculty, who will be helping to moderate the panel. Well, as you may know, today's event is part of the university's 28th annual celebration of the Reverend Martin Luther King here. What you may not know is that the theme for today's or for this year's symposium is power, justice, and love, heal the divine. And I have to say that as I reflected on that theme, it seemed to me that the topic for our panel today causes of and potential solutions to educational disparity. That topic is squarely at the forefront of the challenges that we face in terms of healing the divide that confronts our nation. Educational disparities are linked to geographic, socio-economic, and racial inequalities, and they really are grounded in the history of our nation in very foundational ways. And recognizing that we have a long way to go to realize Dr. King's goals of freedom, justice, and equality really requires us to think through, to recognize and to address the critical challenges and disparities in our educational systems. So I look forward to both hearing and then discussing in the second part of our panel the insights and solutions that Stephanie Kerwin and Angel proposed to us this afternoon. Before our participants begin, it is my pleasure to introduce Bill Axon in addition to directing ISR Survey Research Center. He's a professor of sociology and research professor at the Population Study Center. And his research focuses on a wide range of issues related to social change, family organization, intergenerational relationships in the United States, and in a call. And with that very disparate set of perspectives, I'm delighted to welcome him to... Thank you so much, Susan. That was lovely. The Survey Research Center at the Institute for Social Research is pleased to be co-hosting this roundtable. The second of our two MLK Day events focused on the issue of educational disparities in the United States. The Institute has a long history of collecting high-quality social science research on education on social science research in the public interest and especially education. The Survey Research Center launched a few years ago its program in educational well-being. It's led by Brian Rowan and Robin Jacob. And it is the core element of our commitment to studying education and launching this kind of work on education. The education of children plays a vital role in ensuring the long-term prosperity of our society. But it's an area where we as a nation continue to struggle. Thus, we thought it was fitting that this year's MLK Symposium focused on the issue of education disparities. Last week, we presented a play about the struggles of a first-year teacher in the Chicago Public Schools. The play underscored the marked disparities that exist in our educational system, particularly in urban schools. We hope our program today can help us explore more deeply the causes, consequences, and potential solutions to those disparities. I'm now going to turn things over to Robin and Brian Jacob, our co-organizers of this event, who will introduce our speakers. Robin Jacob is a research assistant professor in the Education Well-Being Program at the Survey Research Center and at the School of Education. Brian Jacob is the Walter H. Annenberg Professor of Education Policy, Professor of Economics and Professor of Education at the Ford School of Public Policy. Please help me welcome Robin and Brian. So when Brian and I began organizing this panel, we set out to find a preeminent economist, sociologist, and psychologist to offer perspectives on the issue of educational disparities in the U.S. We wanted a group of people who were well-respected in their own disciplines, who could provide insightful perspective on the issue, who could speak across disciplines, and who would appeal to a wider audience. Stephanie, Angel, and Kerwin more than meet all these criteria and were thrilled to have them here. I'm also really pleasantly surprised to discover that they all have something else in common, which is that they all have a current or formal affiliation with the University of Michigan. And in retrospect, this shouldn't have been a surprise. It simply underscores the excellence of this great university. So the way we're going to proceed now is I'm going to do a brief introduction, formal introduction of the speakers. They'll each speak for about 20 minutes and then we'll open it up for questions. I'd like to remind the audience if you have a question for our speakers, please write it down on one of the cards passed out at the entrance. So Ford School volunteers will be collecting cards at around 4.40 p.m. If you're watching online, please submit your questions via Twitter using the hashtag eddysparities. And so just going in alphabetical order, Kerwin Charles is the Edwin and Betty L. Bergman Distinguished Service Professor at the University of Chicago's Harris School of Public Policy. His research focuses on a range of subjects in the broad area of applied microeconomics. He studied issues such as the effect of racial composition of neighborhoods on the social connections people make, differences in visible consumption across racial and ethnic groups. And in recent work he studied the degree to which prejudice can account for wage and employment differences by race and gender. Also, as many of you know, Kerwin is a former faculty member at the Ford School Deacon Department and ISR, so he is a perfect fit for the round table here. Next, Angel Harris is a professor of sociology and African-American studies at Duke University and co-director of the research network on racial and ethnic identity. His research interests include social inequality, policy and education. His work focuses on a wide variety of social psychological determinants of the racial achievement gap and factors that contribute to differences in academic investment among African-American, Latino and white youth. And as Susan mentioned before, he is a distinguished graduate of the doctoral program in sociology and public policy here at Michigan and we're very pleased to have him back with us. And last but not least, Stephanie Rowley is a professor at the University of Michigan School of Education and the Department of Psychology. She is a psychologist with their PhD in developmental psychology from the University of Virginia and her works looks at the development of achievement motivation and how that differs across race and gender. She's recently started a project looking at the predictors of parent socialization and the effect that that socialization has on the outcomes of children how this varies across race and gender. So we are delighted to have all of our speakers here today. Without further ado I'm going to invite out Angel Harris who is going first because he was bold enough to have PowerPoint slides. So please. Can everyone hear me? Thank you all for coming. I'm honored to be here and I hope that I have some interesting things to say. This is a perspective from one sociologist. So I'm going to jump right in. So what I'm going to talk about today is how I think part of the problem is that there's not enough respect for the problem. Also that I think we're spinning our wheels when it comes to explanations for educational disparities and then I'm going to talk a little bit about what I think we should focus. So I'm not going to really present any research. So what is the achievement gap? How big is the achievement gap? Well what I'm showing you here is achievement for blacks, whites and Hispanics in the 12th grade. So this is based on national assessment of educational progress. So this is national representative data. 12th graders. Each test of scale so that 300 is considered proficient. So you see those are being proficient in black and Hispanic 12th graders. But in order to give you a sense for the gap I'm going to put up a red line and the red line represents white 8th graders. So this is four years worth of growth. But the other thing you'll notice here is that on average white 8th graders are actually black and latino 12th graders are graduating high school with skill sets equivalent to whites in the 8th grade. So it's a four year gap on average. This is a different way of showing you the achievement gap. This is based on NAIP data again national assessment of educational progress. So this is national representative data. And here you're looking at the students at or above based on proficiency level. And I'm going to highlight the bar for blacks. Where you see slightly more than half of blacks are proficient in reading and less than a third are proficient in math, science and U.S. history. Hispanics are not doing too much better. Now this is the gap over time. Okay. So here you're seeing every decade the Department of Education collects large scale data sets. And this will allow researchers to conduct studies on education. And so I'm showing you the standardized gap for the Equal Education Opportunity of 1965, National Longitudinal Study of 72, High School and Beyond of 80 and 82, and the National Education Longitudinal Study of 1992, and the Education Longitudinal Survey of 2006. So these are all 12th graders in those years. And so because you're going across decades and across, you know, different tests, different samples, the gap was standardized. And so you're seeing how the gap changes over time. Well, Hedges and Newell conducted a study in which they calculated that the gap at the rate of the climb from 65 to 92, the gap was closing by .12 standard deviations per decade. So in other words, how many times do you have to subtract this number from that number to get to zero? That's how many decades it's going to take to close the gap. So roughly five to six decades to close the gap, given the rate of the climb from 65 to 92 absent any major intervention. Here's the same information for math. 12th graders, again, rate of the climb is .08 standard deviations. So what you're seeing is that it's going to take roughly 10 decades for the gap to close in math. Absent intervention. And that's given the rate of the climb from 1965 to 1992. So those projections are good up to 92. What has happened since 1992? What's happened since the mid 90s? Here I'm showing you the gap on perhaps the most consequential of exams, the SAT. 96 through 2012. The scores are not important. What's important is that the lines are not converging. And so that convergence from 65 to 92 has stalled. And so now we see that the gap is pretty much, you know, it's flat. And that's reading. And there's math. So essentially what I've shown you so far is that it's a four year gap. It's pretty big. And on average, black and Hispanic 12th graders are graduating with eighth grade skill set. It's pervasive. It spans across a wide range of subjects. Reading, science, math, US history. You can't just focus on one particular subject. It's persistent. It doesn't seem to be going anywhere. And given this trend we haven't even begun to cut into those projected declines yet. So I would still say that those projections are probably still accurate in terms of five decades in reading and close to ten decades in math absent any major intervention. Some news that I thought was good at one point was that these lines, so this is 2012, somewhere toward the end of this right here is 2014. And these lines are going to drastically converge by 2014. The reason why is because the No Child Left Behind Act suggested that this was the year for the gap to close. And so when they suggested that I thought that they were aware of these patterns, but that there was some magic planet they were going to unveil that was going to lead these lines to converge drastically in one year. It just makes you think about when people think about policies, are they realistic? Are they aware of the data? Are they connected to the data and the trends and the patterns? So I've talked about the gap in terms of test scores. Here's the gap in terms of GPA. What I'm showing you is the average GPA for students in 2001 was 330. So this zero represents 330 in 2001. And whites were this much more than the national average, so they're 336. So this is showing you how much more they are on the average in 2001 and in 2011. That's whites. This is Asian Americans. There's Hispanics and blacks. So it's a gap that shows up on standardized tests. It's a gap that shows up on GPA, teacher evaluations. So it's a real problem. Now why is this perhaps I think the biggest problem facing this country this century? You're looking at the U.S. population percentage white and non-white in 2010 and from here on we have projections. Regardless of whether you believe the projections it's certainly clear that the country is rapidly diversifying. You cannot have nearly half of your population walking around with eighth grade skill sets on average. There's no way that doesn't affect everyone. And so this is why I think this is a big problem and I think that when you hear politicians or superintendents say that I'm going to close the gap in my term in office. This is going to be closed within the next three or four years. They don't respect the problem. It's like trying to stop a freight train with a BB gun. You don't walk into the office of an oncologist. This is an example I like to use. You don't walk into the office of an oncologist and say you guys have been working on cancer research for decades, millions of dollars have gone into this. I want to cure by next week. We don't say that because we respect that problem. And so this problem warrants that same level of respect when you realize the depth of it. So part of the problem is that we do have a lack of respect and people actually believe that they're going to solve it within the next five to ten years. The other thing is that I think we're spinning our wheels. So one is that there are tons of explanations that have been put forward for the gap. There are a few of the popular ones. One you have is genetic deficiency. This was put forward by Arthur Jensen, a psychologist, and Herstein and Murray revived it with the publication of the bell shaped curve. And this is there really is not much empirical evidence to support this. Although there are some researchers at UNC Chapel Hill who have been doing some work on the role of genetics in predicting social behaviors. They've there's a data set called ad health that has a thousand variables and each variable corresponds to a particular gene and it's measured as 0, 1, 2. The absence of a gene, the presence, or the presence for sure of the gene. And it's only a matter of time before I think that it starts to get linked to achievement. It's a clumsy way of it's what happens when social scientists try to play geneticists. But nevertheless, I like to think that it's not taken seriously anymore, but I can't really say that knowing that there are some scholars out there that are still kind of thinking in this direction. The next thing is that there are differences in school resources in family structure or family socio-economic status. And this explanation is one that, you know, we've all heard various variations of this. The gap still exists when you control for socio-economic background factors. What that means is to give a brief, really, really quick what I mean by control for is when you equalize on characteristics that's where you're controlling. So an example I give I'm not going to I'm a friendly, quantitative researcher. So oftentimes, let's say you have an achievement gap in this room and women are performing the males, let's just say. And you realize that the women are sitting closer to the board. The average distance for the women is closer. Well, we could say let's control for seating arrangement. And so if you randomize, now the average distance from the board is the same for males and females. And so if you have an achievement gap in the rest, then you see you've control for seating arrangement. And if the gap closes, then you say that was it, right? And so imagine randomizing on SES or comparing kids across racial groups who are similarly situated with regards to socio-economic factors. The gap is still there. It exists. It's only a third smaller. So in other words, there's a gap in Ann Arbor and Shaker Heights, Ohio in Prince George County, Maryland. Another one is Bison Testing. This one I'll talk about in a few minutes. Then there's that schools perpetuate inequality. Essentially they tend to, you know, raise kids to occupy the spaces of society from which they originate. And then finally, there's a cultural efficiency narrative. And so we've all heard some variation of this. This is the one that my research tends to focus on. This is, you know, back is they just don't want to learn. You know, Latinos don't want to learn. And so this is the opposition of culture. They're afraid that if they do well in school, they're going to be accused of acting white. So this is that explanation. There are a lot of other explanations, but these are just some. I am convinced that it's not the culture explanation. I've done, I've conducted several studies on this. I've written a book on the topic and I've looked at six different datasets, two from the UK and I've tried to find evidence for this framework and I do not find support in the US or the UK. So I'm fairly convinced that it's not this. And I just published a book this month on parental involvement in which I was trying to test whether or not one of the things that one can look at is if the achievement gap stems from the fact that Black and Latino parents are less involved. And I don't find support for that. So I'm not convinced. So where should we focus our attention? Okay, so here's the answer. I'm kidding. So testing bias is one. Early schooling is another. Lack of a real dialogue on race and lack of understanding for structure. So I want everyone to take a second and read this question. The actors bearing on stage seem blank. Her movements were natural and her technique blank. The answer is C. But I want people by show of hands to show do you think this question is racially biased? If you do, show of hands. Way up high. Okay? Obviously no, the answer is yes. It is racially biased. But it's not biased in a way most people think it is. Most people are going to say well it's biased because lived experience, you know, Black people don't have this lived experience, Latinos don't have this lived experience and so therefore they're not familiar with the theater. We've all heard some variation of that narrative. That's not why it's biased. It's actually biased against whites. This question is biased against whites. A greater share of Black students answer this question correctly than whites. And so it's biased against whites. When you are in the ETS, Educational Testing Service when they construct their exams there's a roster of questions that are on the SAT or LSAT or whatever exams that they create. There's a roster of questions. And anytime they put new questions on the exam, they never introduce an entirely new exam. That's too risky because we believe that these questions have been embedded and they're measuring what we believe they're supposed to be measuring. Therefore, when you introduce new questions you have to do it slowly, one at a time. And so every test contains some test items that don't count toward the total. When you introduce the test, when everyone takes the test, you then determine whether or not that test item is good enough to be part of a roster. And so the only way you can determine how the question performed is you have to see how it performs relative to the other questions in the roster. So keep that in mind. Basics of ETS construction. Each individual SAT question ETS chooses is required to parallel the outcomes of the test overall. So if high scoring test takers more likely to be white tend to answer the question specifically in pre-testing, it's a worthy SAT question. If not, it's thrown out. Race and ethnicity are not considered explicitly but racially disparate scores drive question selection, which in turn reproduces racially disparate test results in an internally reinforcing cycle. Item selection is not random. This is not a quirk of any one particular SAT test. SATs are designed to be very strongly correlated one another. I don't believe that ETS intended for the SAT to be a white preference test. However, the scientific test construction method the company uses leads to this result. The actor's bearing question that you just read looks like a typical SAT verbal question. Yet the question differs from others in one important respect. According to ETS, 8% more African Americans than whites answer the question correctly. So it's a black preference question. Nearly all SAT questions capture something about race that can't be determined under pre-testing. Because it favored blacks who score lower than whites over blacks overall, this question which was pre-tested in 1998 did not favor high scores and therefore was rejected for use on the SAT. There are several questions that are black preference SAT math questions that are also rejected. So is it fair? On October 98, every single one of the questions on the SAT favored whites over blacks. Latino test takers are similarly affected, faring only better than blacks. The same pattern holds for the LSAT and other popular students. If you are black and you did poorly on the SAT, don't worry about it. It was rigged. It was rigged. If you're white and you did really well on the SAT, don't get too happy. It was rigged. This is how this pattern makes sense. Now this pattern makes sense to me. If I construct the test and I have 100 items and I give it to everyone in this room and I only select the items for which females score correctly, a greater share of females score correctly, and I only select those items. I'm going to have a gender gap and that gap is going to be persistent because I'm only choosing the questions that a greater share of females score correctly and that's how these patterns make sense. That's why they're so consistent. They're so persistent. So that's one thing. That's bison testing. The second thing, the second place where we need to focus is on early schooling. Here I'm showing you achievement in the fall of kindergarten. This is based on data from Fryer and Leavitt. What I did was I plotted their findings. I just made a visual image of their findings. This is fall of kindergarten. Blacks, Asian Americans, whites. Whites are the average. I'm just showing you if whites scored a 20 on average and Asians were 22, I'm showing you the plus two. Here is the same information for social class. Social background factors listed below. Here's the gap after you control for social class in the fall of kindergarten and keeping those factors controlled moving across time, that's what happens to the gap. Essentially if you compare blacks and whites who are similarly situated in regard to these factors in social class, blacks actually do better than whites in fall of kindergarten in reading. That's reading and that's math. Blacks and whites are most similar to one another academically when they walk into the school system. Here is data based on the CDS, Child Development Supplement of the Panel Study of Income Dynamics and it's not panel, but what I'm showing you is that the gap for first graders, second graders, this is how much worse blacks are doing relative to whites, reading and math, Woodcock Johnson and you see that the gap widens and it kind of settles down somewhere around middle school. But they're most similar here. So here I'm showing you two kids, a white kid and a black kid and they're growing, so this is years, this axis, this is knowledge, make no assumptions about the gap here. Here they're entering the school system and this is what happens and this is what actually should happen because they're most similar when they come in here and it's after they come in here that you start to see the divergence. So the final thing, the final place where I think we need to focus, so that was bison testing, that was early schooling. The other thing is understanding probability and distribution and that's something that we don't understand and this is important for educators is that here's an example that I give in order to bring across a sociological imagination. Say you have 80 red students and 80 blue students and the red students took a test prep course well I'm putting a test course down here, so this is one blue kid and that represents one person. So here you have 80 red students and 80 blue students. Each group is going to have that's the average for everyone but each group is going to have its own distribution. And so what you see here is that not everyone who took the test passed and not everyone who did not failed. So these are the personal problems and this is a success story. This morning I actually learned from Kerwin the phrase gai hu gai hu and that will become obvious now in a second. So examples I like to use this for is let's say if you took all these people and randomly sorted them into Hawaii and Kansas and come back 10 years later in which place do you think you're going to have a high proportion of swimmers? So let's pretend that this is score on a swimming exam and these are the people sorted into Hawaii and these are the people sorted into Kansas. Well everyone in Hawaii can swim and not everyone in Kansas can't. This is the gai hu this is the person who says it's not true that if you come from Kansas you can't swim because I know a gai hu swims like a fish and he's from Kansas and then somebody says I know another guy who is from Kansas and can't swim at all and so the point is this becomes that charter school that's working and we want to build policy around gai hu and that's not understanding the notion of distribution and where in the distribution are we looking this data point that we're looking at where in the distribution does it sit and I think that that's something that we need to learn more about this is another little example I like I have one minute left but I want to kind of go through this so this is a Florida hood town so here you have nice cars and you have some potholes and this person didn't make it but these people made it here it's not quite the same these people can look at them and say hey we made it to the end we have fewer car repair bills we're better drivers than you what's wrong with you guys right and that's often not understanding the distribution that they belong to that this person here is not the same as them because look at all they have to go through to get to where they are and so it's lack of understanding of proportions of distributions and then the final thing is that and I'll get to the the punchline is that we don't know how to talk about race in this country and if we're going to talk about the achievement gap and how to solve a racial problem we have to be comfortable talking about race and that's not something that we do very well and so there are studies that show that teachers one study I'll just briefly summarize one study in which a black kid and a white kid were video recorded and they were told look today you guys are going to be white for the day y'all know what that means they were white for the day you know they were white for the day and in the next day they were recorded and they were told to be black for the day and you know what that means they had swagger for the day and these videos were shown to teachers and teachers were asked to evaluate these students in terms of problem behaviors the need for special education and achievement guess which pair was rated worse on all three just on the basis of the video and so these are biases that people have and it's a result of us not knowing so this is my final slide I think that there's a dialogue among blacks about race each group has a dialogue about race and so you know if you go to the living room of black folk you know after dinner and they get to talking you go hear some dialogue about race right and you know there are things that black folks say when white folks ain't around white folks have a dialogue about race but see this is segregated white school and they have a dialogue he ain't part of it it's probably about him but he's not a part of it so he doesn't know what they're talking about right and so what's in this dialogue what do black folks say well we say you can't trust white people they wear shorts in the winter they don't see their privilege they stereotype us, they're racist in denial these are the kinds of things we say what do white folks say when we're not around this is what they say I'm going to tell y'all no I don't know I don't know but the point is we have very little overlap we have very little overlap in terms of the kinds of things we actually talk about and so we have to be able to talk about race in a real way in this country before we can solve the achievement gap thank you well good evening good afternoon everyone so again I'm Stephanie Rowley and my primary appointment in psychology so as Robin and Brian asked me to think about how people in my field think about educational disparities thinking primarily about psychological approaches and I guess when I was thinking about psychology I was thinking about the stereotypic psychology which is thinking about what happens in the brain though psychology is technically the study of behavior so first I just wanted to say thank you to Brian and to Robin and to Susan and to Bill for having me today on my way over I got a I don't know you got a Facebook post shout out from a good friend of mine from undergrad who was reminding me that 28 years ago he and a bunch of my friends who were here as undergrads at Michigan back in the late 80s I guess I might as well say since it's been 28 years we're the first to really fight for the celebration for Martin Luther King's birthday and so what an honor to be here very exciting so Brian and Robin again said okay what did your field say about educational disparities so I started thinking about it and what I realized is that psychologists say relatively little about educational disparities that is when you look at the literature in psychology starting again in the late 80s coincidentally what you see is a lot of discussion about the nature of race comparative research and one of the things that happened right here at Michigan actually I was an undergrad at the time folks like Bonnie McLeod who's one of my mentors started studying the way we look at race in psychology and one of the things that came out of that body of work is that we do a really poor job of comparing people across race and oftentimes we don't equate people for socioeconomic status and in fact we're choosing middle income whites and comparing them and we're ending up with these deficit models of learning and development that are really damaging to the low income ethnic minority children in particular so this was primarily in developmental psychology but not exclusively and so there was this movement to move away from race comparative research not because there's something inherently bad about race comparative research because we were doing an inherently bad job of carrying out this research so relatively few people are doing what Angel was talking about in terms of comparing whites and blacks and finding those variables that explain disparities between them in psychology but there are certainly a few areas where people are doing some really nice race comparative work and demonstrating that issues around discrimination, stereotyping classroom culture are all reasonable explanations of achievement gaps, I would say reasonable in that there's still not really explaining a lot of the variability so there's clearly something going on that's beyond socioeconomic status, that's beyond stereotyping it's difficult to really gauge discrimination, I would say perhaps it's not so far beyond discrimination but I really was taking trying to take it as or think about it in terms of psychology proper so the interesting thing though is if you pick up psychological journals and you read anything about African-Americans or Latinos the very first paragraph of those papers is going to say blacks do less well in school than whites Latinos do less well in school than whites there is an achievement gap that has persisted over time and yet when you actually look at the research is that they're not race comparative at all that you're saying oh racial identity predicts achievement for African-Americans or maybe that parent involvement predicts achievement for Latinos so I think that one of the things that's happened in psychology in particular is that we have problematized African-Americans and Latinos and we begin every debate about blacks and Latinos from the perspective of this deficit or difference so I've had colleagues actually and others say to me but how can you talk about black people without talking about the achievement gap you can't talk about how well black students do in school without pointing out the fact that they're not doing well right and in fact this is the elephant in the room if you don't say that first and foremost I often struggle with my students because this is where they want to start and to me the real elephant in the room is why don't we talk about that Asian white gap that Angel had on those slides why are we not concerned about the poor performance of white students in the United States and why is it that when we talk about achievement gaps we start at low income ethnic minority students there could be lots of reasons for that I don't know but I think there's something in the way that we frame the achievement gap even the very term that begins at this place of of deficit of vulnerability of pathology etc and I think that that then pervades experiences of these students that we're talking about so I'm really convinced that in many ways we're doing damage to the very students that we're trying to help by framing their very existence in terms of the way that they are less than different from etc so I got really interested in this a couple years ago so I was reading the Ann Arbor news back when there was an Ann Arbor news and there was a story there about the achievement gap in Ann Arbor so again as Angel pointed out there was a sizable achievement gap in Ann Arbor and people say but the schools are great and you know there's so many resources here so many opportunities I just don't understand it topic for a different day but my point was in reading that article what I noticed is that the way they were talking about the achievement gap and the way that local schools were trying to address this was through addressing the children that is they were saying well we need to throw more money at them we need more special programs we need more tutoring we need people from U of M to come in and sit with them no offense against people from U of M who come in and sit with students because I think it's great but we problematized the child rather than the school so who said what can we do to the school context that would change things and in fact there are a number of achievement gap interventions that have been really successful in North Carolina for instance so one of my favorites is where the school invited high performing african-american students to visit just visit a gifted classroom where they didn't have test scores that were high enough to actually get into gifted programs but they were allowed to visit for a year so at the end of the year they retested these kids who were visiting and lo and behold their test scores were high enough for them to legitimately be in the class well what happens you put a kid in the classroom with the best teacher in the school with the best students in the school with the most rigorous curriculum in the school and they flourish but instead what we often want to do is segregate these children and special education programs and special tutoring and special this that and the other we don't think about the larger context and so my first point is really that we need to think about how indeed we are framing the problem of the achievement gap and where we locate the problem so in one of my own studies they came out of this so I wrote a small grant in response to my thinking about this because I said what do parents and kids think so as a black parent when I said my kid to school every day am I thinking I'm sending him off to be part of the gap you know kind of and what happens to parents when they do that and so what we did is we interviewed black and latino parents and their kids and we gave them the story about the achievement gap half of them read about the achievement gap and then we asked the parents after reading the story to help their kids on like a math homework type task and we you know we played it up it's really difficult and even the parents who were in the priming group were told that school is hard and the task is hard and things like that and what we found in those parents who were reminded of the achievement gap before they helped their kids was that these parents were more intrusive so when they help their kids with the task they were more likely to take the task over for their child we know that autonomy is really important for kids at this age these are mental school kids we found that they were more negative and anxious during the task and that they were less supportive of their kids so our interpretation is that these the framing of the achievement gap the constant reminders in the minds of parents and teachers and kids are then leading parents to eventually maladaptive parenting so you're not allowing your kids to explore things on their own because you're so nervous about how they're going to perform and of course this is going to trickle down to the kids and how they feel about themselves obviously we know that this is also affecting teachers as well so I think that this deficit approach also addresses the kinds of questions that we undertake another one of my colleagues Dr. Hagan did a study of the outlets the top outlets for adolescent research in psychology and he gathered all of the articles from these journals from a period of time and what he found is that the studies of black and latino youth were almost all pathology based saying what is wrong with black and latino youth the studies of white and asian students of white and asian youth were primarily normative so how does their life unfold over time and so of course we're beginning to get to this point where a child comes into the classroom and the teacher can tell by looking at them that they are a problem because they are part of a particular group and we know from studies of young teachers in particular that they're heavily swayed by these stereotypes again as Angel pointed out simply by looking at children by hearing them speak by looking at their clothing, their hairstyle etc. they're making judgments about who these children are and what they need to do to fix them so there are certain kids who need to be fixed when they walk in the classroom which has serious implications for their ability to move within that space so this brings me to my second thought is that we need to move outside of the box of just thinking about what's happening academically in the classroom and think about some non-academic things that are affecting children that down the line affect performance one of the big things that I've been thinking about a lot for the past several years is the school to prison pipeline and the fact that black and latino youth are much more likely to end up suspended and expelled from school these rates of differences are gigantic three fold four fold in some cases and so what's happening is the ACLU has put together a lot of really wonderful resources on the topic and what's happening is a black child engages in the same behavior as a white child and the black child is more likely to be suspended or expelled the other thing is African American children are more likely to be suspended for what I call soft defenses things like being disrespectful to the teacher or disruptive to the class whereas white students are more likely to be suspended or expelled for more concrete defenses like smoking or bringing drugs to school et cetera so that means there's more question or more subjectivity I think in line with what's happening with a lot of the African American students so the psychological issue here is that what then happens to the student who is suspended so you go home for like two weeks my son had a friend who was suspended in the sixth grade for sending an inappropriate note to a girl he was suspended for 14 days in the sixth grade right and no he got 10 days she got 14 because she initiated the contact but the point is for 14 days he sat at home where were his parents they had to work he's at home by himself for like 14 days so do you think he got his homework do you think that he was missing out on instruction and then psychologically what happens to him as he returns to school so he comes back to school and his friends are looking at him this has disrupted his relationship with the teacher it has disrupted his learning process the whole class has moved on without him they're on two units further and he's ready to go to sleep so he kind of checks out or he engages in further misbehavior and so I think this is a serious problem because again the disparity in disciplinary action is leading to disparities in achievement one other kind of classroom issue that I wanted to raise as thinking again about psychologists is the issue of physical health and well-being so we know that African American children in particular are losing many more days of school to illness and asthma is one of the biggest illnesses that is keeping African American children home they're missing many more days of school due to asthma also obesity related illness and sleep related disorder so lately I've been really interested in the causes of stress and how they manifest in achievement and it's via these health related issues so we know that asthma is exacerbated greatly by stress we also know that sleep is disrupted by stress and that African American students are getting less sleep than pretty much anybody else poor quality sleep than anybody else and we have found recently in some of the work we're doing in the study of black youth in context here at Michigan we're finding that this disordered sleep is then leading to difficulties so lower persistence in the classroom lower engagement in the classroom less preparation for school you're less likely to do your homework and of course you're sleepy all day so it's causing problems in social relationships and a whole host of other things and so what you know I think with the adult literature on things like racial discrimination what we're finding is that there's lots of connections to psychophysiology and these physical kinds of outcomes we're thinking about this less I think in the lives of children and how these diseases are racialized in some way at least in terms of the rates of that African American kids in particular are experiencing them so to conclude psychology does not tend to study educational disparities by and large so we tend to study predictors of performance within African American within Latino within other ethnic groups however we tend to still frame things in terms of educational disparities but only for certain groups so the disparities are only relevant at certain times although identity related processes are probably most explained so I didn't talk about the whole host of other things that predict achievement for students of color there are so many things outside of the academic things that I think psychologists study but we haven't brought to bear on this question of educational disparities in race and ethnic related disparities although consideration of context is integral to psychological studies so we do study context all the time I also think that more should be done to fully explore the role of school resegregation so what happens as schools are rapidly resegregating around the country in Michigan there's the schools of choice initiative led to high concentrations of poverty so this is where children can go to a neighboring school take their money with them and go to this other school because it's better academically or in whatever way and what it has left is a number of districts that are primarily minority in the state that are highly concentrated in terms of poverty and so there's a question of how some certain contexts are working in tandem with race and then also some of these policies so one of the things that's interesting is Angel kept saying without intervention but we're thinking of no child left behind in effect as an intervention and what we know is that the gap has closed almost not at all since 2003 and so you know we have to think about how these interventions are also causing unexpected negative consequences things like high rates of instability among these schools where principals are having to leave as part of restructuring whole teaching staffs are being fired and these children's base of support is being eroded because someone said you have to do something radical because things are not working out other things school start times access to health insurance I think are a number of different policies that are directly affecting these issues thank you I look forward to your questions so thank you very much for having me I'm very happy to be back back home as I was saying earlier today lots of the things I wanted to touch upon have already been mentioned by my other panelists in particular the fact that the racial achievement gap is massive is very well known to you so what I'm going to do for the first few minutes of my talk is in fact the entire talk is I'm going to speak both about this has been described thus far and attainment years of completed schooling etc so when I say attainment I'm going to mean both things so when you look at the data the striking thing to the social scientists is that the gap however measured achievement GPA or whatever thing or propensity to graduate high school these are big gaps as has been said they're big they appear to be durable and we observe them irrespective of the measure we look at precisely why that should be is very puzzling I think something we have not talked about today and something I've been hearing about more and more is the possibility that the gaps might differ by gender among blacks which if true I don't know this to be true but I've been hearing about these very first that if that is true that's a deeply tantalizing result like why should it be the case that girls should be fearing better than boys on some tests if their environments are almost exactly identical I say almost exactly because a brother and sister do not live in precisely the same environment parents interact with them differently thus and so but access to various resources are roughly similar leaving that point aside the thing is big it's not clear exactly why it's so big it's not obvious what to do about it and so I will spend 10 minutes or so talking about three questions one is what accounts for the gap a second question is why should we care that the gaps are as huge as they are and finally how might we fix them okay and so I'm going to begin with the middle one why should we care about the fact that the racial attainment slash achievement gap is big from the perspective from the perspective of an economist I think about consequences of these gaps manifest in themselves chiefly in the labor market okay so I look out at the world most people receive their material well-being from their earnings and so I say if I look at people's earnings I observe dramatic differences by race blacks earn way less than whites on average Latinos are in between good what accounts for that adult material well-being differential one striking thing we see in lots of research is that education attainment or achievement explains the overwhelming bulk of indeed there's a very well-known paper by my colleague at Chicago, Derek Neal in which he argues that you can explain the entirety of the black-white earnings differential that's conditioning on schooling some of those results have been not called into question in terms of the quality of the research conducted but looking at things slightly differently it appears that the overwhelming majority of the gap can be explained some believe by schooling attainment differences and so to the extent that we care about people's consumption or well-being or access to resources over the course of their lifetime the fact that education explains much of it is one reason we should care yes the second reason we should care is that if you turn your gaze away from earnings and you focus on things like incarceration probability or you look at things like marital durability yeah whatever you look at it turns out that being more educated I've not mentioned race in this part of the conversation that higher levels of education act as a kind of insurance against these negative outcomes and so for all these reasons the fact that in terms of earnings in fact of outcomes like the ones I've mentioned incarceration marriage whatever that schooling so care so sharply protects you from bad outcomes is one reason we should yet another reason we should care there's a third reason I think a third set of reasons I don't know exactly to frame this I think that one of the really dangerous things about durable gaps in attainment is that even if the explanatory power of education for earnings or incarceration were zero let's think for a second imagine it were the case that education differences played no role in my earnings later in life and did not in fact affect whether I went to prison or not yeah education and the exposure to education and childhood is something we all share the fact that there are durable differences by race in my view causes people to question the otherness the otherness of blacks in particular that what is it with these people kind of like that and they don't say that exactly but when I see a pattern that says the 12th grade African Americans on average have four years less schooling than whites there are people who would see that result many millions of people and say I told you about them people they might not even say that but think it which is even more dangerous and when you think or are convinced about the otherness of a kind of person then bad outcomes that afflict that person or that type of person do not strike you as anomalous and worried of intervention and so we should be concerned about the gaps for at least these three reasons in my view as has been said I'm an economist and so the economist turns his attention to say we should care about these things what explains them how do they come to be and so in economics we use a particular prism to examine the education decision remember I said I will be using attainment to mean both achievement and say the completion of a given level of school economists tend to regard education as principally the result of an investment decision the logic here is that some agent you yourself if you're an adult or someone acting on your behalf if you're a child causes this agent decides to get more schooling so as to receive greater returns in the future now this investment occurs at a cost and so education has built in within it this necessary dynamic thing you see where I'm incurring the cost today or my child is incurring the cost today and the benefits to her are being received somewhere in the future maybe in the distant future and so why might we suppose there to be systematic differences by race in the cost of attainment so one possibility might have to do with I'm not sure this is me speculating here might have to do with the support that families can provide that's one possibility and so education is acquired at a certain psychic cost if I have in fact had calculus I can instruct my child in calculus thus and so families that have not been so privileged can't do it that's kind of one it's material disadvantage contemporary disadvantage faced by the child herself or her parent that prevents attainment levels from being equal another possibility has to do with expected future returns and so one of the things I think we have done too little of in economics is to link the education decision the decision to invest today for payoffs received tomorrow to phenomena that go on in the labor market might it be the case that African American parents or African American young adults contemplating the college enrollment choice ask themselves a question like this I go to college to this wonderful university like Michigan and I major in whatever thing and let there be some likelihood of labor market discrimination when I'm aged 25 or 30 of some amount might not at the margin affect my decision might in particular cause me not to go relative to my equally talented peer my parent having that kind of belief not nudge her child as far along as she could as she should because of precisely this kind of concern and so there are reasons to suppose that features of the labor market and the way the labor market has historically rewarded families expectations about likely future rewards might systematically affect the propensity of people who differ by race to achieve different levels given levels of school so that's a human capital of you there are other features of human capital I want to briefly touch upon notice that when we talk about expected future returns the investor or the investor's agent Kerwin acting on behalf of his child investor's agent has to form this expectation of what earnings will be in the future discrimination expected future to discrimination might enter into my belief about my child's future earnings but more generally maybe I'm told by the psychological literature there's something about systematic subjective belief differences by race the source of which I do not know but if there are in fact systematic subjective probabilistic differences say blacks are more pessimistic about the country for whatever rational reason it might well be that even the absence of discrimination their belief that investment today pays off positively down the line might be smaller than is true for what I've mentioned things here that economists don't necessarily focus on so much but I thought I'd mention them one set of things we do focus on a lot have to do with the educational production function by the educational production function here I mean the mechanism by which inputs bringing your kid to school depositing him or her how that's translated into an educated kid down the line if you think about the educational production function it has various inputs one is the teachers another is a textbook a third is the unions for that to be key explanatory for it to be the principal source of the difference by race one of two things has got to be true it seems to me it has either got to be true that African Americans and Latinos confront an educational production function fundamentally different from that of whites now the more integrated the society is the less likely that is to be true but it turns out that the society is becoming more and more re-segregated so it turned this argument the possibility that the nature of educational production itself differs systematically by race is something that must be seriously contemplated notice that even if educational production is exactly the same its effects might differ by race and so one of my friends told me that he sends this kid to a school in Chicago where we both live that's a charter school and and it's a discipline and focus charter school and so I cast this person on the school and I'm just this is the thing and so you go in in the morning and everybody has to stand against the wall that way and then you have to raise your hand to do that and then you're kind of like that I thought about this and I thought this would be terrible for my son it would be terrible for him the interaction of his kind of his endowments and rules much to my dismay is not good it's been putatively race neutral education production that nonetheless have usually different racial effects for reasons of history and the like I'll take some you know I'll leave this aside some other points about educational production for the question and answer period I want to make sure I spend a few minutes talking about fixes what might we do about it it's a tricky thing I'm always very nervous about solutions because lots of our contemporary problems were someone's solution the other day very nervous about that well let's think about this it seems to me that we can think of the schooling attainment let's split schooling into junior and senior whereby junior here I mean someone for whom the educational investment decision is made on their behalf by an altruistic parent or guardian and on the other side think of it as an adult decision where a child is deciding whether to study her physics homework or to write the essay or something like that now one of the really interesting things to me about the junior investment decision is that lots of the policies I've seen proposed might not have the effect we intend for if you reflect on it briefly predictable reasons let me give one example one example I have in mind is the role of teachers in education production and so economists in general and labor economists in particular have spent lots of time over the last decade or so coming up with very convincing causal estimates we call them about the effect of teacher quality on whatever thing and let's suppose it's done perfectly it's an R.D. people in the Harris before school knows what I mean it's clean, it's beautiful and so better teachers based on their score on some tests translate into a 7% increase in whatever thing and so you're a mayor of a city you're a mayor of a city and someone brings you this excellent result and report in fact it's 12% not 7, 12% and he says what we have to do we have to improve the teachers in these schools the quality of the teachers the education side to get that process initiated and she fires teachers because lots of them have not scored well you might imagine the black parents curiously would end up opposing that very initiative and why might they do that they won't oppose the initiative because they do not care about the education attainment of their children instead it might be the case that the people fired under this policy represent what currently constitutes the black middle class and so if you think about the people terminated in Washington D.C. under Michelle Ray all of them are African American middle class teachers and so you say on the one hand if I improve the quality of the schools or the teachers children's outcomes would improve but the parents and the children themselves know that the people at church there aren't not everyone they know have lost their jobs under this scheme they will oppose the policy for the very rational reason that the policy has a spillover effect that's inimical to its stated goal that's one possible a second implementation issue I want to mention in my remaining time has to do with the higher level of education folks like people in this room what we've not talked about much is the fact that at selective schools Berkeley and Michigan and Princeton and Yale and so on the share of students who are African American and Latino keeps falling that's another achievement gap it's not just the eighth grade what about that how do we fix that it turns out that fixing that raises all kinds of complicated questions about fairness and justice and yes so my bottom line here is that whatever remedy we propose whatever remedy we propose will require the enthusiastic buy-in not only of the putative beneficiaries of the policy like the black parents in Washington DC but of the broader community because in the absence of that buy-in there will be no program I can think of that will close these differences sorry I'm not such a pessimist and the second thing I want to say is that whereas economics has brought lots of sharp insights to these questions many of the things I've mentioned are not within the purview of economics we can't answer them at all or answer them well and so whatever answer is ultimately devised we'll be going to answer the draws from the expertise of people in political science and psychology and sociology and that's a plea to my economist colleagues, thanks so we have of course more questions to have time for but so I'm just going to jump right in so a question for Angel we had a couple of these so I'll kill two birds with one stone so a couple questions about what would a real dialogue between races look like and do you have any suggestions for how to facilitate that or encourage it yeah well I think that a real dialogue is one in which why folks say what they want to say and black folks don't become angry with them and so for example one of the things that happens often times is that there are thoughts that many people's grandparents might have or you cringe you say oh I hope my friends don't know that my grandmother feels this way you cringe and a lot of us may have these views but if I'm a white person I'm going to be careful about what I say because I don't want that black person sitting in class looking at me like say something say something you know how it is and so the last thing you want to do is be label this racist for a genuine question that you have that's real right because we and so we have to be able to have a dialogue where both groups feel comfortable and you have to appreciate that whatever someone says you have to say okay given their lived experience this view makes sense to them the question then becomes how do I present alternative pieces of data that helps them see a different view or a different side because we're all racist of some degree I'll give you one quick example I remember I was at Princeton and I heard a girl shouting across campus to get someone else's attention and I heard Sarah and I turn around and I thought and I said oh thank goodness she's white and the reason why is because if she was black I would cringe because it doesn't look the same it looks worse this is why the same behavioral infraction for black and white kids the black is punished more harshly and when you look at blacks and whites just think about it see a bunch of white kids outside skateboarding see a bunch of black kids it doesn't look the same so when I hear some kids in the library making noise I always hope that they're white because they can sustain that to their image but for black folks I cringe I'm like oh God because of what it's going to how it's going to be perceived by the other and so we have to be able to be comfortable talking about these things because teachers see this and they're making assessments on what's the same but it's really not the same another question that was came up a few times in the from the audience had to do with the gender differences and so there was alluded to before there certainly is the gender differences among whites but the gender differences favoring girls is even large among African Americans what can this tell us about potential explanations and potential solutions to attainment among African Americans so I mentioned gender differences and I think that I tried to say but didn't come across clearly perhaps we know let's imagine a brother and sister in the same household it is the easy thing the casual thing the careless thing to say it's the same environment let's imagine that African American children are disproportionately likely to live in single mother households my colleague Dan Black and I have done this very simple thing and that's why we examined that single mother's knowledge that's why I asked this question about parental familiarity with the teenage child's friends the mother knows a lot about the girl's friends like when she's going out with what she's doing and knows way less about her teenage son I make no causal claim don't leave and say Kerwin said that's interesting to me and there's a kind of logic internal logic like I can imagine a mother saying what do I know from being a teenage boy I think this is an area demanding more investigation but at a minimum it suggests that coarse kind of unsubtle arguments about the nature of material deprivation on outcomes need to be modified and sharpened if outcomes are that different by gender and they are different I think it depends on the differences you're looking at so there are a lot of basic grades in elementary school and middle school for instance the gap between white girls and boys and black girls and boys are quite similar attainment certainly there are large differences so who's going to college there are larger differences as you move through high school in terms of course taking and things like that but I think there's also this tendency to distort some of the differences and assume that there's this double jeopardy for black boys where in black boys are doing exponentially worse than black girls or white boys in school and that's not necessarily true I think in psychology there have been some similar kinds of models where it's been found that among African American parents well no among parents mothers tend to have higher expectations for their daughters and to put more demands on their daughters for maturity and fathers tend to do the same for sons so fathers have higher expectations for sons and put more have greater demands on them and the issue then as it relates to socioeconomic status is that low income African American parents or families are likely to be headed by single women and so if you're a son you get the sort of warmth and support of the mother but not necessarily the high expectations of the father also don't want to assume that just because the father's not residential that he's not involved there seems to be some suggestion that that's one of the reasons for some of those disparities I think a lot of it has to do with the fact that we all navigate different spaces and so there was there was a movie by shallow how which was a movie with Jack Black in which he fell in love with he was put on him and he was supposed fall in love with people's see people's inner beauty and if someone was so he fell in love with so he saw as Gwyneth Paltrow but the world saw as a woman in a fat suit but it was a very well done fat suit so it wasn't like Eddie Murphy the clumps you know it looked like a real person so he sees he falls in love with her and during the making of the movie I saw Gwyneth Paltrow being interviewed in the fat suit and she says you know it takes hours to put this on and take it off so we get takes as many takes as we can so in between takes she's walking through the lobby of the hotel and she says you know where's that I that I received she's for the first time I know what it's like to live life in a body like this an obese person will look at her and say oh yeah that's my life that's every day for me all of us are navigating the world and the world's interacting with us in different ways right and so as a as a white woman there are things that you pick up about the world and things that you can access that I can't and so just think about this the amount of distance that we have to cover in social interactions spaces like this the amount of distance you know we all have a switch we have to be on the amount of distance that white people white males have to have when they turn the switch on it's not as much distance as a black person in this space you know here we're on we're formal and then when we go home it's what's up man you know you you know it's a big distance you have to cover and then if you black you add to that distance the whole male you know criminal thug jail you had all those stereotypes and so just imagine that the world is kind of um perceiving black males in a particular way uh the worst of all groups uh and they have to navigate that in schools as children teachers are kind of you know I'm gonna fix you you know teachers are projecting these negative stereotypes on them even black teachers themselves especially because their thing is I'm gonna fix you so you won't shame us when you get out there you see what I'm saying and so you're always communicating that there's something wrong with your blackness or your black maleness you have to be fixed and so I think that that plays a role as well um so there's a question here about the role that federal policy plays in all of this and I'm gonna expand it a little bit to say more generally since we're at a policy school what what role can policy play in this is it always a negative role which some of you highlighted in or can it be a positive role and I um look I so I'm suppose suppose we didn't measure by race suppose we didn't measure by race at all what would be the thing about the world suppose we didn't know right that NAEP scores in Alabama or whatever or X percent smaller for black children and we weren't walking around with that in our heads actually don't have an answer to this question I'm just saying it's interesting to think about right suppose we didn't know that um incarceration rates are that's clear federal policy and I think we never question things that we have inherited from history there's no reason it seems to me to collect data carefully by race we don't collect data by height um and if you believe it to be an equally uninteresting datum which I do not believe but work with me um then why not add it to the right um testing is interesting maybe make tests lower stakes perhaps um I think it's good that we know how children are faring um and I think it is peculiar that the state the country varies so widely based on the accent of where a child happened to be born in what is expected of her that she be taught or that she know in high school or middle school for that matter um I know this is a very controversial thing and it's particular implementation under this and the previous president make it even more so um I like that policy approach and the third one is it seems to me that our efforts to narrow racial differences in schooling and other things if applied too late are almost doomed to fail um that my own conjecture is that by senior year of high school is too late yeah I don't have an education expert that's my guess kids six spend do you know so tilt federal dollars away from higher ed and tertiary ed towards um primary school and maybe even before again I say all this without any there's no regression there's no yeah somebody yeah that's just yeah yeah I would say I think we need more creativity and so clearly what we're doing now is not working um and there are some people who say they have the answer some school some charter school here some charter school there but clearly overall it's whatever we're doing it's not working so what we need is we need uh anytime you see an intervention it's usually an incremental change so they're taking the basic model of what we have now and they're just adding this let's add a little more dollars let's add less less increase teacher training here and so it's incremental changes and then we wonder why it's not working because it's the same fundamental model what I'm saying is perhaps we need an entirely different model of knowledge delivery I don't know what it looks like but if you told me hey change basketball I'm saying okay I'm putting three goals on the court and two balls I don't know what that game looks like but if you force me to come up with a set of rules to accommodate that structure that's a different game and I'm saying that that's what we need with education we need an entirely different game maybe we can have multiple models you know uh you know that you over here try this model and you know let's see what happens because right now what we're doing is not working so let's try different models we have to get creative and so this is again with no data this is all from the gut as an empiricist I should be uncomfortable talking like this but given that nothing is working I think that you know we need to take more risks and just be more creative and be open to completely drastic models of just of educational delivery I think we just have one more question because of the time um this is something this is something that a few people have mentioned there's uh the role of what economists sometimes call as non-cognitive skills motivation, grit, determination you know other things has been you know received a lot of attention certainly in social science literature recently what um what can we learn from the three disciplines about ways in which these type of non-cognitive abilities uh you know could be used to improve the uh the outcomes of low-income and African-American children the psychologist well I mean I think there uh so psychologists also think about the non-cognitive factors that are involved in education um and I really think a lot of what happens in the schooling of African-American kids in particular Latino kids and low-income kids and you know thinking about this concept of uh of social distance and the meaning of that social distance right so if you're an immigrant from anywhere there's a social distance because you're coming to a new country but that that distance may mean something different in terms of how you uh are being read by others and how and how you view yourself and I think that um you know I wouldn't want to come down to a certain set of variables but I think that a lot of the uh disruptions at school come from the um the perceived distance the perceived otherness of um of the kids and so I think the the interventions that I think really work either make those factors unimportant so that is if you figure that uh parent involvement is the problem and disparities in involvement are the problem then you set up a system where in either you can get all the parents involved or you make parent involvement unimportant so I think um you know thinking about some of the charter school you know the kid categories or wherever that that are are working in some ways um I think they've been able to overcome the social distance expectations and um provide high quality social relationships but I think it's a model that's difficult to write. I was going to say you know maybe it's just me but people who persist and stick to it they usually do it at things that they're good at I don't know too many people that persist at things they're not good at you know it's you know usually they give it up at a certain point and so um perhaps what needs to happen is that students particularly low achieving students need to have more victories um along the process uh because that keeps you persisting it's when you realize that you have a chance to be good at something that you continue to persist that's why you enjoy basketball because you think you're going pro um and that's why you give up on golf because you realize I'm not good at this and so I think that um to to have a system where you have a series of victories in there real victories you have to have real victories where you know you teach them something and point out the victories and not always put out the failures um I think that that's really important because I don't know people who persist at things that they're bad at or or they don't know if they do persist they don't know they're bad at it so they continue cooking they don't know they're bad at it I actually think that non-cognitive skills that this is a really important area for us to think about going forward um if you reflect on your own life and you think about people who you admire people who think you're very successful you've got to grow up to be like you know they're people grit by grit I mean you know they can get up after a punch kind of like that I don't know what the word is for that but that's a thing can you get up after a punch life's about punches people are punching all the time um and um I'm told that there exist racial differences in these gaps but in these attainment these non-cognitive measures but I also am told that we have no sense whatsoever about how the gaps are produced the these skills are produced it's not inconceivable to me that something about black material disadvantage could increase non-cognitive skills you see what I'm saying it's conceivable that women confronting an environment of gender discrimination might become stronger as a result I don't know whether that's true or false but my point is we have to think more carefully about the production of non-cognitive skills if you know too little about before we start messing around with them it leaves aside the point the excellent point made here about the fact that people will only persist or tend to persist either when they're good or when they don't know how terrible they are everybody for coming I want to thank our panelists for really very thoughtful and thought provoking comments there was a reception in the great hall which you're all invited to attend and you can interact with our panelists there and thank you all very much that was fun