 performance assessments support deeper learning and equity. My name is Finn and I'll be in the background answering any WebEx technical questions. If you experience technical difficulties during this WebEx session, please dial 1-866-779-3239. Please note that as an attendee, you are part of a larger audience, however, due to privacy rights, we have chosen not to display the number or list of attendees to everyone on the call today. As a reminder, today's call is being recorded. We will be holding a Q&A at the conclusion of today's presentation. If you would like to ask a question, you may do so at any time using the chat box located on the right-hand side of your screen. Please type your question into the text field and click Send. Please keep the Send to defaulted to all participants so that the panelists may see the questions. With that, we invite you to sit back, relax, and enjoy today's presentation. I would now like to introduce your first speaker for today, Ronita Guha, Senior Researcher of the Learning Policy Institute. Ronita, you now have the floor. Good afternoon. I'd like to welcome you to this one-hour webinar hosted by the Learning Policy Institute. I want to let you know that the webinar is open to the public and is being recorded. The recording will be mailed to you in a few days and available at the link just shared in the chat. We'd also like to announce that we'll be holding our next webinar, Leading the Way, How States Are Using Deeper Learning Assessments. That webinar will be held on May 17th at 11 a.m. Pacific. You can register and find more information about that webinar and the webinar series by visiting the Learning Policy Institute's website or using the link pasted in the chat box. We'll begin with a presentation by Ann Cook, Executive Director and Co-Founder, New York Performance Standards Consortium. She'll be discussing the performance assessment system used by schools in the consortium. We'll also hear from Young-Lun Choi, Manager of Performance Assessments for the Oakland Unified School District. She'll be talking about the district's efforts to take performance assessments to scale. And Paul Leather, Director for State and Local Partnership, Center for Innovation and Education. She'll be talking about state-level efforts to enhance performance assessments. And finally, Deb Delisle, Executive Director and CEO of ASCD. And she'll be talking about what her organization is learning from practitioners about the value of performance assessments in their work. At the end of the line to respond to questions from the audience. We encourage you to submit your questions throughout the presentation in the chat box. And again, make sure to select all participants from the dropdown. Before I turn the webinar over to Ann for her presentation, let me briefly introduce her. Ann is the Executive Director and Co-Founder of the New York Performance Standards Consortium and was also the Co-Founder and Co-Director of the Urban Academy Laboratory High School, a New York City public school. She's taught both high school and college level students in courses such as children's literature, the history of the civil rights movement, and don't read that book, of course on censorship. She's the author of three series of children's books, including the Monster Series. I'll now turn the webinar over to Ann. Hi, everyone. As the New York Performance Standards Consortium has developed a proven practitioner developed student focused performance assessment system for its 38 member schools that are in New York City and state. Its validity has been established by the New York State Education Department and the New York Board of Regents starting in 1998 and reaffirmed who variances most recently in 2017. Its main components are it's a practitioner designed in student focused assessment tasks. It has external evaluators for written and oral student work, moderation studies to establish reliability, extensive professional development, and predictive validity based on graduates' college success. There are other components that include an emphasis on inquiry based teaching and learning and discussion based classrooms. So the consortium schools have taken this and developed a culture that's focused on deeper learning skills. They're freed from the pressures to teach to the test so that the consortium teachers are able to develop a multi-layered student focused curriculum in addition to and beyond the assessment tasks. The tasks that are up there include English and analytic literary essay, social studies research paper, an original science experiment, and an application of higher level mathematics. Those are required of all the schools in the consortium. And then individual schools will add on additional tasks, creative arts, art criticism, internships, foreign language. So that schools will have those in addition but everyone uses three, four that are up there as the required tasks. The tasks are there are multiple ways for students to express learning and exhibit their learning, their writing, their literary essays they can use, research papers, literary essays. There's oral presentations, all of the components, all of the tasks have oral components. There's discussions, debate, poetry reading, dramatic presentations. And also in the creative arts area of course in the performing arts it would be artistic renderings, performance, sculpture, painting, photography, and so on. So those are the kind of bare bones and people always want to know what it looks like. And what we would like to do is show you a clip of a film of one student. The work grows out of classroom work. Students will have had a number of hands on science classes and what you'll see in this film is one student's journey through preparing over a course of time the work that he then presented for his final performance assessment. And the very end of the clip you'll see the oral defense. So he's going to take you through the process that he experienced as he was developing his science performance based assessment tasks. The experiment I presented today was how does pitch affect loudness? I was studying how different subjects perceive the different levels of pitch. I got this idea from my sound physics class and we were kind of going around talking about instruments and the physics of the sound from the instruments and I was messing with the guitar and I remember that started to make me think of pitch. And so I think he came in with interesting questions and in the sound physics class where he started developing the project he you know he picked out a good question fairly early on and then could work on devising the experiment to carry that out. My experiment is that I am testing out sound. Okay and I'm going to ask you I'm going to play two notes for you. All right and I'm going to ask you which note you think is louder. And your only options are the first note is louder or the second note is louder. He read in a book that we looked at in the class that higher pitched sounds tend to be perceived as louder by people and he wanted to test whether that was actually a case. So he recorded several sounds from electronic keyboard at different pitches. I would play a higher note for them and then a lower note and see which one they thought was louder purely based on the pitch of the note. The courage that I got in previous classes I always felt like I was more welcome to kind of come up with whatever idea I wanted and to take that seriously and to turn that into a scientific question. And I mean that's really where my idea came from you know I didn't it was like there was no question that couldn't be tested almost and I knew that from the previous classes that I have done. Teachers always allowed us to you know answer whatever question we wanted answering our own way try to answer it whatever way we wanted so that kind of made me way more confident going into this experiment. It is their experiment it's their work it's their question and it's not ours. And you don't get that same sense of being part of a community that sense of being a scientist if you're doing somebody else's question. It's for a survival instinct and so that made me think of all the times that I feel like I just felt like people are more attracted to high pitch sounds than low pitch sounds. Particularly you know when I think of like the sirens sirens are very high pitch as opposed to low pitch car screeches are high pitch and people turn when they hear them. And so I just combined with you know his what he said and my real life knowledge that kind of made my happy business. So I think some of the things he learned along the way were the importance of doing preliminary work can really help you learn things you can't anticipate. I've probably been working on this whole project for maybe a year never thought I'd be able to type something that long but as I worked on it and had more ideas to put in it just kind of kept on getting longer and longer. And a lot of that did have to do with the fact that I was just gaining more knowledge over the past year and kind of I feel like I'm really able to talk about what I've learned. He finished collecting the data and finished writing the paper and prepared for his defense. There's a rubric that's used to look at the work both the written work and the oral work is on the screen now. You can access all of this to film the full film and all of the rubrics are on the website. If you go to the website at www.performanceassessment.org you'll see the rubrics across the different disciplines and you'll also be able to access the film and there are a lot of examples of student work across the disciplines. So you'll get a better picture of what it really involves to do the performance assessment. I would say this is a system it's not in one subject. It's used for graduation. The students have to pass all four of the required tasks as well as the ones that the schools add on in order to be awarded a Regents Diploma by the state of New York. So they go on to college. We have a trial program with a pilot program with CUNY. These kids are doing very well. We've been doing this since 1998 and we've had a lot of success with students when they go on to college. They all come back and talk about the fact that they really know how to write. They've had so much practice. They know how to speak in class. They have a lot of the problems that freshmen have coming out of our schools. It doesn't seem to affect these kids. They have a lot of academic skills as well as ability to know how to ask for help and know where to go to get help and know how to use it. So I think that what we've got is a system that has given them a real support and a real way of approaching learning. And we've gotten, had successes across the student body. If you, at the end, we can look at some of the, at the end of this webinar, we can look at some of the results. Thank you. For your presentation. And just a reminder that in the chat box, we did include a link to the resources and to the video from the New York Performance Standards Consortium. Also just a reminder that if you wanna ask any questions or engage in discussion, please do use that chat box and select all participants from the dropdown. Now I'd like to introduce our next speaker. Yang Wanchoi is the performance assessment manager for the Oakland Unified School District. Yang Wuan has been a public school teacher in New York City, Providence, Rhode Island and Oakland, California, during which time he has developed expertise in project-based learning, curriculum design and school-based internships. As the manager of performance assessments, he leads the Ethnic Studies Program and supports schools to provide high-quality instruction through a performance assessment system aligned to a rigorous and meaningful capstone project. And I'm gonna turn it over to you, Yang Wuan. Okay, it's great to be here. And I just wanted to say that 20 years ago, I was a first-year teacher in that same building that we were watching the video of upstairs from Urban Academy at a small school called Vanguard High School, which was also part of the consortium. And so so much of my teaching has been influenced by the work that Ann and others have done. And it's exciting to be in a position now in the district here in Oakland where I'm supporting the growth and performance assessment systems from a district perspective, as well as at our school sites. So I want to take you back to Oakland, maybe five or six years ago, where we had a graduation requirement that was a senior project, and it was articulated in the board policy in very vague terms. It was a student had to create a oral presentation or do a research paper that was a demonstration of key standards. And with that, schools were left to go off and design these senior projects. And what we ended up with was something that one could describe through these two pictures in the slide. These are two different Lego projects, clearly, but they are also very, very different in terms of their complexity, in terms of the rigor, the critical thinking, and the demand, the cognitive demands that would go into creating these Lego projects. And so this was very representative of what was happening in Oakland at that time, which was there were, in fact, things called senior projects, but what you looked at across school sites would vary greatly in terms of their complexity and quality. And the teachers and students at the time really were not happy about this. This was not fair for students to be held to different standards and they raised this as a real concern, which then led us to decide that, you know, we need to convene some teachers and think about what we should do next. And so I just want to point out here that our approach, I think, was really one of the key things about our approach that was different was that I think a district, as a bureaucracy, often has a response to problems to try to manage those problems centrally. And so one way that we could have responded to this situation was to build Lego tract housing and essentially require every school to do exactly the same thing. But instead of that, we said, okay, well, let's bring the teachers together and let's listen to them and help them and support them to adjust this problem. So what that led us to do was engage in a process with the teachers where we looked at our graduate profile, which you can now see, and it has six different attributes. And this was, you know, in some ways, too overwhelming. So the teacher said, okay, well, where can we focus? And he decided to focus on three different areas. One was academic proficiency, another was civic engagement, and then the third area that they wanted to focus on was essential communication. And they said, okay, well, can we design a project that aligns to these three outcomes? And even these are really, really big. So they said, well, what's the slice within each of these that we can effectively support students through? And so we landed on a research paper to be aligned to the academic proficiency. We landed on a field research experience where students have to do an interview or a survey or focus group in their community, and that's their civic engagement and then doing an oral presentation in front of a public audience. And that's their way of demonstrating essential communication. And again, rather than trying to create exactly the same project everywhere, we decided that instead, one way that we could approach this was to focus on standards of quality. And so our work really became about identifying and adopting rubrics that school sites could agree on that would be measures of quality for research writing, for field research, and for oral presentation. And so fast forward six years. It's 2018 and where are we now? We started off in that first pilot year with four high schools and six pathways within the high school. So pathways are smaller learning communities within the school and so six of those pathways piloted. And now if you look at the schools using rubrics, we're at nine high schools and they represent 24 pathways within our districts. And then there's six schools that are not using those rubrics. And then the number next to each of the schools represent the current senior enrollment at each of those school sites. So when we tally all the numbers, we're looking at 1,440 seniors who are going to be assessed using these common rubrics and 643 seniors who won't be. And that's 69% of our graduating seniors from the class of 2018. And so we're talking about roughly seven out of 10 of our seniors who are being assessed by a set of common rubrics that were designed by teachers, that were adopted by teachers, and that this process really was led with teachers at the center and that it was really a grassroots campaign that allowed for the spread of this work. And when we think about central mandates, to get even 70% of people to do something when the district says everybody has to do this, that's a pretty hard thing to see happen. And we were able to get to this number without a mandate and without any of the kind of so it will that sometimes comes from central office decisions. And now we feel like we're at a place where we can adopt a graduation policy. And since we already have so much investment on the ground level that it will help us tip the last 30% of our schools towards participating. And so we're excited this year to be working toward the board policy where we'll be including the use of these graduate capstone rubrics as part of the expectation. The final point I'd like to make is that while we did focus a lot on developing the rubrics, but that was never the end goal that the point of the rubrics was for our teachers to have common language to be able to discuss student work to engage in cycles of inquiry and to be together in a professional learning community and that all three of those components, the common rubrics, the cycle of inquiry to our process and the professional learning community were all aimed at transforming instruction and that ultimately the end goal of the performance assessment system isn't about creating yet another perfect standardization of what it is that we expect students to do, but it's really about giving us an opportunity to push for instructional change that's going to be really meaningful and relevant for students. Great, thank you, Young Wan, for your presentation. If anybody has questions or wants to engage in discussion, I encourage you to use the chat box on the right of your screen and remember to select all participants from the dropdown. Now I'd like to introduce our next presenter, Paul Leather. He's the Director of State and Local Partnerships at the Center for Innovation and Education. Background and Experience in Education, Counseling and Administration in New Hampshire stands for decades. He served as the Deputy Commissioner of the Department of Education in New Hampshire for eight years and for 18 years served as the Department Director of the Division of Career Technology and Adult Learning. In 1997, as part of the New Hampshire School to Career effort, Mr. Leather began the journey to create a state model for competency-based student transcripts. This effort resulted in the development and implementation of the New Hampshire competency-based assessment system and ultimately to the student mastery model now in place as part of New Hampshire's School of Proval Standards. More recently, he has led the development of a first in the nation next generation educational accountability model called Performance Assessment of Competency Education or PACE, approved as a pilot program with four New Hampshire districts in March 2015. Go ahead, Paul, and take over. Good afternoon, everyone. In my current work, where I now at the Center for Innovation and Education in the University of Kentucky with Gene Wilhoide and Linda Pittenger and others, one thing has become really clear advancing performance assessment at the state level is a little more complicated than I thought after working on it in New Hampshire for several decades. So to help work through some of the complexity, I thought I would address three things before I get down to business discussing where some of our states are nationally. And so I wanted to show this first slide, which is a summary of key performance assessment formats where we talk about performance assessment formats. This is taken from a paper by Scott Marion and Kate Buckley. We see several kinds of formats the first where there are discrete performance tasks, often developed by teachers and where they are more or less embedded in instruction and in courses. The performance-based assessment task model used by the New York Performance Assessment Consortium that Anne described and then showed in her video is an excellent example of this. And the performance assessment of competency education or PACE in New Hampshire also uses this a very similar model. And they're characterized by being embedded in curriculum and instruction in most cases. And that the curriculum and instruction is really driven by the guiding questions found in the tasks as young Lon talked about. A science experiment where the student actually designs, conducts and reports on the findings is a good example of this kind of extended performance assessment. Often performance tasks are connected to exhibitions as Anne's video showed. And portfolios and exhibitions are also often found together where students are rolling up products and demonstrations of learning over time to demonstrate knowledge and mastery as well as growth of learning. And the student describes their learning in an exhibition. Envision schools in California is often recognized as a national model for these kinds of performance assessments where they're woven together and we hear the Envision folks talk about portfolio defense where a student stands and delivers around their learning based on their portfolio of work often used for summative purposes for a student as they move on including student graduation from high school. Schools and systems involved in profile or portrait of a graduate often use this model. These are there are some examples of course a capstone project in Fairfax County, Virginia may look different from a portfolio defense as an Envision high school but the models are similar. I wanted to point this out though that there are these different models because often when we talk about performance assessment we talk about it as if they're all the same even though they are often quite different from one place to another. Another dimension worth noting is how schools that use performance assessments are organized. This is often defined at the state level either through leadership by the SEA or through legislation. We've seen roughly three levels of organizations some emerging from the field through networks of like-minded leaders as we see in Maine, Michigan and with charter organizations like Summit Envision and others and of course the some will work by Ann Cook and folks in New York City. This first level is often referred to as the innovation network whether it is set up by the state private foundations or like-minded educators are all three and is characterized as putting in place systems of performance assessment for the purpose of advancing teaching and learning. Well-known examples where you see this established in state statute are Kentucky and Illinois as well as a number of other states across the country. CPAC in California the California Performance Assessment Consortium is widely recognized as a network of schools and large and small districts across the state affecting thousands of students. The next level if you will is what I am calling pilot networks. Really the only difference between an innovation network and a pilot network is that the pilot network has been set up through policy as a precursor to going statewide. The current competition in section 1204 and the Every Student Succeeds Act was taken from the New Hampshire PACE pilot. The intent here is to over time put such a system in place statewide if and that is a big if the system proves itself to be value-added, sustainable and scalable. Lastly there is the full scale implementation. States like Virginia and Colorado by having state policy that either allows for or encourages the use of performance assessment statewide with a new system are modern day examples of this model. Many policy makers worry about going right to full scale implementation statewide as a state's ability to do it well is really defined by how prepared educators and school leaders are in putting it into place both in terms of will, skill and capacity. Early efforts in the 80s and 90s encountered many problems due to these issues and educators have long memories even though many of the technical concerns have been addressed since that time. The third idea that I just wanted to quickly touch on is the idea of a system of assessments in this case a comprehensive system of assessments aimed at supporting not just local formative and summative student level work but also state level accountability. One of the best known models of such a comprehensive system is referred to as the 51st state model after a paper by Linda Darling-Hammond, Gene Wilhoit and Linda Piddinger in 2014. The local assessments are conceptualized as a part of a larger system of state assessments and grade spans and other quality assurance methods all rolled into one system. The New Hampshire PACE system was designed after this particular model and some states that are thinking of using performance assessment as part of accountability are looking at this kind of an approach. Now that I have the dimensions put out there for your consideration I wanted to share with you some observations of four states. This slide was actually prepared by two grant managers at the Hewitt Foundation, Dennis Udall and Chris Shearer that formed the analysis that led to my current project the four state performance assessment network. What you see here is a summary of some of the key drivers, partners and potential opportunities in the four states of California, Colorado, New Hampshire and Virginia. Each state system evolved from very different places modeling graduation portfolios to replace the California state assessment for example versus a menu of assessment options in Colorado. There are similarities however of course as well. Each state has been looking at performance assessments as alternatives to the now traditional standard space top down, large scale state assessment model largely dominating the country. Each state by moving to performance assessment is looking to close the gap between curriculum agnostic state level assessments and the which information gleaned through curriculum embedded assessments in classrooms locally. In all four cases the states and local partners are very interested not just in going deeper but also broadening their way of understanding about student learning from purely academic knowledge to better understanding student skills and using their knowledge to solve complex often authentic problems. Dennis and Chris also really saw the development of the engagement in performance assessment in terms of the need to build systems. Systems that have much utility and support in the classroom and in the school is at the state agency and the state house. They saw rightly that to put such a system in place requires the involvement and development of teacher and leader capacity in the field as well as tech solutions and validity reliability and in some instances comparability studies up and down the system. They saw these systems as evolutionary as much as they are transformational. At the center for innovation and education at the University of Kentucky we see this as a spiral of inquiry if you will not as a closed loop. One where educators at both the state and local levels are involved in a culture of learning and complex transformative change where they are examining the conditions necessary to create that change where they are identifying the key dimensions entry points and steps and processes necessary to accomplish change as well as the critical policy and practices that need to be in place. We believe that it will only be when student learning and growth and greater educator capacity become the central and essential goals overall that we will see the system actually change to support performance assessments and other ways to see teaching and learning and thus finally serve the need of preparing our students for the futures facing them in coming years. Lastly I'd just like to share this last thought. I would remiss with our next speaker if I did not mention that it took New Hampshire three years of teacher preparation design and development to come to a place of being able to transform our system and that was with the redoubtable Deb Delisle serving as assistant secretary of education at the U.S. Department of Education. Even with her encouragement understanding support and leadership it took us three years to construct a system like I have described. As our folks in New Hampshire like to refer to it it may be hard work it may be the hardest work we ever have done in this field but it is also the right work and we would not have it any other way. Like I'll stop there Anita and hand it over to you in depth. Great. Thank you so much Paul for your presentation. If folks have questions for any of our presenters please do type them into the chat box and select all participants. We do have time at the end for a question and answer. Now I'd like to introduce our other and final distinguished panelists Deb Delisle. She's the executive director and CEO of ASCD. Her 40-year career in education over her 40-year career in education she has served as a teacher, gifted education specialist, curriculum director, elementary school principal, district associate superintendent, superintendent, state superintendent and university instructor. She served as the U.S. Assistant Secretary of Elementary and Secondary Education from 2012 to 2015. As the Assistant Secretary of Education she played a pivotal role in policy and management and she was affecting pre-K, elementary and secondary education for the U.S. Department of Education. Prior to this role she was a senior fellow at the International Center for Leadership and Education with a deep interest in educator performance system and creating transformative cultures in schools and districts to support educators, students and their families. From 2008 to 2011 Deb served as Ohio's 35th state superintendent of public instruction and prior to that she was a superintendent of the Cleveland Heights University Heights City School District. I'll now turn the webinar over to Deb. Thank you so much and I'm going to apologize to everybody for my voice and suffering from what everybody else has dreaded across the country. I'm waiting for spring to come. Thank you so much. It's really an honor for me to be the closer here before we get into a conversation and I want to thank everyone for their really good thoughts and I'm hoping that folks on the webinar is understanding of the complexity of this work but as Paul described recently it is the best work and it's the right work to do for kids. I also want to thank Paul and acknowledge his work because much of what I believe in what we're striving for at AOCD lies in the work that we did together in New Hampshire. But I think his which should not go unnoticed is that three-year time span he talked about. So if in essence if there weren't enough people at the U.S. Department of Education ready to embrace it and it took three years to get that done you could imagine how complex it is in local school districts as well as across the state. So I don't want to underscore that but I also don't want to steer people away from trying it just because it's hard because it is the right work to do. So what I really want to talk about is on a broader spectrum kind of lifting up all of the conversation that was mentioned earlier in the exemplary work that's being done at the local level with the great information that was shared. So you see the mission of AACD on here and in Essman's performance tasks really speak to the how of our whole child framework. We have been in existence for 75 years. Our primary purpose is and goal is to enhance the craft of educators no matter their role in the system. So we're the only membership organization that actually has a place for everyone from student teaching up through retirement. We've been at it for 75 years but our whole child framework has been in existence for the past 10 and you can see what the primary goal is at the end of that mission statement. So that every child is healthy, safe, engaged, supported and challenged. At AACD because of this whole child framework we have a keen interest across the world in engaging all of our students in relevant and meaningful learning activities because we firmly believe that performance tasks seek directly to the social and emotional safety of students as well as the support of their aspirations and they become challenged and they're able to compete not just with themselves but with others as they see student work being shared through networks and other examples. You can see here the depth of our membership and I highlight this because we have members across 127 countries that 100 close to 115,000 members and we have 56 affiliates. I share this to show you both on the left side the cross section of individuals we have who are members of our organization and what's very interesting to me I came to AACD two and a half years ago is that I have not found anyone who has a difference of opinion across typologies of school districts or states or other countries and while we have affiliates in countries as different as China and Ghana their commonality across all of those and across our state is that everybody wants to do what's right for their students and so most of our members are striving to figure out how do you do that in an environment particularly in the United States which seems to have a very great love attraction with summative assessment. Excuse me so this is cut down to say that equity matters it's one of our areas of focus and a prime driver of our work. What you can see here excuse me I'll take a drink of water what you can see here on performance assessments is this is our work this is actually why we have focused on performance and support performance assessments. AACD has long prioritized multi-factor assessments our authors and researchers have readily shared their importance and members recognize the performance assessments impact on learning. We have a desire to make learning and teaching relevant not just I'm so sorry not just to the lives that kids lead and teachers lead in the classroom but also those that they lead outside of the classroom. We have a belief that what we offer to our students tells them what it is that we value. We recognize that personalized learning can be achieved through performance tasks and assessments and most importantly we recognize that the time is now we have to get passes over the lines on one type of assessment and if we don't do it meaning we inclusive in education who will do it. What you'll see on this slide is that these are the lifting up I've done both through focus groups as well as the comments we hear in our conferences what our researchers are telling us if you scan down that list what I want to reassure you is that these concerns are surfaced across all kinds of typologies across socioeconomic groupings of students in schools cultural and specific groups and this is what we try to push really hard through our professional learning is that schools and districts educators want a common research based definition of performance assessment perhaps it'll differ in some semantics from district to district but in each school people are looking for district leaders that come together and form a definition so that they can remain true to it and have fidelity to it there's a recognition that collaboration is required to develop and implement performance assessment they want to know how to communicate this with parents and students this is especially true if districts and schools are willing to have students progress at various levels of achievement where competency is attained rather than being held to a large cohort of kids enrolled in a class or in a grade level to all move together there's a recognition that this serves our our most vulnerable students our challenged students as well as our students who are gifted they don't want to be bound by one entire group of students in a class level of achievement but how do you explain that to parents how do you explain it to kids because so often parents will come in and say is that me is that a me right there's a need for time what it would look like to be coached and effective of limitation in youth and how do you align with the philosophy of teaching and learning with standards in the school what are developmentally appropriate tasks and assessments most often we hear from teachers they don't want to watering down they want to lift up standards for kids and provide support with high goals for all kids and with the support needed to reach those goals and the most especially how do you have fidelity across a school or a district so that a teacher in one grade six is not different than a teacher in another grade meaning they have different standards for kids and that's one thing that Paul and his team in New Hampshire really did a wonderful job at so all in all our educators our members just want one thing they want us to be able to help them to be better at their craft because they totally believe that they need to be the best that they can be for their kids so it's really all about professional learning and allowing educators the time and space to deal with performance assessments both in terms of an understanding as well as a policy and a procedure to get to the performance assessments to be adopted within the system so I'll end there and I really apologize for my voice great thank you so much Deb now I'd like to begin our discussion and address some of the questions we've received from the audience and the first question is actually directed to Ann Cook and Ann the question is whether you could speak to the demographics of the students who attend schools in the New York standards performance standards consortium and I'd love for you to talk about that as well as how you're ensuring that all students have access especially those who are for this from opportunity so we address the equity question and then maybe also talk about the evidence of the impact that you all have collected over the years and you might be muted yeah someone has chart one yeah so this is a comparison of consortium schools which we are all public schools not charters in the city we're all public schools some of them are 6 to 12 some of them 9 to 12 then this shows a comparison of the consortium with the New York City public high school data that's a data derived from the city school system so you can see that we have who's in the schools you could see that we have equal numbers of kids in poverty we have more L kids then in the consortium then the percentage if you work out the figures then in the high schools if you go down and you look at the average eighth grade proficiency you'll see that our kids enter eighth grade ninth grade based on the eighth grade statewide exams they're performing at a lower level than the city average but then if you look at the four-year graduation rate the six-year graduation rate and you go down that column there of African-American students Hispanic students L students the consortium students are outperforming the city similar population kids on almost every category I don't know I think it's Gretchen Wright who asked that question so this really addresses the issue of who's in our schools so the students in New York City come into high schools in a very very complicated way most of these kids are coming in a lot of them are neighborhood schools still there are some degrees of choice also where kids will rank order their selections but it's what we have in these schools among our 38 schools we have five that are L focused the kids in those schools have failed the NYSES LAT which is the English proficiency test and they have had to be in the country four years or less in order to get into those schools but you look at the L figures as you'll see that their the L graduation rate almost 30 points higher for the kids for those kids in the consortium and they would be in the city as a whole we have six transfer schools the kids are coming from other schools either they were overage and under credited or they were homeless or there are a number of categories that kids tend to leave the school they were assigned to originally and end up in a transfer school we have six schools that are transfer schools that are in the consortium so we have a very we have a high percentage some of our schools have 30 percent of our kids are our special ed kids so we we have I think it's clear from this chart a very representative group in fact probably more challenged than the city as a whole but the results if you look at the graduation rates if you look at the dropout rates and so on the results are much better than the city average great thank you Anne we have another question about the cost of performance based assessment and the question is is performance based assessment feasible from a cost standpoint and how is that addressed and I'm wondering Paul if maybe you could kick us off by responding to that question sure I think in my comment I did say but some of the technical barriers had been addressed cost of course is goes beyond that it's a kind of a systemic issue all around but some of it has been solved through through technology of course I think as we start to see platforms where performance tasks can be constructed asynchronously where where calibration of student scoring can be done asynchronously also on platforms then we start to see some of the the big costs of performance based assessment go down particularly from a system standpoint just by way of example for New Hampshire our accountability system we really didn't have a lot of money to be running a second system of accountability as we constructed pace using while the the large-scale state assessment system continue to go on and so we our accountability model cost approximately $300,000 for the last four years consistently as we went from four schools to 28 schools now the where the cost can can grow is is as you build deeper and more significant professional development and and looking at ways in which you build tasks together where you score tasks together and use that as your professional development you can get economies of scale across districts and with large groups of educators and that's what New Hampshire has done and that's what a lot of states that we're seeing whether they're doing innovation zones pilots or full-scale engagement as I talked about are doing it so the costs are differential depending on your model but I think there are ways to solve cost issues if you go about it in a smart way can I just add to that in terms of the consortium the consortium schools don't cost the system anymore and then the regular schools in fact there have been times when the when the consortium schools have actually been funded at a less less rate we had interim assessments that where the state the city was purchasing interim assessments from major commercial testing companies to prepare the kids for the region's exams those those costs worked out to somewhere around 14,000 per school when we went to them and said went to the department and said we're not giving those exams what we want to use and we want to develop a system of interim assessments they funded us at about 5,000 per school we did we did the work with that so we were being funded at a less less of an amount I think if they were to increase this across the state the thing that Paul is mentioning about the professional development is really the key and if you're talking about scaling up anything that's what needs to be scaled up the workshops the kinds of curriculum planning sessions the sessions we do moderation studies where we re-grade blind all of the papers that are sent in by schools that is a very valuable aspect of professional development that's generated from the bottom up not from the top down so I think a lot of it has to do with rethinking how we go about thinking about some of these costs we're certainly a lot cheaper than the tests this is Deb Delisle I just wanted to add and build on what Anne said and of course what Paul said because the reality is is that it's re-prioritizing resources but also what value people have so one of the things we've had to do at ACD is totally change a mindset that professional learning was only about sit and get or go somebody and hear a speaker and time and time again pushing 90% of our members always stress the importance of on-site job embedded professional learning and they want to learn from one another so as Anne was indicating it's really critical to provide those opportunities when we've been working with states and districts one of the things we find is that for some reason districts are more apt to provide substitutes so teachers can get ready to help one another prepare kids for it to have if you utilize that format and that time in a different kind of way you tell teachers that it's valuable for them to learn one another from one another in creating the performance tasks themselves coming from the U.S. Department of Ed I can say that not everyone uses all of their federal funds as thoughtfully as perhaps could be done so we hope to uncover some ways whether it's Title I funds or other kinds of of course Title II is now going away but there are ways to access federal funds in different kinds of ways when you're supporting the growth of all students Great. Thank you Deb. We have another question that came in and this one's to Anne the question is to what do you attribute the dramatically higher English learner in special education graduation rate versus other subgroups among the New York consortium schools? That's a very interesting question because we we thought a lot about that I think that that you have to start by thinking about what is it that is happening here what I think is going on when you have a performance-based assessment system really is changing it's not being driven by the tests the curriculum and the instruction are driving the assessment so if you're freed up from teaching to the test you're developing the kinds of courses that are engaging students at a very high level there tends to be much more inquiry-based teaching and learning you have a lot more discussion going on these are much more discussion-based classes there's very little chalk and talk and I think that what happens in these classes is that you just get kids using the language in these schools there is a real commitment because the performance assessment system means that you're developing a culture in the school since it's not just a class that's doing this it's the whole school it's all the subjects this is the way kids are assessed so every class kids are talking more they're engaged in writing more they're listening to teachers talk at them less and all of that contributes to kids learning the language that's how they're learning they're using the language and I think that has a lot to do with the way these kids are responding I've had them attend conferences with me kids who've been in the country four years three and four years you can't tell they were it's really embarrassing when you think our kids have taken three and four years of language and they can't really speak it these kids are speaking English extremely well that is the language of instruction and they're using that language all day long every day so even in schools where there are 45 different language groups the language that they're being instructed and that they're talking in this English there might be another student maybe who speaks Punjabi or some other language that they are that that is their native language but their instruction in the classroom is in English and they're immersed in it and they're using it and I think that is what's making the difference great thank you Anne so we have about a minute left on this webinar I want to thank Anne and young Juan and Colin Deb for a great discussion today I'd like to remind the audience that we are recording this webinar and we'll email you in a few days when it's available we'd also like to invite you to join us for the next webinar in the series leading the way how states are using deeper learning assessments again this will be on May 17th at 11 a.m. Pacific and as I mentioned at the beginning you can register and find more information about that webinar using the links links pasted in the chat box this webinar is part of a series achieving equity through deeper learning and we'll be having additional webinars throughout the year we'll send in notifications by email if you'd like to sign up for our email list and finally we'd like to share the following online resources which will also be posted on this webinar's page again thank you and have a good afternoon