 Hello everyone, thanks for joining us today for this one-hour webinar. Before we begin, I'd like to thank our district partners in Los Angeles, Oakland, and Pasadena for their support with this event. I'd also like to let the audience know that this webinar is being recorded. The video recording will be emailed to you in a few days, along with the slides from today, and those slides are also available at the link in the chat box. As people are logging on, I'd like to encourage you to introduce yourself by clicking on the chat button and sharing your name, where you're from, and your position or job title if you'd like to share that. You can find that chat at the bottom of your screen and do please feel free to introduce yourself. And I do see introduction coming in, which is wonderful. It's great to see that we have folks joining us from all across the country. Throughout California and other states as well. Welcome to everybody who's joining us. My name is Anna Mer, and I'm a research analyst and policy advisor at the Learning Policy Institute, where I co-lead the whole child education team. As part of my work on this team, I oversee the California Performance Assessment Collaborative, which I will introduce in just a moment. First, I'd like to share our agenda for our time together, and I'm continuing to see many introductions coming in from across the country with current colleagues and many others as well. And it's wonderful to see you all joining the chat. Thank you. Today, we'll be sharing some research and practice insights on how district leaders in California have used Performance Assessment to support student learning. This webinar will provide useful context for educators and policymakers who are interested in implementing or supporting performance assessment. I'm going to start with a brief overview of new research from the Learning Policy Institute, which documented the culminating performance assessment initiatives in Los Angeles, Oakland, and Pasadena Unified School District. Then I'm going to moderate a discussion with the district leaders who partner with us on this research. Young Lan Shoy will represent Oakland Unified in this conversation. After Solomon will represent Los Angeles Unified, and Christina Turley will represent Pasadena Unified. We're then going to expand our conversation to address audience questions. As a reminder, if you have any questions, feel free to click the Q&A button at the bottom of your screen. If you'd like to engage in the discussion, go ahead and click the chat button and type in the chat box at the lower right side of your screen. And I'm continuing to see a lot of introductions coming in from both California and states across the country, which is exciting to see. So before diving into the research, I would like to take a moment to define what we mean by performance assessment. And we're going to start with a quick poll to see how familiar the audience is with this term. So you should see that poll popping up on your screen. I'll give you a few moments to read and respond. As a reminder, if you select other for your response, please share what that means for you in the chat. I'm seeing a lot of responses coming in, which is great. We'll give it just a few more seconds and then we'll look at the results together. We'll close the poll in just a few seconds. Okay, let's go ahead and take a look at the results. So it looks like the majority of our audience has a good understanding of the term performance assessment and almost somewhere between a quarter, around a quarter, a little more than a quarter are engaged with directly with implementing performance assessment. The highest response category was, I have a solid understanding but I'm not directly engaged. And we also have some folks who've heard of the term, but I'm not sure exactly how to define it. So that's really helpful context to know as we dive into the presentation and discussion. Thank you, everybody. So, at the Light Policy Institute, we define performance assessment as a strategy that can require students to show what they know by performing, creating or producing something with real-world applications. So, all the quality performance assessments are designed to surface students' facility with core modes of inquiry in the discipline. This can include scientific investigation, mathematical modeling, literary analysis, social scientific inquiry, and artistic performance. Performance assessments can be thought of as existing along a continuum of complexity with different types of performance assessments serving different purposes. The other end of the continuum are short-term performance tasks such as composing a few sentences in an open-ended short response or developing a thorough analysis in an essay. These tasks enable students to demonstrate applied skills in a way that can be useful for comparable reporting and accountability purposes, as well as for diagnostic purposes. At the other end of the continuum are longer, deeper investigations such as culminating portfolios of student work and graduate capstone projects that are the focus of the research I'm presenting today. These performance assessments require students to design, conduct, analyze, provide, present, and reflect on their work. They can inform classroom instruction and enable students to demonstrate readiness for college and career. These types of performance assessments are examples of assessments of learning and also of assessments for learning and assessing what students know, also enabling them to do more, and assessment as learning, which is the process itself is a learning experience. The California Performance Assessment Collaborative, or CPAC for short, is a statewide network of districts and school networks seeking to expand the use of high quality and equitable performance assessment as a means of driving teaching and learning. You can see the link to our site in the chat. Since 2015, CPAC participants have come together regularly to exchange ideas and resources about how to implement performance assessment systems with the goal of influencing K-12 and higher education policies. We currently have membership throughout California, consisting of six school networks and 17 school districts, which represents hundreds of schools and thousands of students in our state. Through CPAC, the Learning Policy Institute has developed a research practice partnership that has enabled us to conduct in-depth case study research on the culminating performance assessment initiatives in three participating districts, Los Angeles, Oakland, and Pasadena. This newly launched research series, which you see pictures here, features a full report and a brief with our cross-district analysis, as well as individual case study reports for each district. Our research included interviews with district administrators and principals, teacher and student focus groups, observations with teacher professional learning sessions and student presentations, and review of administrative documents. We collected data from three school sites in each district. I do want to emphasize that this study documented the implementation of these performance assessment initiatives. It was not an impact evaluation. However, as you'll hear in just a moment, we did collect preliminary self-reported outcome data from students and teachers at a few schools in each district. I'll start with a high-level overview of the district's performance assessment initiatives that we've studied. Los Angeles Unified is a district with roughly 600,000 students, the second largest in the country, and has implemented performance assessments through link learning, which is a program of study that integrates a college preparatory curriculum with career and technical education and student support. And link learning pathways are organized around industry themes such as biomedical sciences. Students and link learning pathways curate a portfolio of their high school work, which they reflect on and present in a 12th grade defensive learning. In 2018-19, approximately 4,000 12th grade students participated in this process. Oakland Unified serves a little over 50,000 students and has a graduate capstone in place, which is a year-long original research paper and presentation that students complete in 12th grade. The district is also working on incorporating action projects aligned to students' link learning pathway themes into the graduate capstone. The capstone is an opt-in process that builds on a 2005 district-wide senior project graduation requirement. In 2018-19, Oakland teachers assessed over 1,000 seniors using common graduate capstone rubrics, while other Oakland seniors completed a senior project defined by their schools. Pasadena Unified serves just over 17,000 students and passed a district-wide graduation policy requiring all 12th grade students, starting with the class of 2019, to collect and reflect on a portfolio of their high school work, including a research paper, which they present in a senior defense. The Pasadena School Board approved this policy in 2014 as part of an effort to revise the district graduate profile. In 2018-19, all 1,067 graduating seniors in Pasadena completed a senior defense. As part of our research, we asked, what are the conditions that need to be in place at the classroom, school, and district levels in order to implement high-quality performance assessment? But first, what conditions matter for districts? We found that having foundational policies and practices in place to support the performance assessment initiative, such as board-approved graduation requirements or graduate profiles, as you just heard about in Pasadena and Oakland, as well as structured onboarding processes to join initiatives that require performance assessment, as is the case in Los Angeles-Link learning pathways, are important for this work. Next, we identified a supportive state and local policy and practice environment and found that this played an important role. In California, this included a focus on deeper learning competencies, such as critical thinking and communication in the Common Core State Standards, as well as state and local investments in career and technical education, such as Link Learning, that encouraged innovative assessments of student learning. Finally, we observed several common key starting conditions across the district. These were access to technical assistance providers, opportunities to observe performance assessments in action, and having a strategy to develop and scale performance assessments that evolves organically in response to site-level needs. Next, we asked, what conditions matter for teachers? First, we found high-quality professional learning opportunities were important. In all three districts, central office leaders offered professional learning focused on performance assessment, including ongoing communities of practice in Los Angeles and Oakland. These sessions were often opt-in and focused on setting expectations for student work, as well as calibrating the scoring of student work. On-site coaching was another important form of support. We also identified the importance of supporting and recognizing strong teacher leadership. Not surprisingly, we observed that teachers were the key to successful implementation at school sites across all three districts. Teachers took on many roles from content support, such as designing instructional materials, to logistics management, such as scheduling student presentations and recruiting judges. In many cases, teachers received extra planning time and compensation to recognize this work, although more time and more compensation also emerged as areas of ongoing need in our studies. Lastly, we asked, what conditions matter for students? We found that instructional leaders, such as principals, coaches, and lead teachers were best positioned to adapt the performance assessment to the needs of their students in the school community and to ensure adequate support for students. Students benefited from access to mentorship, peer support, time to prepare for the culminating performance assessment in class or independently, or both, and exposure to curriculum to build relevant skills, such as writing research papers and giving presentations. As I mentioned earlier, we spoke to students and teachers at a few schools in each district, and we heard about a variety of benefits associated with their experience. Students described how they were able to demonstrate deeper learning competency, such as critical thinking and communication. They felt greater confidence in their college and career preparation, and they experienced growth in social-emotional skills, such as perseverance. Teachers shared how they were able to align curriculum instruction and assessment across subjects and grade levels, including faculty mapping their instruction from the culminating defense. They also shared how they were able to continuously reflect on and improve their instructional practice and how they were able to develop closer relationships with their students and more collaborative relationships with their fellow teachers. Across the three districts, we identified eight lessons learned along with accompanying recommendations for district leaders and state policy makers. I'm going to share a few highlights here. I'm not going to go through all eight, but I do encourage you to take a look at the research brief or the full report in order to learn about the full set of lessons and recommendations. I'll start with recommendations to support districts with implementing performance assessments, then I'll move on to recommendations to support teachers and students. First, we recommend that state policy makers consider opportunities to support innovative performance assessment initiatives in local districts. This can be achieved through educational standards, assessment and accountability approaches and funding opportunities. Next, we recommend that district leaders enact a performance assessment policy that balances teacher innovation with a shared district-wide vision and clear paths for scaling up. This requires balancing an often collaborative approach with centralized supports and eventual expectations for all students at school to participate. To support teachers, we recommend that district leaders develop an implementation strategy that includes strong support such as staff time for planning, coordinating and mentoring students, as well as professional learning and on-site coaching. To support students, we recommend that district leaders equitably allocate sufficient resources across academic programs, student demographic groups, including English learners and students with disabilities, and school sites to ensure that students have the support they need to successfully participate in the district performance assessment initiatives. As noted earlier, this support may include access to mentorship, peer support, time to prepare, and exposure to relevant curriculum. And with that, I would like to introduce our panelists, who are the real experts on the initiatives that I've just described. As I provide brief introductions, you can also read more about each panelist at the link that will be shared in the chat. I'd first like to welcome Yanwan Choi, manager of performance assessments in Oakland. Yanwan has been a public school teacher in New York City, Providence, Rhode Island, and Oakland. During this time, he has developed expertise in project-based learning, curriculum design, and culturally relevant teaching. Currently, he leads the Oakland Unified Graduate Capstone work, as well as the district's ethnic studies program. He also produces and hosts the Young in the World podcast, and I do recommend you give that a listen. I'd also like to welcome Esther Solomon, career technical education and link learning administrator in Los Angeles Unified. Esther has been an educator for the past 39 years, including five years as principal of the Los Angeles High School of the Arts, which was the first pilot school and the first link learning certified pathway in LA Unified. Currently, Esther oversees 422 career pathways of which 78 are linked learning. Her office is focused on revising the district graduate profile, deepening performance assessments, and expanding the portfolio and defense as a means of demonstrating proficiency on the graduate profile. And last, but certainly not least, I'd like to welcome Christina Turley, college and career coordinator for Pasadena Unified. Christina has spent her career working in the district as an elementary school teacher, a coach, a site administrator, and now a central office administrator. Her favorite part of her job is watching her former elementary school students become ambitious, thoughtful, and focused high schoolers. Christina has a passion for personal growth and considers herself a lifelong learner, having earned both her MA from Cal State LA and her doctorate from the University of Southern California in Education. So I am very excited to focus on our panelists and I want to make sure we can see and hear young one as well. So young one. Can you go ahead and turn on. Yes, we can hear you. It says I can't open up. Well, we can we can hear you. So let's jump in. You know, I'd like to acknowledge that implementing a district performance assessment initiative is challenging endeavor. It's not simple. It's not easy. So I'd really like to hear what drives each of you to do this work and why does it matter. And let's start with Esther. Thank you so much, Anna. At this point in my career, having started teaching in 1979, I do this work because exactly as Anna said I am driven. I've taught in public private schools, middle high college on the East Coast in the Midwest, but mostly in LA unified and I think the work that we're currently doing with link learning and performance assessment is the most meaningful purposeful work we can offer our young people. This work supports students finding their voice. It empowers them, and often our students are so self effacing and many don't have the privilege of growing up with parents who provide who are professionals and who provide a sense of confidence. And the many project based learning opportunities that link learning provides assist students in understanding that they do have a place in this world and that they can play an important role. Providing a specific example of this will demonstrate my point. One summer a young student was complaining that her father was constantly getting tickets for selling food on the streets. He was a food vendor and the faculty got together and created a performance assessment, a PBL. It was a digital media pathway so students created a video where they interviewed street vendors they interviewed legislators. They edited the film in their English class they wrote persuasive speeches in their government class they learned how do you change the law how do you advocate for a law. Why join a neighborhood community. So they learned the importance of this but with real application. They went to city hall they delivered the speech they delivered the video, and within a year and a half city hall changed the law. So those are the kinds of performance assessments that are really meaningful for our kids. Thank you it's really powerful kids to hear those that story, and one of I think for our delti team one of our favorite experiences just getting to attend student presentations and really seeing the power of learning and action. Christina what drives you to do this work. Be sure to unmute yourself. There we go. Can you hear me now. Yeah. It's very similar to what Esther said it's helping students make meaning of what they're learning so that that application of their knowledge that they gain through through content skills through their CTE pathways. That's super important. I, you know, learning in isolation to me is not true learning. If students have this knowledge, and they're not able to apply it. I think that is somewhat dangerous right we're not giving our students the types of skills are going to need in the real world. So this work is really exciting and I think for me in particular with students in our Pasadena public schools. For those of you who don't know who are around the country. And, you know, some of you in California may not know that we have the highest number of private schools per capita. Anywhere in the US. So for us in the public school setting it's super important for us to make sure our students have the skills to make them marketable to make them viable in the world economy. And I don't want them to just be able to perform with the private school students that they are competing against I want them to out do those students and to have skills that they can apply outside of our classroom. Awesome. And young one I'd love to hear from you as well. Right. Good afternoon it's first of all just great to be on this panel with my colleagues from Los Angeles and Pasadena as well as from lpi and, and most importantly to all of you are in the listening audience it's just so great to be in a space with people who have the ability to make such a significant impact on the lives of students. And I want to start by saying what may be obvious, but the system is broken. The system of standardized assessment the system of competition, the system that's based on individualism. That's a broken system and performance assessment. And the work that we're all trying to do is, is an attempt to build a more humane system it's an attempt to build a system that's based on collectivism. And it's, it's an attempt to address the fact that performance or that standardized testing is really intended to create an intellectual hierarchy for sorting purposes. And so just as a quick example like hypothetically let's just imagine that all students one year taking the SAT score to 1600 which would be the top score. Right. Would we celebrate and say wow we did such a great job preparing students they all scored at the at the top. I don't think so I think we look around and say well what's wrong with the SAT there's something broken about the test we have to fix the questions. Because the way that the standardized tests are designed is to create a bell curve of performance and there's always going to be a subset of students that are designated to be failures. When it comes to a standardized test and that subset historically has been our black and brown students. To me, performance assessment is a civil rights issue. It's an issue of racial justice performance assessments. I'm super passionate about because of all the reasons stated by the other panels as well it's an opportunity for authentic assessment of learning it's opportunity for growth and reflection. But ultimately for me performance assessment is about creating a more humane system and recognizing the humanity of all of our students. Thank you. Thank you for sharing those those words of wisdom. And I think that also really speaks to what I mentioned briefly earlier which is that assessment is not just have to be a judgment of what students know it can support instruction and student learning and also be a learning experience itself. And that's really powerful to see. So, I'd like to thank everyone who's submitting questions, we are going to put some audience questions into the conversation a bit later and just encouraging I know we saw in the poll that there's a lot of folks in the audience who also are engaged in implementing performance assessment so please share your thoughts and reactions as well in the chat. So, I crammed two years worth of research into about 15 minutes of talking on webinar, which was not easy. But we really did land around the key conditions for supporting district students and teachers is being some of the key takeaways. And so I'm really interested to hear as you were listening to that presentation and as you've been our partner in this research. What do you see as being examples of conditions that you find to be most important, or if you can share examples of the way that these conditions really look in the real world. And I'd like to direct the question to Christina first. For us, you know, the LPI research really served as a guidepost for where we want to go next. We're always in a state of continuous improvement. So this research really gave us a space to turn to and ideas to kind of point to your research is very narrative in nature as well as data filled. So it had a little bit of something for everyone to kind of latch on to and using that we, you know, we realized that we need to have a collective ownership of the work that we're doing with our senior defense. And as Anna described earlier, it is a graduation requirement for all of our seniors. So that involves all of our teachers, all of our staff at all of the sites. So that that is something we, we definitely need to push forward. And that's, that's a place that we are pushing our practice. Right now we have site coordinators and if you read the research, you can read a little bit about our site coordinators. But our site coordinators drive a lot of the work and we at the district level drive a lot of the work that's happening. And we are pushing ourselves to make the senior defense part of the culture at the school site that have the school site kind of take ownership of the work that's happening and create a culture in which students not only know they have to do a senior defense, but they're excited to do a senior defense and they know it's a celebration of their learning and they know that every staff member on their site is invested in the process with them. And they are encouraging them to be reflective learners throughout high school from, you know, their ninth grade year all the way to the last day of their senior year because, you know, that reflection pieces is is pivotal for this to be meaningful to our students. Thanks for sharing that Christina. And I think your point really underscores that it's helpful to have a pall of foundational policy in place but that's just the beginning of the work. Then it's really about looking at those key starting conditions and transferring the work in partnership to the school play. Yeah, that's true. And without that without that foundational piece we wouldn't have been able to move forward. Excellent. Sure. So the condition that I think resonates most strongly with me is the one around professional learning. And within professional learning. For me, it's not about an individualistic approach, but really a collectivist approach to how we think about professional learning it's not a sit and get with an expert at the front of the room it's really about bringing the wisdom of our teachers together so that they can problem solve. So the way that we've designed our professional learning is primarily into different setups one is a summer professional learning series which is usually three to five days and then we also have a school year professional learning, which happens quarterly. But in either case, the key foundation to that is that teachers are speaking to each other from across different campuses that the primary agenda is for them to have space to talk and discuss and problem solve together. So we, we don't, we of course bring things to kind of challenge and push their thinking, and we, and we encourage them to share practice. And so some of the things that we've seen emerge from the spaces is we've created a resource bank of some of the best practices of our teachers over the last eight years. And that's something that every new teacher to the capstone gets as part of their kind of induction process to the capstone. And so we have a process where my co facilitator and I Cameron Frederick will, you know, go and observe in classrooms and spend time understanding the conditions that our teachers are teaching and what their students need. We'll listen to them to find out what the teachers need. And then we bring something to the table and we model a lesson based on what we're seeing as some of the challenges in the classroom. Not as experts but just as a starting place for a conversation so teachers are struggling with like how do we support students to take notes in the digital context when we were all used to doing research paper on note cards. So we came up with something we put it out to them and said, we're going to we're going to go through this simulation and now let's talk about what would you do what worked about that what didn't and give them the bulk of the time to think and adapt those ideas for their own classroom space. So again, it's really, it's really about empowering the teachers or giving them the agency that they deserve to be able to make the instructional choices because at the end of the day, they're the ones who are going to be there with the students teaching and leading and so we're just trying to create a space for that collectivism. Thank you. And we definitely saw that face that collective safety created as we attended professional learning sessions. Esther, let me hear from you as well. I would say two important conditions one young one totally captured when he talked about the idea of having rich repositories of work. Websites for portfolio and defense and we have repositories of pbls and exemplars. We have practitioners centers so other pathways can go and observe at those practitioner centers. But I would say the other area exemplars are really, really important. The other area I would identify is also in our district. It's an opt in because as young one said the system is broken in terms of standardized assessments but it's also broken in the way that we treat our teachers. We are there inundated with requirements mandates bulletins memos. And so an opportunity for them to learn about something that they might want to become a part of is what we offer them so we have information sessions that we provide for them. Then we go to the school sites and we talk in more detail answer any questions they have. So it's an opportunity to opt in if 75% of the teachers are on board in the administration because you know we're providing money we're providing coaching we're providing work based learning instruction. And so we want people to be serious about this but if they decide to opt in and they come to the five onboarding sessions which are held on Saturdays. So if they come to the five onboarding sessions from January through May, then they can become a link learning pathway and get that kind of support with the coaching and the modeling and visiting other sites and talking to other folks and and learning from our team. So I would say those are the two big conditions. Thank you for sharing those thoughts. The question that we all reflected in it in some of what the audience submitted in advance and also that we just hear a lot in general is how do we support consistency and quality in this work, both in terms of the in terms of the consistent scoring of work that students produce and the consistency of student work. And so we would just love to hear how how each of you have approached supporting consistency and quality in this work. Thanks Anna. So this is a very important question. Because I think there is a push and a context that we're in that values the validity reliability of the data that comes out of performance assessment. And at the same time I think it's important for us not to forget like sort of what the fundamental purpose is of looking at student work, which is to be able to understand, you know, what do we expect of students and our students achieving what it is that we want them to and then how do we support them to get there. And students also to do that same reflection so, you know, fundamentally, when you're working with teachers are in a performance assessment system you have to look at student work, you have to create time for them to discuss it, and you have to have a common rubric to score it otherwise you're you're talking about completely different things. And so, in our experience in Oakland, you know, back in 2005 when we had our board policy, not a single school was using the same rubric for assessing our senior project. And even in 2012 when I went around and I did a listening campaign in our high schools, there was still not a single high school where, you know, students were using the same rubric as another high school. In the district it could be an easy answer to say well let's just mandate everybody to use the same rubric and then we're done with this problem. But really, you know, these kinds of technical fixes for these are like adaptive challenges and technical fixes don't really work and so we really needed to bring our teachers together, because they were the ones who understood the nature of the problem and they were the ones who were poised to fix it. And the critical point in that first year when we actually had a student come and give their presentation, and we scored the student on a rubric. And we had an amazing conversation about like what is it that we expect in terms of quality from our students across the district. And the fact that we were using the same rubric as a basis for that conversation really changed what we were able to say to one another and also what we were able to share in terms of resources and support. And the process that showed the teachers there was value in us using the same rubric and since that moment over the last eight years, we've gone from zero schools using the same rubric to 70 plus percent of our students being assessed on the same rubric for the capstone. And that was all built through the practice that the teachers engaged in, and it wasn't driven by the policy. I mean, to say there isn't a place for policy, I think we do need the policy. But, you know, there is a question of which comes first the practice of the policy. And in our case, we had a policy that didn't have good practice and then we developed good practice and now we're going to bring in a policy that hopefully will reflect the practice. So, I think it's, it's a journey right and you start where you are but ultimately it's about the work of the teachers to develop the practice. Yeah, I mean, maybe it's the, you've named the chicken and the egg question, I guess. And we need both. And I think we do have some quotes in the case studies about teachers just sharing how powerful that professional learning is for them of sitting down and really looking deeply at student work and talking about it. I'd like to open up space for Esther or Christina, if either of you would like to add thoughts on this topic. I think for us, for sure, it was the consistency in order to get the consistency, you know, what are we looking for? Are we looking for similar things? And because ours is district wide, those common rubrics were really important to the work we were doing. As young one alluded to that work is very hard work. Those rubrics were vetted with community groups and student groups and educators with various different partners, you know, Envision Buck Institute. So it took a good while to vet rubrics that were deemed worthy with educators and students alike. You know, students had a big say in our rubrics. It's a two point rubric because students advocated for that. So we definitely listened to student voice, you know, and you have to think about the iteration process of your rubrics if you're going to use common rubrics for all. So at a certain point, you just have to go for it and use your rubrics and hold off any comments or questions and then, you know, then open that back up. Another thing we utilize for our validity and common practice is we have a group that we bring together monthly in order for them to share best practices. And it's our site coordinators at all of our school sites along with an administrator. And they're able to share best practices and bring up questions that are happening at their school sites in particular. And that often answers questions for others or together collectively we're able to create a process or a document or someone's able to go to their site and and put it into action and then come back and let everyone else know how it went. So that that has been an asset to to that work. I would just briefly add we have more time for questions but I would just briefly add I'd want to thank scale and envision learning partners for helping us to validate those rubrics that we do have for our artifacts and also to say yes I think it's about in the beginning was just the process of like what is link learning how do we demonstrate link learning how do we develop link learning how do we help and support our pathways. We're developing the practices first and now we're coming to the point of like OK, we're reaching 70 some pathways. How do we provide an example to the district into the state that what we're doing can be validated that what we're doing can be measured in a way that's meaningful and purposeful. Absolutely. Thank you. Thank you. And it's time to start addressing some questions from the audience. So we've received a lot of interesting questions and comments from you all as I mentioned both in advance as people were registering for the webinar and also in the chat in the Q&A today. So we'll probably have time just to address a few of the questions but do keep in mind that we're going to be sharing contact information in the slide so that there is an opportunity to follow up if anyone has questions remaining after this conversation. So I think I'll start first with the issue of the moment, which is how has the pandemic been impacting the work that we're discussing. And I do want to note that in each of the district case studies and in the cross case report we did add an appendix giving a very brief overview of how each of the districts responded when schools shut down last spring. But I would like to give the panelists a chance to address how did you adapt to your performance assessment work last spring and how are you thinking about assessment, assessing student learning moving forward this year and what is currently a distance learning context and which may be a blended learning context at other points in the year. So I'll start with Christina and we can go from there. Sure. Yeah. It definitely brought about some like instant decisions that needed to be made. And like all of you initially we thought, oh, we'll be up for two weeks. And then it turned into, you know, what it is currently. So for us, we knew immediately that the senior defense was still priority for our students. So we quickly drafted something that the board approved and our CAO approved stating that number one, we weren't going to hold, we were going to hold students harmless as far as any COVID related anything that that happened in their lives. But beyond that, we still wanted our students to present so we created online ways for that to happen either with full panels. At the time we were using Google Meets, so a student can come into a Google Meet session with a full panel receive that Q&A portion and feed feedback in real time, or if that was not feasible for the students, they could do a screen castify where they presented alongside their slides, or they could video record, we had some students stand beside a computer and video record themselves and turn in that recording. We learned a lot through that process, for sure. And we've decided for this year that we'd like all of our students to have the full panels because there's a lot that you miss out on when you don't have the Q&A portion of the presentation, and students aren't able to maybe defend something they didn't necessarily bring up during the presentation. And the feedback is just such an important part of the process. That we definitely want students to experience that piece. I feel like it validates what our students are doing and what they're, what they're putting into the project. So that's how we foresee moving forward. Great. And Nuan, did you want to add anything about that? Sure. So, you know, it's interesting because, you know, earlier talking about standardization, right, standardization in and of itself is not always a bad thing, right, and actually our work around building investment in a common rubric is a form of standardization across our district. But what's happened in this particular moment is because the needs of each of our school sites at the high school level are really different and what our students need, you know, at a newcomer or high school continuation students who are older, like a lot of them are working, you know, jobs and so they're not able to access distance learning as much. So like a uniform district policy around our capstone, you know, hasn't been the direction that we've gone forward with. We are continuing to expect that everybody does it, but we're giving the sites a lot of flexibility on the way they define that. So even though I think we've been moving towards like standardization around the rubrics this year, we're taking kind of a step back and saying like, okay, every school sites got to figure this out based on the particular needs of their school community and we're there to guide them and support them and share what's happening in different places. And that being said, like the vast majority of our schools are still going forward with the research paper and oral presentation and just moving everything online. So, you know, that's, that's kind of where we are in Oakland. I would say we're kind of in the same place. Every pathway is making its own decisions about this. We're encouraging them to move forward with it. We had some pathways that did do a portfolio and defense at the end of the year, some that didn't. We have some, I think our, our link learnings team is working really hard to help teachers understand that they can still do PBLs easily in this situation. And, but there are just so many things happening across the district that we're trying to just be there to support them and, you know, and listen to what their needs are so we can support them. Make fun. Well, let's see. I think we got me a time for another question or two. One thing that I'm seeing brought up in the chat is the questions around equity and supporting students thinking particularly about differentiating support for students with disabilities, English learners, others with different needs. Um, so we just love to hear some thoughts about what it can look like to really address equity in this performance assessment work and differentiating support for students. Um, start with someone. Yeah, so for us, a lot of that work has come in the form of, you know, trying to stay grounded in the rubrics that we're using, but talking to people who, you know, might teach in a newcomer pathway or they might teach, you know, a high proportion of special needs students. Um, and asking them to think about like what kind of rubric modifications would need to be made in order for students to continue to demonstrate success as well as like what modifications do we need to make to the task. And so this work around revised tasks and revised rubric criteria is kind of new work for us. It's in the last couple years that we really begin to dive into it. Um, so we do have some schools that are beginning to pilot different things. Um, and for the most part what I'm seeing as most promising is it's not about creating a whole new rubric, but it's really just about modifying the rubric. You know, instead of it being, you know, that every student has to hit proficient that you know a modification might mean that they hit proficient in some of the categories. But you know, let's say in language use, if you're, if you're a language learner that that might be a category where hitting approaching would be fine. Um, so, you know, making the kinds of accommodations that make sense, given, you know, what the particular learning needs are of the community you're trying to serve while not, you know, completely diluting the experience or watering down the expectations for rigor. I think our pathways are adapting as needed, providing a lot of support. And what I find with the portfolio defense, it's really exciting when I go and watch students defend their work that the whole team of kids, the whole senior class is behind those kids and they're like, Okay, yeah, you didn't you didn't you didn't get a pass this time but we're going to go work with you and we're going to help you get past that. So that sense of students helping each other that sense of collaboration is really beautiful to see. I would also say that some of our pathways have allowed students to do for EL students to do it both in Spanish and English. And that's really exciting. You can see the grappling and the work on speaking the English portion and then when they get to the Spanish how fluent they are and how knowledgeable they are and it's exciting to give them the opportunity and to validate that they're bilingual. So those are some of the changes that we've made or some of the support systems we've put into place or schools have done that. Yes, Christine I'd love to hear from you if you could add to I do want to flag that. I know we've shared a link in the chat that the California performance assessment collaborative as a group actually came together and we drafted a desire guide around best practices for designing accessible performance assessment. So that's a great resource that we really came up with collectively. Yeah, and that resource actually drove us to look at our rubrics with a with an equity lens. And I just want to say that, you know, kind of going back to what young one spoke about at the beginning of of this panel discussion is these assessments are an opportunity for our students who don't traditionally do well on standardized tests to really show their talents. And that's what we've seen our special ed parents are like that they should get teachers made they are all for this type of assessment for their students. They are very strongly support this for their students and as far as our rubrics are concerned because we do have district right wide rubrics. We have taken a look at the rubrics with our special ed English teachers, and they've given input on. Really we open it up and ask them if they wanted to create their own rubric and modify it and they said no that the targets they expect their students to hit, and they will get them there. So that was a really empowering conversation and like Esther mentioned you know those are students in certain subgroups like they are cheerleaders for one another and they have probably a better understanding of what this of the meaning of these performance assessments than our students in traditional classes. So it's exciting work to do for students who may not be able to represent their talents in other way. I would just add one quick thing to that which is we did similar kind of outreach to some of our special special needs teachers and they looked at our rubrics and also said like you know our students can meet this but you know we need to change the language in these rubrics so that they don't start from deficit. At the beginning and then proficiency in advanced and so actually the way the feedback they gave us change the way that we the way the language of the rubric is written for all students. So that you know students can see like even if they're in the, you know, you know at the very beginning of, you know, their process they're still doing something right so the language reflects that they're doing something. They're missing something and then they're doing a little bit more as they approach and then they're and then they're meeting and then they're exceeding right so that actually was some some feedback that we got from engaging with our special needs teachers and it actually impacted the way that we assess all students in our system. I think we have time for one more question and I do want to share that there are some really, I think very powerful in the research reports where we actually describe what we saw as researchers. Seeing students present their work and there's also an element of cultural responsiveness in a lot of this work as well that I think is another facet of this conversation on equity that students are. And what we heard in the student focus groups is that students are being invited to bring their whole selves into the classroom with this work to address issues that are important to them, and to share elements of their what's important in their lives. So I really encourage folks to also check out the stories which we heard us to share at the beginning, but we can never fully capture that. We also do have some videos on the learning policies to website showing Oakland students presenting their work and I think those are from maybe 2017 2018 for their their couple of years old but they're still really relevant. And I'm sure we shared those links earlier, but maybe we can share those again in the chat, just to see what what is happening in action. But this my closing question would be what one minute each the panelists, what advice would you have for other educators, particularly district leaders who want to do the type of assessment work. And I'll start with Esther. So I would say, bringing stakeholders together, we're in the middle of revising our graduate profile right now and it's been really exciting to hear from industry and parents and students and administrators and teachers about thinking about what is it we want students to know what what role should they have what attributes should they have in order to demonstrate that they're ready to graduate. So I think that that's really exciting. The other thing I would say is when you start up something really different like this, because it is a huge shift for our teachers to make in terms of instruction that you provide opportunities for them that you, you understand they need time to do this. You set aside time for them to work together you set aside time. You say this is important you're not just giving them a once a week 15 minute meeting time, but that you really are devoting time and quality opportunities for them to come together. And that you're also providing exemplars. I think that's so important. So we have to see envision learning partners for their graduate defense and portfolio. It was, we took 12 people and they were all, it was like, okay, you don't have to tell us anymore, we want to do this. So seeing is really so much more important than just hearing about it. So we've, you know, we've developed videos we've developed repositories we've developed practitioner centers. These are really important ways in which you can bring people on board to want to make these kinds of changes because I think most people really understand that this is very different and they want to do something like this, instead of the typical work. Thank you. One minute from Christina. I want to echo what Esther said scene is believing that we took a group of teachers practitioners to to see some envision portfolio defenses, and then we created our own many defenses we grabbed a few students and from all areas in of learning. And we had a roadshow of students that we would take from site to site. So the idea that our students can do it was valued when the presentation happens. All right, I'm on close without me. So I would echo some of what's been said here and and say that, you know, if you're working in a district context, look for your existing practice. Where is this already happening where people already pushing the envelope where they already creating the kinds of things that you're excited about that that you can highlight internally because I do think to Christina's point. There's sometimes skepticism about what happens outside of a district, at least in Oakland it's always like well that that can happen here. The fact that it is happening here and happening in some of our most challenging context was part of the big sell for teachers to see like, oh wow this this really can happen here in Oakland. And then the other thing is just to really, you know, get back getting back to sort of my initial points here it's really about the purpose like what is the deep meaningful purpose behind why we do this work. And that isn't necessarily a top down vision but that has to be a collective vision has to come from listening to the teachers it has to come from from the bottom, so to speak, of you know where people are closest to the students, what is the value of this kind of system, and really being and leading with that vision leading with that sense of purpose, a purpose of being, you know, a civil rights issue a purpose of being about racial justice a purpose of about you know growth mindset a purpose of a really changing the way that we think about students and addressing so many of the challenges I think that dehumanize young people in schools. Thank you for those great closing words and thank you to all of our panelists. This is a great discussion today we've learned so much from you in the course of our work together. So if you are a California educators who's interested in joining the pack. Feel free to reach out I know we have a link that we're going to share in the chat that you can get you on the mailing list that we have a mailing address. For everyone on the webinar today we invite you to dig into the research reach out to the LKI team if you have any questions. District leaders you may want to start out by looking at the key conditions in the report and thinking about what conditions you already have in place and where what you might want to start working towards. And we also do have wonderful resources available from all of our partners as well, which we were sharing out both in the chat and we also will send us a follow up email along with our recording of the webinar today. So I just like to say that as you're closing out you're going to see a survey pop up as you leave the webinar and we would really appreciate your feedback. Thank you everyone have a wonderful day.