 Today's webinar is the first in the series state efforts for building an effective diverse teacher workforce. We at NASB are delighted to do this in partnership with the National Conference of State Legislatures, a key partner for building shared state leadership and education policy across legislatures, state boards, education chiefs and governors, and the Learning Policy Institute and Learning Forward. Both recognize leaders in policy, advocacy and work with states and districts in advancing learning and teaching. Let me say we want this to be a very interactive discussion today, and we'll have plenty of time for questions and answers after the presentation. Please keep your phones in silence mode and enter your questions in the chat box and we will get to them then. Before introducing our panelists today, I want to briefly frame this webinar series in today's discussion. First, it is important to put this in the context of the Every Student Succeeds Act. We believe strongly that F is both an opportunity and responsibility for states to commit to leading for equity to close persistent achievement and graduation gaps. First of these 10 equity commitments in this report is to prioritize equity, set and communicate an equity vision and measurable targets. It is also important for states to know and leverage ESSA Title II opportunities for educator development and to see best practices in what other states are doing. And we need this title funded fully, now and in the future. This is not the Title II of NCLB, it has been completely overhauled and reimagined around the use of evidence of effectiveness for the greater impact of teachers and leaders on students and reimagined around this research-based definition of effective professional learning which is sustained, intensive, job-embedded, collaborative, classroom-focused and data-driven. Stephanie Hirsch is learning forward with us, leading advocates for this definition in ESSA and will address this in her presentation. Recognizing students have different needs, equity can be defined as simply giving students what they need when they need it. Our equity vision for this series is equal access to deeper learning, which I will define shortly, so that every student graduates ready for college, career, and civic life. This robust definition here emphasizes the importance of access to an education focused on meaningful learning that equates students to the contemporary world of work and society taught by competent and caring educators who attend to a student's social, emotional, and academic needs. So these are the cross-cutting themes of this series. I'm going to identify them briefly here, and the panelists will unpack them more fully, make them more explicit, and of course this will come up in your questions later. First, state leaders should have and communicate a bold vision of student learning, grounded in equity and research as the core of their education system. This vision is a powerful lens for viewing all other state policies, especially for policies that develop and support the kinds of teachers needed to bring this vision to life in their students and schools. We see teachers as engaged in a complex, highly skilled profession exercising expert judgment in the thousands of decisions they make every day. These next two things go together. The importance of a systemic approach to teacher pipelines, that advanced teachers on career progressions for greater mastery and leadership, and then also concrete state and district policies and actions at all points along those pipelines. We need to close teacher diversity gap. National average is about 18% teachers of color, or over 50% students of color. But these gaps vary significantly locally. We need to close these because diversity and effectiveness go together. Students benefit in seeing themselves and their teachers, but the greatest benefit in parity is that all students can interact regularly with teachers of their own and different races and ethnicities. And finally, local context matters. The use of regional and local labor market data is very important for analyzing issues of teacher shortages and specific needs for diverse, effective teachers. Keeper learning supports greater academic content mastery through understanding the wide of learning and applied learning. It engages students in intrinsic motivation to learn and strengthens their problem solving and critical thinking. Most importantly, for educating students to navigate the rapidly changing world of work, technology, and society, it strengthens their ability to transfer knowledge and skills to new situations, jobs, and problems. David Connolly did a crosswalk of deeper learning against the College of Career Readiness Standards a few years ago, and he found that deeper learning reinforced and exceeded those standards with its greater emphasis on communication and collaboration, and its additional focus on learning to learn and a growth mindset. This graphic I think is a very good way to show how you can integrate deeper learning with academic content. You can see on the two columns on the left of the academic content, and then the two on the right are the deeper learning. This is also a good illustration for graduation pathways that focus on three components of academic content, post-secondary goals, and employability skills. Finally, I have emphasized deeper learning in this opening as a primary equity strategy to say that this is what teachers need to be able to teach and also exemplify themselves. These are the three high leverage policy areas in a teacher pipeline where this can happen. We want teacher preparation that's closely aligned to school expectations and best practices and instructions, so graduates are learner ready day one. We want professional development as ongoing coaching, training, practice, collaboration, teachers learning from each other, and reflection for their greater mastery. And then finally, we want career progressions that are based on incentive, support, and accountability for continuous improvement in teaching and exemplifying deeper learning. All of us on this panel strongly believe that schools are the primary places of adult learning. And this last point brings me to our panelists today. I'm leveraging SS Title II for job-indeaded professional development. I'm delighted to introduce Stephanie Hirsch from Learning Forward, Maria Hyler from the LPI, Shelly Rouser from the Delaware Department of Education, and we hope that maybe Donna Johnson, the executive director of the Delaware State Board of Education, may also be able to join us and do the calendar changes through the snowstorm. Please note their bios are in the chat box. And also, I do want to apologize. Again, we're in a snowstorm and there may be some audio issues, and if you do have these, try to enable the audio via the telephone. And again, I'm very sorry, so now I will hand it over to Stephanie. Good afternoon, everyone, and thank you, Don, and thank you so much to our partners at NASB, at LPI, at NCSL. And I also recognize that there are many, many experts in the room today. And so I look forward to brief remarks and then the opportunity to engage in a conversation. And I hope that you all will do that with each other. We have representatives here from numerous state boards of education. We have experts from CCSSO and many experts from our SEAs. Also, I noticed our Learning Forward President, Alan Ingram, is in the room as well, a former deputy commissioner himself. So don't rely just on your presenters. Rely on all of the expertise among the colleagues who are sharing this time today with you. I love the subject of this webinar at school. Everyone's job is to learn, and that's where I intend to focus my remarks. And I'm having a bit of a challenge forwarding, but I'll get it to work. There we go. So I am hopeful that most individuals in the room are familiar with Learning Forward. If not, you can find our website below. We are very proud of the fact that we focus solely on the relationship of improving educator performance in order to improve student performance. And Don laid out a beautiful vision for what we want for students, particularly using deeper learning as the frame for that. No matter the vision that we want for students, because we all want more than what we have today, we also have to recognize that in order to achieve this, that we need to transform classroom instruction and if we intend to really transform classroom instruction, then we need to think about how we transform professional learning. And it can't be the sit and get or drive by, neither one of those will help us get there. And it's fortunate that the change in ESSA and changes that states made long before the change in ESSA are helping us to move in this direction. As an organization, we hold some assumptions about the importance of professional learning to achieve this new vision for deeper learning and for this particular webinar, I focus on that fourth one that we hold that the most powerful professional learning occurs among teachers and learning teams committed to continuous improvement. I invite you to look later at the other assumptions and to see to what degree you agree or disagree or are they part of what underlies the planning within your state and within your district to focus on and improve professional learning for educators. But if you believe that the most powerful professional learning occurs among teachers in learning teams, then hopefully you'll embrace this very simple vision, a learning team for every teacher. No teacher should work in isolation. No teacher should have to rely solely on their own knowledge and skills in order to support the diverse needs that every teacher faces every day in classrooms. And as Dawn mentioned, we worked a long time with a lot of people in this room and people outside this room today to change the definition of professional development that could have been considered a laundry list to one that really acknowledges what teachers have said for years is most valuable to them. The fact that it is intensive collaborative, it's focused at the classroom level. It is job embedded. It focuses on what do teachers need to do tomorrow and it is driven by data. So at Learning Forward, what we tried to do was translate that definition into meaningful collaborative learning for teachers. And this is one piece from the work that we do around intentional professional learning communities. While I'm sure many of you know many places that engage in professional learning communities, often they may have good conversation, but they may not result in powerful changes in the classroom. And so we propose a process that is very intentional about the steps learning teams work through in order to get the outcomes that we want to see for both the educators and the students. My intention today is not to go deep into those stages, but to recognize that no matter what system you adopt for supporting collaborative teams that you can clearly identify the stages, the steps, and the outcomes that you want from each, then in order to build the conditions and the context for supporting highly productive learning teams, we have presented in many of your states have adopted the standards for professional learning. It outlines what is absolutely essential if we expect those learning teams, if we expect job embedded professional learning to truly provide the kind of support teachers need to meet the needs of all students in their classroom. We're also proud of the fact that many of the states represented in the room today have adopted the standards for professional learning. Some of you have adopted the most recent version. Some of you are still operating under older versions, but just the commitment that we want all teachers to experience great professional learning, and so we have these standards that define them for us. And when I think again about the subject matter of this webinar and how I can, and we can be most helpful to you and think about, so what's the role at the state level? Three things to consider that we can talk about later is do you have a vision for professional learning? Does it include a definition? Does that definition align with the new definition within ESSA? Then once you set a definition and adopt your own standards or guidelines that you want to be used in the planning and the implementation and the assessment of professional learning, then take time to assess where you are currently. How well does the professional learning within your state align to that vision? Does it really meet the new definition under ESSA? And once you've done that assessment, determine the next strategic priorities and actions that you'll take for your state, and I always encourage people limit, limit, limit. What are the two or three things that US state can do that can have the greatest impact in terms of moving closer to that vision? I'd like to mention that we have lots of resources to help you. I'd say most of these up here, I think, except for a couple, are absolutely free, and I'm always willing to share anything if you don't have the dollars to buy a book or something like that. And one of the best resources will be through the Learning Forward ESSA page, and I've given you the link there as well. So I hope that just serves as a big picture for the definition because I know that the next part of this, Maria is going to go into even more detail that I think will be helpful to you. Thank you. Thank you, Stephanie. Thank you, Don. I appreciate the opportunity to be here with the partners NASB and Learning Forward and share some of the research that we have done at Learning Policy Institute around supporting effective teacher development. Looking specifically at what the research says about teacher professional development, that impacts student learning. So as we all know, since as Stephanie acknowledged, there's lots of experts in the room, there's been an active conversation about both the value and the design of teacher professional development, and fortunately research has provided some insights to the questions that have come up. While it's certainly true that professional development for teachers is not all high quality, we've learned a lot in recent years about what makes some of the programs more effective for teacher and student learning than others. Traditionally teacher professional development has often been lecture based, what might be called sit and get or drive by learning, that offers the same content and strategies to all teachers regardless of their skills and experiences and is largely divorced from teachers day to day practices in their classrooms. And what we want for teacher professional development is to reflect what we want for our students in K-12 classrooms. We want rich deeper learning experiences that really impact teacher learning and improve practice. So the paradigm shift is noted in rigorous research. Shows that PD programs with an impact on student learning follow a different model defined by seven core elements. Strong PD is focused on the content that teachers teach in their classrooms. Programs demonstrate this characteristic when, for example, they offer opportunities for teachers to construct lessons and units for a new curriculum or investigate how students learn specific context in a subject area. Unlike lecture based or sit and get PD, active learning offers teachers a chance to meaningfully engage with new concepts and teaching strategies by actually doing them. Active learning strategies include analysis, discussion, observation, and direct practice. Stephanie spent a lot of time talking about collaboration and collaboration can take many forms with teachers working one-on-one with a coach in a small group or as part of a professional learning community that extends beyond the school. And it can occur remotely using technology or be in person. Collaboration often occurs in job embedded contexts as teachers together offer each other feedback and problem solve specifically about their students. The fourth key feature is models. By providing teachers with models of effective teaching, PD programs offer educators a clear vision of best practices. There are many types of models that can be employed to accomplish this purpose, including curricular resources like lesson or unit plans, student work, teaching cases, and observations of peer or master teachers. Effective PD programs also provide coaching and other expert support to facilitate teacher learning. These experts, usually educators themselves, often lead teachers through the active and engaging learning experiences I've been sharing about, but they also tailor specific advice and counsel to the needs of individual teachers. Rather than providing a one size fits all experience, coaching enables programs to meet individual teachers where they're at and supports improvement. Coaching often entails feedback and reflection on teacher practice, but we found that powerful teaching teacher learning opportunities, regardless of format, offer time for teachers to engage in these activities. While these practices are distinct, they often work together to support teachers as they move towards the expert's visions of practice articulated by PD. And we know that accomplishing all of this takes time. So effective PD must be sustained, must be a sustained duration. Research has not offered any magic number of hours for effective programs. Instead it indicates that providing opportunities for teachers to study deeply and then apply their knowledge and cycles and inquiry over time is essential. Often this involves intensive workshops that set teachers up to apply new approaches in the classroom and then opportunities to reconnect to debrief and problem solves together. But even the best design PD may encounter challenges with implementation that limit its effectiveness. For example, there are any number of school level challenges that have been shown to be obstacles to effective professional development, including inadequate resources. This includes financial support but also can be materials such as equipment for lab experiments or project-based learning and even in the most extreme cases the lack of books or basic materials. Teachers also sometimes contend with limited opportunity to use newly acquired knowledge in our classrooms. So in one study that we reviewed a teacher received PD related to science instruction in her elementary classroom only to have time for science instruction removed from the schedule entirely. School culture can prove to be a powerful obstacle to effective professional development. For example, in one school teachers mistrusted the leadership and therefore lack buy-in to the mandated PD offered even though it was of high quality. This is one reason that school leadership is very essential when thinking about professional development for teachers. Challenges to implementing effective PD extend beyond the classroom and the school. These challenges include a lack of alignment between the teachers between what teachers feel they need to learn to best meet their students needs and district initiatives and priorities. Likewise, there's often a disconnect between state and local policies. For example, states generally require seat time for recertification which in turn encourages districts to organize one-off workshops that are easy to schedule and require less human and financial resources than do evidence based approaches to professional development like I've shared. Relatedly, very few states and districts have robust tracking systems for professional development that include both quality as well as quantity. Without such systems in place, it's hard to adopt and implement professional learning for teachers that is evidence-based and designed to address potential obstacles. All of these challenges can be addressed through thoughtful policies and more strategic implementation. That's the good news. So just as the standards that Stephanie mentioned we can start to adopt standards for professional development to guide the design evaluation and funding of professional learning provided to educators. Evaluating and redesigning the use of time and school schedules to increase opportunities for professional learning collaboration including expert coaching and collaborative planning is a way to think about school schedules and redesigning schools. Key to this are leaders that understand and have skills necessary to undertake organizational redesigns which we'll talk about later. Conducting needs assessments to identify what teachers feel they most need to help their students learn best is important because teachers need to have input into the types of PD that are available to them. The development of expert teachers as mentors and coaches to support teacher learning. States and districts could also create professional learning into S of school improvement initiatives such as efforts to use student data to inform instruction or create a positive and inclusive learning environment. There could also be provisions for technology facilitated opportunities for professional learning and coaching especially in supportive rural communities and provide opportunities for collaboration using funding under titles two and four. And finally policy makers can provide flexible funding and continuing communication units for learning opportunities that include sustained engagement and high impact professional learning opportunities not just traditional workshops. So I'm going to pass the mic over now and look forward to having a discussion. Thank you. Good afternoon everyone. My name is Shelly Rouser and I serve as the director of K-12 initiatives and educator engagement at the Delaware Department of Education. So far this afternoon you've heard about the opportunities that come with ESSA for equity and excellence in our schools and at the local and state system levels. The Learning Policy Institute has set the stage by naming the paradigm shift and policy implications and learning forward has spoken to the necessary conditions for and shared tools that will help us as leaders change the effectiveness of and impact of professional learning for educators. We all know that there's a certain cascade effect of the quality of professional learning for educators and we know that the quality of learning is the quality of learning in our schools as a whole. As we adopted the new college and career ready standards for students in Delaware it became increasingly clear that we needed equally rigorous standards for the quality of professional learning for educators as well. So in 2012 Delaware adopted the learning forward standards as our own but quite frankly they were not getting the same level of attention as we focused on the standards for instruction and less on our own adopted professional learning standards. In fact as you see here our most recent health survey in Delaware reveals that there's still a lot of work to be done in this area. 93% of teachers are reporting that they were being held to high standards for instruction with students but just 50% felt like the PD was differentiated to meet their needs. And less than 50% reported that there was a culture for evaluating the impact of that professional learning and that those results were communicated to them. So we knew that there was work to do. So in 2012 we launched an initiative called Common Ground for the Common Core and we kept that initiative going for three years with two goals in mind that are aimed at really both levels of professional learning. On the one hand it supported our teachers and principals with implementation of the newly adopted college and career ready standards. But at the same time we were modeling the expectations that we held for professional learning. Putting the adopted standards, the learning forward standards that we adopted, putting those standards into action putting a laser focus on implementation and evaluation two of the learning forward standards that were highlighted. And responsiveness to needs, a program that evolved each year responsive to what we were seeing in the field with implementation based on our feedback loops with the participants. So we were challenging ourselves to provide the same sort of professional learning that we wanted for all educators in the state. So just as we were winding down this Common Ground initiative and adjusting our approach the Mirage Report study was released and it played a key role in our efforts to shift our attention from state-led professional learning to an approach that helped our schools and the district that supported them in redefining, reevaluating and reinventing professional learning that's based on their very own specific local needs. We launched a new initiative called Reimagining Professional Learning. It incentivized schools with grants and they were able to paint the picture of what they could do to reimagine professional learning in their school if only they had the resources to do it. So our strategy changed in the right way and at the right time but our focus remained the same. This slide actually begins to capture the two approaches that we had. We on the macro state strategy level through the Common Ground professional learning initiative as well as what we did with the Reimagining Professional Learning grant at more of the micro local district charter level as far as our strategy there. So first the focus on implementation of the standard through understanding the shift and then that shifted into us having schools apply that knowledge of the standard through high quality professional learning and high quality instructional materials. Next there was a focus on evaluating the quality of professional learning that was certainly probably the biggest paradigm shipper for the state and it continues today. At the state level when we initiated the Common Ground initiative we knew that that was going to be the biggest shift in our approach to professional learning and we had Dr. Thomas Gueski come to Delaware and do training on evaluating the impact of professional learning on practice and student learning using his five levels of professional development evaluation. That training was especially geared towards central office directors of instruction. In fact Dr. Gueski who is actually doing a webinar for Delaware as we speak know or shine for those grant schools. We introduced protocols also such as feed forward feedback and tools such as a Gueski placemat that now reimagining schools going down to the micro level are using those same tools to evaluate the impact of their school specific professional learning on student learning and teacher practice. And we have DOE liaisons that check in with those schools on evidence of their impact. Also on implementation of the professional learning standards as a whole what started as modeling of a backwards design from the professional learning standard through our state template with more state level goals around understanding the standards deeply has now evolved into a similar state but one that's grounded in the needs assessment revealing where both the student learning needs are and using learning forward standards assessment inventory the SAI measuring where the needs are for school systems to really support the professional learning of educators. And it's also worth mentioning that just as the school check ins with department liaisons we too update our state board from time to time with evidence of impact and I'm happy to share more about this and Donna Johnson will share more about this when we have time for discussion in just a bit. We're happy to share resources and the tools that we use with schools the processes that have been helpful, the lessons learned, the partnerships that have made all the difference such as those partnerships with learning forward and with the Mid-Atlantic Comprehensive Center Mac at WestEd. We're also going to share those behind the scenes routines of our department that kept the work going along but in a nutshell this is what our schools are committing to and this is the work of the DOE liaisons to support them in doing it. It's about responsive professional learning that's grounded in clear learning goals and evaluated. It is about high quality instruction through high quality instruction materials for students and it's also about something that Stephanie spoke to earlier a cycle of inquiry just as we asked teachers to engage in the cycle of inquiry or on student learning. We're applying those stages of learning that Stephanie referenced the analysis of goals, the setting goals, etc. so that they are equally focused on the quality of the professional learning. It's also about ownership at all levels while they are school-based plans. It's about the districts engagement with them as well as the teacher voice so all three levels sign on to the plans that they propose and finally a willingness to align their learning systems and to reallocate resources to align with those priorities. This next slide summarizes the grant requirements but especially like to call attention to the last two bullets. We have found that two things really came out with our schools. One was around a focus on evaluating PL and on leading change and especially the learning forward standard of learning design and so we found that having periodic professional learning sessions for the grant leaders was helpful and also changing the course of our check-ins with districts to more focus in on those areas. At this time we're able to have Donna join us. I'm here Shelly. Thank you so much and I am just excited that Delaware is really able to show the partnership that our Department of Education and our State Board of Education has been able to foster on this and many other issues and as we often talk about in many of the webinars that NASB puts on is really trying to take a look at connecting all of this great work and initiatives and grounding that back into what is the role, the opportunity and the responsibility of our state boards and one of the things that Delaware has really tried to stay true to is making sure that the responsibility of voting on and making policy does not end with that action. That the Board has taken that responsibility in terms of monitoring and asking for feedback on the implementation of the policies that they have approved. So one of the things that we've really taken a look at is asking the department as Shelly talked about was to regularly bring in an update on where are we with standards implementation, where are we with the implementation of our professional learning standards, how is that connecting into the performance gaps that we're seeing and what supports are being given to schools. So really utilizing those three powers that the State Board has which is the power of the question and taking that policy action that we've had and following up on it. The policy to convene bringing together groups of people for workshops or presentations to the Board or presentations to stakeholder groups to talk about these initiatives and further share these ideas and then most importantly the power of the voice and making sure that Board members are utilizing their voice to advocate for the changes that are necessary to improve student learning for each and every one of our children. Thanks Donna. Okay well thank you so much Donna and Shelly, Maria, Stephanie and now we have to take full advantage of the time we have for questions and as Stephanie mentioned we really do have extraordinary expertise in all of the participants. So please feel free to use the chat box to ask any questions and why you think of those. I'm going to start with a follow-up for Shelly and Donna in regard to the collaboration that you've had with the State Board and the Education Department and given that these teacher pipelines have so many entry points, can you talk about some of the lessons learned, how you determine priorities and identifying strategies for our other State Board members and Education Department members? You know I think I think I would want Shelly to kind of talk about the lessons learned from the aspect of the work that they've done around reimagining professional learning and then using the data and the feedback that they've received really kind of and getting that information back to State Boards helps State Boards advocate for and drive to address whether it be opportunities for improvement or addressing any hurdles that policy may get in the way of success. So I would like Shelly to talk about some of the ways that they've used the data that they've collected to improve their process. Thanks Donna. And so it was really important for us that as we thought through the elements that were necessary for our schools, our districts to move forward professional learning that we were constantly flipping the mirror and looking at our own State practices and the degree to which we were modeling them. And so that's why we took this approach in even sharing with all of you this afternoon this macro level and micro level and really us looking at a State Agency as to whether or not we are studying our own evidence of impact of what we are doing on the field just as they are looking at their own school level professional learning and how it's impacting students and teachers. And so our team actually had processes that start in the summer before the school even starts the plan. They're building their plans right now as we speak. We're bringing on year three schools. But once we name those schools, our staff actually immediately reaches out to the schools to even work with them on their own evaluation instrument. And that was actually a change between year one and year two. We noticed that the evaluation is, we knew that the evaluation part would be the toughest shift. But we also underestimated I guess the need to support our schools with creating tools for that evaluation to take place, not after the fact but as the professional learning is rolling out throughout the school year. So in the summer months our DOE liaisons are signed to a school and they reach out to that school to sit down with them and help them create tools unique to their own area of focus and actually identify what will be evidence. What you predict will be evidence that what you're doing is having an impact. To the point of the information that we're able to share with the State Board of Education, that data is helping our schools have professional learning, but that data is also providing support to us as a State Agency. And we're able to leverage the power of the Board to convene the voice to their questions to help make this stronger and stronger year by year I would say. I hope that's helpful and I guess I'll stick to the question. Yeah, absolutely. I would add one thing to Board when you have this information being presented to you from whether it be your department or if you have the opportunity to bring schools in and hear directly from them what's working and what's not working. One of the most helpful questions that our Board members have asked people that are implementing this on the ground and working directly in our schools is what policy or practice gets in the way of doing what you believe needs to be done to meet the needs of our students. And really trying to look at and then unpack is that a policy hurdle? Is it an implementation hurdle? And how can we address what those challenges are to help our educators meet the needs of our students? Well, thank you both. Let me continue with some follow-up questions that we have in the chat box and then we'll go on to some other questions. But again for you, Shelley and Donna, we have question about how are you planning to sustain funding the new models beyond the grant program period in schools? And then Shelley, can you give further examples of the types of evidence in school districts for use to demonstrate professional learning? Sure. Well, one of the things I'll say is we are committed to putting our resources to this for the next few years. And so we have planned for that with our budget to support schools. And as we did with the former initiative that I mentioned, Common Ground, we'll be looking at how we have to have a gradual release of responsibility and what the next level of support will be. And we're not sure what that next level of support will be and certainly resources will play a part in that. But there is a commitment to that for the next couple of years until we begin to see we kind of reached that tipping point where we have these, we were able to reach enough schools to see enough districts with schools that have had this opportunity. As far as the evidence that we are seeing from schools, one thing that we, well of course there's the obvious evidence, I'll start with saying that the evidence is at the multiple levels of Gusky's evaluation. And so when they build your plan out, we help them look at each of those five levels, which are things like the satisfaction of professional learning, the obvious things, like the surveys that Gusky just jokingly calls their happiness quotient. As they're walking out of the door of training, how are they responding to it? But bigger than that, it is about the knowledge and skills that we see. And so that often comes in the play, in the way of walk through tools. It can also come from flipping focus groups, flipping professional learning communities into focus groups. And with key questions, the information coming from those focus groups, those PLCs, funnels up to the schools, it's a qualitative measure. Sometimes it's very real things like pre and post things to see where teachers are. And then as you go on, it's also about Gusky's level three, the support systems that are in place and finally things like teacher practice and walk through tools and peer visits that teachers do of each other. And of course the evidence that's bubbling up from PLCs as far as student learning artifacts. So those are some of the things, and our schools have gotten really creative with some of the things that they pull to serve as data sources to inform their professional learning. But those are a few examples. To help everyone, I just added into the chat box presentation from the June State Board of Education meeting in Delaware where we had not only the department but also representatives who had received these grants showcase and demonstrate some of the work that they had done and the results that they were getting. There are also a couple ways in which the department has given that feedback back to the board when they presented assessment data. They've connected that back to talk about the impact of increase in performance that was seen by schools that had taken advantage of some of these grants. Okay, that's great. And one more question for you. Is there a sample or a copy of your Delaware survey available for participants? Our surveys now, we as far as analyzing where schools are in terms of their quality of professional learning, their implementation of the learning forward and Delaware professional learning standards, we utilize the SAI. So the standards assessment inventory is our go-to tool for giving a pre-post and letting schools actually plant or pose what their next steps are in terms of implementation of the standards. In terms of professional learning surveys themselves we have developed a bank of questions that we have our schools tap into so that their questions go beyond did I like it to addressing some of those other levels of Gusky's framework. But we do not have standard surveys. In fact, we actually encourage our schools to take our sample questions and really customize them to the very specific things that they are focused on in their grant. We suggest the same thing of them when it comes to their walk-through tools so that their data is specific and not general. But we certainly could give you examples from our schools and we'd be happy to share that if you follow up with me after the webinar. Okay, great. I'd like to open this now to other panelists. And Stephanie can you talk about the role that partnerships played in your work with states? Sure. And one of the things I'd like to say before I answer that question is I just want to give accolades to Delaware and all of the work that it's done in implementing the standards and when we had the opportunity to visit the schools and to watch the work that they did in planning, effective, school-based, job-embedded professional learning it's pretty amazing. And so just accolades there and I know if I visited other states and places I know I would probably see similar work that would just make me swell in pride about the great work that takes place within our districts. In terms of partnerships really we think everything is about collaboration. I know Maria reinforced that that was kind of the message of my remarks at the beginning and it's also about readiness. People are at different stages of change and recognizing that different people have different kinds of expertise. I think bringing as many different voices to the table dealing with the challenges that you can anticipate up front will always position you better for making the kinds of changes that you want to make going forward. And all of the organizations represented in this webinar as well as participants in the conversation I know are all eager to help states. It's the right time for that organization, its perspective and the resources it has to offer. We want to give anything away that we think will help move a state along its agenda toward helping all students achieve at high levels. Well thank you Stephanie and I'd like to amplify your prayers for Delaware. They really have done so much good work in really teacher pipelines as a whole. One of the problems with these webinars is we just don't have enough time for each panelist. But let me ask this next question for Maria. Maria, given this vision of professional learning that we've shared in this webinar, how can that support state and district effort to recruit and retain a more diverse teacher workforce? Thank you for asking that question, Don. I think that we have to look at what the research says and there's reports that LTIs have put out, several reports recently on recruitment and retention of teachers in general and in teachers of color particularly. And what we find is that the research shows that teachers of color are leaving the classroom at higher rates than white teachers and so what are some of the reasons? We know that teachers of color two times as likely to enter through alternative routes and that also speaks to the fact that teachers who are underprepared are often teachers who are underprepared to teach leave the classroom at higher rates than those with adequate professions. And we know that preparation programs look different from traditional and alternative and quality varies. But one of the things that professional development can do is step in for some of the gaps in learning and preparation that teachers of color might have missed through their preparation. The other thing is that teachers of color are more likely to teach in schools serving students from low income backgrounds and in low resource schools. So we can ask what type of professional development is even available in those schools and look at school funding to help to improve the quality of professional development for teachers of colors in under-resourced schools. Additionally, professional development in general can also help to improve the work conditions for all teachers. One of the things that we know from research is that working conditions matter a great deal in terms of teacher retention. And schools can be places where just like in society there can be racism or microaggressions and so when we have professional development opportunities that help all teachers to understand positive working environments and think about issues around race, culture, language, social economic status, all of those social identifiers. When we have teachers learning together about those things, that helps to improve the learning environment in schools not only for students but for teachers as well. Thank you Maria. And I do want to let everyone know that we will be addressing the diversity gaps of webinars that we'll be hosting and I'll mention those at the end. We are running late on time but I would like to ask one more question because we have such a great panel and participants and I want to take advantage of this opportunity. Let me open this up to everyone. How can states support districts in creating communities of practice where teachers can learn from each other? Well, this is Stephanie and I'll start and then the rest of you can add to my list. I think some of these things were mentioned. I think Maria you mentioned about time. So I think states can help districts find different models for creating the time that you need for communities of practice. I think states can take the lead in building training for teacher leaders and team leaders for communities of practice so that you have quality and consistency in what occurs within those communities. I think that the state often in some states they're bringing together schools and or systems with common needs and common problems so that they have the opportunity to work collectively on sharing the problem as well as experiencing a high functioning community of practice and lastly I always want to think about how you can incentivize districts to adopt community of practice models that meet certain criteria that the state may choose to identify. This is Shelley and I would add to that I would especially pay attention to what Stephanie shared about teacher leaders. I think that states can use the power of convening to provide opportunities for teacher leaders that's been integral to our successes. We may not be able to as a state reach every teacher but if we can empower our schools and build their capacity we found a lot of success with teacher leader initiative at the school level pulling teachers who are able to then be that go to person in their schools. Not underestimating also the power of bringing them together not just to focus on the content, the standard the learning as we would traditionally think of it but also when we bring those teacher leaders together to also make sure that there's equal time paid to the skills that they need to actually serve as teacher leaders. One thing that we learned early on is that we pull teacher leaders together. It's one thing to provide good learning. It's another thing to empower them to be able to leave leaders back in their schools and so we've been spending a lot of time on building up their confidence along those lines too so that they really are a strong resource for school. Okay and we also have a question from Lisa about Gusky Is there a go to site and or progression of learning? Can anyone address that? I can. We have of course depended on his book evaluating professional learning so we are deeply grounded in that. There are some other resources that you'll find. One of them that I would say is just super important and really simple and straightforward is Gusky has a nice one pager, two pager that outlines the five levels and it gives examples of the types of evidence that you would look for at each level and it also gives examples of the kinds of questions that you would ask yourself in planning and in executing to think about professional learning, evaluating professional learning in new ways and that simple two pager has been key for us and in fact is what inspired what we call in where our Gusky placemap which basically is just a tool that schools can use to plan with the end in mind, plan with their evidence and then back map from there in terms of what they're actually going to train on. So they use the levels kind of sequentially levels one to five to evaluate but when they're planning they start with the end in mind student learning and work their way back to what they'll actually do and we're happy to share the Gusky placemap. Thank you and now I'm just going to finally point out and recognize Seroja Warner and please do look at the resources she provided the webinar that shows the implications of biases of teachers on school disciplinary practices and in particular the impact on black girls. We are now going to close and I can't thank our panelists enough and I can't thank our participants. I do want to show you that we have these resources and we can also all of us are very happy. Here's our contact information to provide any additional follow up view. Please feel free to contact us. I do want to say that we are so excited about this launch. I can't believe the incredible work of our communications director Renee and everyone and all of you having just an absolutely great launch in this snowstorm under difficult conditions. So here are the upcoming webinars in this series, April 25th. We will have one on teacher leadership, career ladders and relicensure in support of deeper learning and then on May 30th we'll have a webinar on bridging the continuum. Teacher preparation and induction for deeper learning and there you see the details for getting more information about those webinars. So with that I just am so pleased and thank you so much and I really do hope that all of you will continue to join this series and this conversation. So thank you. Thank you. Thank you.