 And now, I will turn things over to your host for today, Linda Darling-Hammond. Please go ahead. Thank you. It's wonderful to have everyone assembled here for this briefing. We are, as you know, issuing a new report that is going to be available online. If you check in the chat box, you'll find a link to the report and the brief and to these slides that you can refer to them thereafter. Our agenda is that we will get a presentation about this research. I'll queue up the sort of context. Beth Malloy, our Senior Researcher and Policy Analyst and Marjorie Wexler, our Principal Research Manager, will present the research. We'll have an opportunity for discussion and questions and answers. And that will be handled by Maddie Gardner, Beth Malloy, and Marjorie Wexler. You can, as you heard, put questions into the chat box along the way. And we'll get to as many of them as we can. And let's see if I can get this to advance. We're going to just sort of remind everyone on this call, you're all very familiar with the history of early childhood education and research, that we are at this moment in time building early childhood programs across the state based on the great expectations that were put in place by the well-studied preschool programs of the 1960s and 70s. Harry Preschool, the Child Parents' Center, the Abbasidarian Project. All of these were very, very high quality programs that were shown to provide significant benefits to the children who attended them relative to comparison groups who had not attended any kind of preschool in terms of improved cognitive development, their preparation for school, a much lower likelihood of being identified for special education or retained in grade, and then much longer term benefits as well, all the way into adulthood for the studies that were able to follow young people that far, much more likely to graduate from high school to attend college, less likely to become pregnant, to commit crimes, to depend on welfare as adults, higher earnings in adulthood. All of these benefits which have been much touted in the news and in political conversations have led to enthusiasm about preschool and investments in early childhood education. The various studies have found the benefits pay off as much as $17 for every dollar invested when you look all the way to adulthood and capture all those benefits that I just talked about. And even for the short term studies that just look ahead a couple of grades into primary school, you find benefits of at least $2 for every dollar invested in terms of the reduced cost of grade retention, special education placements and so on. And of course every dollar, anything above a dollar means that preschool is paying for itself. So there is this basis on which to advance public policy. There is of course the fact that because of this state investments have skyrocketed, 43 states have provided preschool, lots of money invested, many children enrolled, many more evaluations of these new efforts. But that has created a more complex research landscape with some studies finding less significant benefits, what is sometimes even called fade out of those effects into the later years of grade school and questions about the effectiveness of particular programs and those sometimes in the political discourse end up being questions about the effectiveness of preschool overall. And so this briefing is based on a report where we've really dived in, looked at all of the preschool evidence, unpacked the studies like the Head Start Study and the Tennessee Evaluation that have raised some concerns and confusion and Beth Malloy is going to take the ball and explain the study, what we found and be available for comments and questions thereafter. Beth. Great. Thank you, Linda. As Linda said, there are now many more evaluations of preschool effectiveness than there are programs and they vary widely with regard to how rigorous they are, ranging from studies that simply document growth in preschool students to longitudinal tightly controlled experiments. So we set out to do our review of this literature, joining many others who have come before us and compiling the best evidence to inform discussions about the effectiveness of preschool. With the goal of examining whether preschool improves children's outcomes and untangling those nuances at the heart of those prominent evaluation findings of Tennessee and Head Start in particular, we wanted to equip policymakers and others with the tools to assess the weight of their accumulated evidence and draw conclusions about the case for investing in preschool. In our review, we categorized evaluations into two time frames, grouping those that examine child outcomes at the end of pre case, separately from those that examine outcomes into elementary school and beyond. For that first set of outcomes related to children's school readiness, there are many evaluations of contemporary pre case programs. So we limited our review to only the most rigorous studies, and we ended up with evaluations of 18 public pre case programs that measure children's outcomes at school entry. And you can see the programs and evaluations listed here. Fewer programs have been evaluated longitudinally, and so we expanded our inclusion criteria slightly for programs that follow children into elementary school and middle school, and we wound up with evaluations of 14 different programs. And it's important to note here that the majority of these contemporary programs are simply not old enough to have followed children into high school or adulthood as the foundational programs that Linda talked about have. What did we find? Well, the first thing that we found is that pre-k undeniably prepares children for school. Most studies examine children's early literacy and mathematics skills, and they find benefits. All 18 of the programs included in our review measured children's early literacy and all but one evaluation found benefits of preschool on measures of children's alphabet knowledge and ability to associate sounds with words and letters and other pre-literacy skills. 16 examine children's early mathematics skills, including skills like the ability to count and understand concepts of measurement, and all but two found positive effects there. And these benefits are found consistently for children across the socioeconomic spectrum. Fewer studies examine children's social emotional skills and executive functions. But of those that did, two thirds found benefits on at least one measure ranging from teacher reports of child behavior to assessments of social and emotional skills to multiple measures of executive function. And this slide summarizes these findings. Overall, evidence overwhelmingly points to preschool's effectiveness at preparing children for school entry. Importantly, the few findings of no difference, those gray boxes on the figure, generally showed positive influences of preschool, but they were not statistically significant, usually because of small sample sizes. Of course, we don't stop at school entry when considering the literature on preschool effectiveness because policymakers also want to know whether investments in preschool pay off, and those outcomes that have clear economic impacts show up later in school and in life. And we consider evaluations that follow children into elementary and middle school. We find that several measured outcomes related to school progress, specifically, special education placements and grade retention are important. Four out of the seven program evaluations that measure special education placements in our review found reductions among preschool participants compared to children who did not attend. And six out of 10 found a reduction in grade retention. When we look at grade retention, it does get a bit more complicated. Here we start to see that the early learning experiences of children to whom preschool participants are compared matters. For example, two evaluations of Tulsa's ECE programs found no effective participation on grade retention. That is, both the preschool participants and comparison children have the same retention rate, but both had low rates. Evaluators in both of those cases noted that many comparison children had access to other ECE arrangements, suggesting that Tulsa's programs and other ECE programs available in the Tulsa area may have been equally successful at reducing the incidence of grade retention. Proving children's progress through school does in fact pay off. School districts spend an average of 13,119 dollars per child each year, and those costs double whenever a student is retained. Further, retaining a child once increases the likelihood of future retention, and that compounds the cost. Likewise, providing special education services can cost more than twice that of general education. So lower rates of grade retention and special education placements from preschool come with significant and immediate cost savings for school systems and society. When it comes to academic benefits in elementary and middle school, the findings are promising, but they're also less consistent. Some studies find enduring effects underscoring that lasting, long lasting benefits are possible. However, others see few of any benefits from pre-K and later grades. Specifically, about half of the studies in our review that measure children's literacy beyond school entry, usually through standardized tests, find benefits on children's reading performance. And in several cases, those benefits persist up to the fifth grade, but the other half find no evidence of an impact. Among studies that examine children's mathematics performance, again, measured by state standardized tests, most document benefits, including some that persist well into middle school. But two of the studies find that preschool participants performed less well than children who did not participate on at least one measure of math skills in the early elementary grades. And we'll talk more about those two studies in just a moment. But I want to pause to highlight here how study methods, again, make a difference. In the case of North Carolina's preschool program, two evaluations of the same program had very different findings. One found a positive impact on children's reading performance into elementary school, and the other did not. But these two studies, although they evaluated the same program, had very different designs and measured literacy skills using different tests. They also used different comparison groups. So we were reminded that these findings are strongly influenced by the research methods and the timing of the evaluation. Here again, this figure depicts the summary of the research on preschools effects into elementary school and beyond that I've just discussed. It's understandably harder to draw a single conclusion from this evidence. However, I think two things are clear. The first is that lasting benefits from pre-K participation are possible. And the second is that those benefits are not guaranteed. There are several gray boxes and even a few red boxes in this figure. So how do we make sense of the findings? I want to take a moment and address the issue of what many call fade out studies that find little impact on children's performance into elementary school, especially in reading, despite finding children were doing better than comparison group children at school entry. The most important thing to understand about fade out is that finding no difference between participants and comparison group children does not mean that children stagnate or lose skills or knowledge over time. On the contrary, preschool participants and comparison group children continue to learn. But over time, the difference between them diminishes. This means that their performance converges. Findings where preschool children were actually doing worse than comparison group children in elementary school. Remember that the bulk of the evidence points to the overall effectiveness of preschool, but findings from Head Start and Tennessee have sparked questions about whether that evidence should be trusted. Both studies found evidence that preschool participants did less well in mathematics and early elementary grades. And the Tennessee study found that children were more likely to be placed in special education in elementary school. And these two studies have received enormous attention, both because of these seemingly contradictory findings and because they were designed to be extremely rigorous from a methodological standpoint. However, research can help guide our understanding of these two evaluations and specifically points to three key questions, the answer to which are critical for understanding how this evidence fits together. Those questions are first, what are the early learning experiences of comparison group children? Second, what is the quality of the preschool program? And finally, what is the quality of instruction in the early elementary years? Let's look at the evaluation of Head Start to see how those questions matter. First, I'll review their findings. Again, initial findings were encouraging and Head Start found that it boosted children's school readiness. However, as children were followed throughout school, impacts on children's literacy and grade retention were undetectable by third grade. And in mathematics, children who attended the program appeared to score less well than comparison children by third grade. And this was an unexpected and unwelcome finding. But what might explain it? Let's think about our guiding questions. First, we need to understand the early learning experiences of the comparison group. And what we know is that many children to whom participants were compared did in fact attend other early learning programs, including other Head Start programs. When a reanalysis took these experiences into account, it found larger, longer lasting benefits for children who would not have had access to ECE without the Head Start program. Now, what about the quality of Head Start? Well, although Head Start is often viewed as a model program, the reality is that Head Start program quality is highly variable. And this was even more true at the time of the Head Start evaluation. In fact, the federal Office of Head Start was so concerned with this variability that it recently overhauled its quality guidelines for programs. Furthermore, another study went back and tried to account for program quality and did find that certain elements of quality, including whether a program was part day or full day, accounted for the size and persistence of impacts on child's outcome. To consider the quality of the elementary school instruction that Head Start participants receive, this is extremely important because Head Start is a targeted program, which means that it's designed to serve mostly low income students. And as such, Head Start children are more likely to attend low quality and low performing elementary schools. Again, research confirms that the benefits of Head Start are larger when concurrent investments are made in K-12 education in Head Start communities. So what do the Head Start findings mean? Well, they tell us a few things. First, just like many of the other state pre-K programs in our review, Head Start prepares children for school. Second, we can say with some confidence that whether the benefits of Head Start last likely depends on a number of factors, including what alternatives to Head Start are available to children in a given community, the quality of individual Head Start programs, and the quality of elementary schools and Head Start communities. Now let's turn to Tennessee. The evaluation in Tennessee was a bit more straightforward because only one set of evaluators have looked at the data. Tennessee's program was another success story. Evaluators found that the children in the program were more ready for school than children who did not attend. But again, when they followed children into elementary school, the story changed. By third grade, children who attended the program were no different from those who did not attend in their literacy, and they were actually doing less well in mathematics than children who did not attend. So again, we're left to ask why. In order to understand these findings, we can turn back to our guiding questions. First, we need to understand what were the early learning experiences of the comparison group children. Tennessee evaluators have not accounted for the early learning experiences of comparison group children, which means that we don't know whether the impact of the program is dependent upon the early learning opportunities that comparison group children received. There is some evidence that despite its rigorous design, comparison group children may have actually been more advantaged overall than preschool participants, which could account for their better performance in school. Perhaps the most compelling explanation for Tennessee's findings lies in the answer to our second question about program quality. Tennessee's program report card from the National Institute for Early Education Research earned only five out of ten on the new metric of preschool quality, a metric that was revised to better reflect the elements of quality that research shows matter for child outcomes. Furthermore, an evaluation of the quality of Tennessee preschool classrooms found that many teachers spent only a little more than half of their time engaged in learning activities, and that may reflect poor classroom management or difficulty embedding learning into everyday routines in play. This evidence suggests the quality of Tennessee's program may have been meaningfully lower than programs that demonstrate effectiveness. Finally, we must consider the quality of Tennessee's elementary schools. Although we don't know the quality of instruction that Tennessee's participants received in the early elementary years, we can hypothesize that it was not adequate to sustain the learning gains that preschool participants had made. The recent national study shows sustained benefits of preschool are often dependent upon high quality early elementary instruction. And I do want to pause here and just note that the Tennessee Department of Education has acknowledged this quality problem to a certain extent and has been spending some investment in some time and energy really aligning preschool and kindergarten to help smooth the transition and potentially benefit the outcomes of children who have participated in the program. So what do the Tennessee findings mean? Again, the program may be more effective for children without early learning alternatives. And perhaps more importantly, Tennessee's program may be meaningfully different from other preschool programs, making it less effective. It is also possible that its elementary schools simply may not be of high enough quality to sustain early learning gains. So an answer to our primary question is pre pre K effective. We find that it can be. In fact, it absolutely is an effective strategy for improving children's school readiness. And it's clearly possible for the benefits of pre K to persist into elementary and middle school. But the inconsistency of lasting benefits from program to program illustrates the importance of asking a different question altogether. Rather than focusing on whether pre K works, we need to move on to asking how policymakers can ensure that their investments pay off. And with that, I'm going to turn the floor over to my colleague Marjorie, who will share some insights from research about just how to do that. This is Marjorie Wexler. As best described, we know preschool can generate both immediate and lasting benefits for children. However, as she said, quality is an important factor in whether the promises of preschool are actually realized. So what makes for a quality program? Research has identified a suite of features that are commonly found in preschool programs with strong evidence of success, like those described in best review. First, high quality programs have program structures that facilitate opportunities to learn. They offer children sufficient time in the program, small class sizes and an appropriate adult to child ratio. They provide engaging and meaningful learning experiences based on comprehensive early learning standards and curriculum aligned with appropriate child assessments. High quality preschools have a highly skilled workforce, which has knowledge of early childhood development and age appropriate instruction. And these skilled educators receive ongoing supports such as coaching and mentoring. High quality programs offer comprehensive services for children, including supports for healthy whole child development. Supports are tailored to the specific needs of the children, including those who are learning multiple languages or have special needs. High quality programs also offer support for families and meaningful family engagement. And finally, high quality preschool programs engage in continuous quality improvement that includes a focus on classroom interactions. A quality rating and improvement system often facilitates ongoing quality improvement. So we know what the factors are enacting these features of high quality preschool on a large scale is complex work. So we conducted case studies of four states, Washington, Michigan, West Virginia and North Carolina, to understand the ins and outs of designing and implementing successful preschool programs. And we chose these four states in part because each had research showing strong outcomes for children. In fact, the literature review that Beth just described includes research from each of these states. The preschool programs in the four states very considerably in design and implementation with each following its own path to quality. However, we did find commonalities across the states about what they are doing to bring that high quality element to life in their preschool programs. And these fall into five big areas. One being keeping a laser like focus on quality. They're paying attention to workforce development, coordination of programs, funding mechanisms and programmatic support. So one, each state has invested in strategies to improve the quality of its early education program. They all developed robust program standards that reflect the elements of quality that I just described, including everything from small class sizes and appropriate adult to child ratios to guidance for how educators interact with children. These standards are reinforced through quality rating and improvement systems that both provide a basis for program accountability. But importantly, they support program improvement through technical assistance in class coaching or financial incentives. In Washington, Michigan and North Carolina, quality ratings are linked to program funding as a strategy to incentivize high quality care. And the states rely on local expertise to ensure program quality. States can't operate early childhood systems on their own. Administrators closer to the programs, such as those in county offices of education, they have more capacity to manage and monitor programs and to address those regional needs. All four states place heavy emphasis on the quality of preschool teachers. They all require lead teachers to have a degree with an emphasis in early childhood education, child development or related field. And as the states increase teacher requirements, they gave teachers several years to meet new requirements while supporting the workforce in meeting the new bar. The states make that training accessible by offering courses regionally or online, which we saw in both West Virginia and Washington. And they encourage teacher advancement through strategies such as scholarships and salary supplements. North Carolina, for example, developed TEACH, which awards scholarships for teacher education, a program that now operates in 26 states. It also developed wages, which subsidizes teacher salaries based on their levels of education. And the states provide on-site coaching to provide tailored support to the preschool teachers. Michigan, for example, employs experienced master's degree level coaches who support all preschool teaching teams in the state throughout the year. Historically, Pre-K has operated separately from K-12 and even from birth through age three in many states. But these four states are at least beginning to work on creating aligned learning continuance for young children. And as Beth mentioned earlier, there is evidence to suggest that this alignment may be very important for sustaining the gains from preschool. So one way that they have fostered alignment is by putting all children's services under one umbrella at the state level. For example, Washington created the Cabinet-level Department of Children, Youth and Families, which brings together early learning and child welfare programs under one roof. And putting the programs together under a single administrative entity makes it possible to have a coherent vision and a coherent strategy for providing services to young children. Another way that states are improving that coordination is by developing more robust early childhood data systems. So West Virginia, for example, has created a data system that enables preschool and kindergarten educators to share relevant child data and to smooth those schools' transitions. And the states are also improving coordination by aligning curriculum and assessments across the early learning grades. So Washington uses the same child assessment tool for both preschool and kindergarten classrooms. All four states rely on a combination of funding streams, federal, state and local, to garner the funding necessary to expand access while supporting that important program quality. All combined, dedicated state dollars with federal funding from Head Start, Title I, or Substitized Child Care. And they also leveraged short-term funds to build necessary infrastructure to deliver high-quality programs. So, for example, Michigan, Washington, and North Carolina all used federal rates to the top early learning challenge grants to build and improve their quality rating and improvement system. And private investments also can play an important role. So Thrive Washington, for example, is a public-private partnership that coordinates government agencies, businesses, and nonprofits across the state working on early learning. So designing the high-quality programs, we know it's hard work. But these states also all managed to build and maintain momentum for implementing them well. So how do they do that? Well, one, they all had some broad-based coalitions that were working together with policymakers to promote high-quality accessible early education. So in Michigan, business leaders supported preschool as an investment in a skilled workforce that would generate a positive return. In North Carolina, prosecutors and police officers supported early education because they saw an evidence of long-term crime reduction. And the states also built broad support by offering parents a choice of providers, including public and private providers so that families can choose the setting they believe best meets the needs of their child. In fact, in West Virginia, there's legislation that requires that at least half of the state preschool classrooms in each county be operated in partnership with Head Start or private childcare and having a political champion with key. So that might be a governor or legislators, but in each state, there was someone who made high-quality early education a priority. And these folks represented both political parties even in the same state underscoring that early education is an issue where there is bipartisan support. So I'm going to pause for a minute. You've heard a lot from us this morning. And that is the end of the presentation part of this webinar. And now we'd like to take a few minutes to answer your questions about this research. And so we, as you know, we as part of the registration, we received some questions. You can pose your questions in the chat. So first up, we'll start with a question from Layla Sanander from the Council of Chief State School Officers. And she asked, how do you address the critics who poke at pre-K effectiveness regardless of how well it's implemented? Yeah, so this is Beth Malloy. Thank you for that question, Layla. I think that to a large extent, the the brief and report and the presentation that you just heard is exactly how we respond to those who poke at preschools effectiveness. I think that you have to turn back to the evidence. And the evidence is fairly clear that, you know, preschool programs, even when they vary widely actually in some of those quality elements, by and large, prepare children for school. And it's clearly possible that children can have sustained benefits. We saw that from our foundational programs, but we see it from contemporary programs as well. And when we start to hammer down and look at those individual programs, they do, in fact, implement those for their their preschool programs with high quality. They focus on supporting educators and paying them well. And these really are the key factors, we believe, in sustaining early childhood gains. Maddie, do you have anything to ask? Beth and hi, everyone. This is Maddie Gardner with LPI. I think that is, you know, exactly right. The only thing I would add, and this is sort of outside the scope of the report, but we know that even beyond the literature that we looked at for this particular review, there's a whole wealth of other resources and studies out there that reinforce some of these messages. So this question of quality is also one that's been well-studied in, again, studies that aren't always necessarily contained in this report, but that we'd be happy to help folks learn about and read if this is a question they're interested in diving more deeply into. Right. Thank you. I'm going to move on to the next question that we received from Tiffany Ferrett from the National Governors Association. And she asked, what was the connection found, if any, to preschoolers who were enrolled in prenatal to age three programs and services, such as Early Head Start, compared to the non-enrolled counterparts in school readiness? So what difference does that early access? That is also a really excellent question. It's the experiences in very early learning opportunities was not something that we considered as part of the review. But again, as Maddie mentioned, it is a well-studied, though right for more research area. We can see, for example, looking back at some of those foundational programs that investing in more than just one year of preschool investing either earlier on or into elementary school or both really has the potential to compound the benefits of early education. And we know from neurological from neuroscience that so much of the brain's connections and early learning really sets the foundation for learning later in life in those first three years. So it makes sense that investing earlier does, in fact, lay the foundation and would propel children to benefit more from their preschool experiences. Okay. Thank you, Beth. Another audience member asks, how did you pick the studies to include in your review? I'm going to pass that one to Maddie to answer. Great. Yeah, it's a good question because I think as you and Linda both said, Beth, there are so many studies out there, including of programs like the ones you were just talking about that target kids before they enter preschool. So I think there's a couple things to note about the way that we sort of approach this particular review. The first is that it's focused explicitly on evaluations of publicly funded preschool. So we really were looking at evaluations of programs that are serving kids in that three and four year old age range. Most typically they were state funded preschool programs. So as you heard from Beth, Head Start was also included. And we also only looked at studies that have been published since 2000 because there have been other great works done on this subject. We were trying to take a more recent lens. And among those that we looked at, as Beth said, we also looked at sort of two different time horizons. And we did that because we were trying to really achieve two things. And the first was to capture sort of a sufficient cross section of literature to allow us to really understand some of the trends, but at the same time balancing that we wanted to really include studies that use methods that are sort of widely regarded as credible for answering causal questions. We know that so often with research, the devil is really in the details about how the methodologies are implemented and that they can study quite a bit even if they use that sort of same broad methodological approach. We were trying to balance those two factors. And what that led us to was that we started first looking at the literature just when kids to finish the program year for preschool. So we called that sort of our short term or school entry part of the study. And for that sort of time horizon, we looked specifically at studies that use two pretty rigorous methodological approaches known as a regression discontinuity design and a randomized control trial. And again, as Beth mentioned, we were in part able to do that because there are so many of these studies out there. And then when we looked sort of into elementary and middle school, there were fewer studies. So we also included some other very rigorous methods, things like natural experiments or well constructed matching strategies that still included a comparison group to really give us that sufficient body of literature to draw some trends. And this is something that we can obviously share some more information about if you're interested, but also there is a full appendix in the full copy of the report that you got that shares a little bit more of our thinking about how we selected the studies for this review because we know there are so many great studies out there. Yeah, and I think the only thing that I would add to that is that we do recognize and believe strongly that all of the evaluations of state preschool programs across the board add to this literature and reaffirm the findings that preschool is in fact an effective an effective intervention for preparing kids for school and can have long lasting benefits, but that we were sort of able to narrow our focus and be all the more confident that the research we were including was rigorously controlled. Thank you. There is a question from Kristen Schumacher from the California Budget and Policy Center. Kristen asked if we have any thoughts about the most recent research released on North Carolina's preschool program. Yes, that is a great question. Kristen, thank you. And it really highlights a phenomenon of this literature, which is that it really is sort of growing rapidly and that the more we learn the more confident we can be that preschool is in the fact an effective effective program. I think that North Carolina's program, as Marjorie mentioned, is a high quality program. They've done a lot of work to ensure that that is the case and it is encouraging to find another another evaluation of that program following children longer into middle school is still finding lasting benefits. And I think that it's just something to highlight that we sort of had to stop at some point in order to be able to get the report out the door and that there will continue to be additional studies that come out of both extending the purview of evaluations of programs that we included and potentially evaluating new programs and that our goal really one of our goals with this report was to really equip folks who may not be deep in the research with tools to take a look at evaluations like that new one from North Carolina and understand how it fits within the broader literature. Thanks, Beth. This is Maddie. I would also just note, you know, as Beth mentioned, this study that Kristin asked about is so new, it's not actually included in our report. And for those of you who might be interested in taking a look at it, the study is a working paper that was released by Kenneth Dodge, Sunny Ladd, Clara Muschgen and one of their other colleagues at Duke University and we'd be happy if you want to follow up afterwards to direct you to the study if you're interested in taking a look. All right. Thank you, Beth and Maddie. Another colleague asked about the Head Start Impact Study and the Tennessee studies, right? These are studies known as the gold standard using randomized controlled trials. Does that make those findings more credible than the other studies you talked about? That is an excellent question as well. And it's one of the reasons that we wrote the report in the first place, which is just to help folks who may not be immersed in what a randomized controlled trial means and what it's trying to do, which is to completely randomize the selection of children either to receive a program or not receive a program and help folks understand that while that absolutely may be the gold standard, that it's just really incredibly difficult to maintain in the real world. And so the Head Start Impact Study is an excellent example of this where, you know, it was a randomized controlled trial and that is how it was implemented. And so children were randomly selected from waiting lists. Either you can attend or you can't attend. But what happened was that many of those children ended up then accessing a state preschool program or perhaps a high quality child care center. And actually in some cases even ended up going to another Head Start program down the street that wasn't a part of the study. And so what that shows us is just the importance of despite that randomized controlled trial mantle of really looking at the alternatives available to children. Because when you look at the alternatives, the alternative early learning opportunities available to children, you can answer two questions. The first one that you can answer is you can answer sort of holistically by implementing a Head Start program in this community. What was the impact on children's outcomes? But then if you consider the alternatives, you can also estimate and understand the impact of making that program available for children who either don't have the means or the motivation to access other programs. And we know in this country and in many states we're far from achieving universal access to high quality preschool across the board. And so we know that there are some children, especially perhaps low income or middle income children, who really do not have access to high quality preschool programs. And so it's important to understand not just the impact across the board, but the impact for those children in particular. Maddie, anything to add? No, I think that covers it best. Thanks. Okay. There was a question from Yasmin Grawalcock from the early edge California about workforce issues. And she asked if we could say a bit more about the workforce support that are offered in the four states that we discussed. So absolutely. Let me start off by saying, you know, there's great detail in the report called on the road to high quality early education on our website, which has also been published. But I want to start with in all of these states, they really focused on on building up the skills and the knowledge of the teachers and really making sure that their teachers or their lead teachers had specialized training, as I said, in child development and teaching and learning of young children. And the community and four year colleges were a critical link in both preparation and professional development programs, right? And so, so working with the higher education system was really important in terms of making sure that the teachers had that specialized knowledge that they need. And making sure that that preparation was high quality, right? That there was in that training that there was that carefully linked coursework and clinical training. And so that's something to pay attention to. And as I said, they you can't just all of a sudden overnight require teachers lead teachers have a bachelor's degree, you actually need to provide the infrastructure for them to be able to understand what's needed to have access to those programs and time because many of these, you know, current the current workforce are working. And so they need time to actually increase their knowledge and skills. There's also a big emphasis on providing support in the classroom. So not just that outside education or the degrees or certifications that they need. And I think Michigan had one of the strongest coaching programs that we saw, where there were what they called early childhood specialists. And these were all experienced early educators. They all had master's degree. And really their full time job was to go in and work with teaching teams, do observations of their classrooms and work with them formatively to, you know, to emphasize the things that they were doing wonderful to have suggestions on ways to improve their interactions with young children. In the state of Washington was interesting because it offered that onsite support, both to their preschool teachers and through their childcare providers, through their quality rating and improvement system. But it's that in classroom, really tailored support that is very beneficial to new and experienced early educators. I don't know what Maddie might want to give the example, talk about the apprenticeship program in West Virginia. Absolutely. So West Virginia has an interesting approach that they sort of pioneered actually almost 20 years ago. And it's a registered apprenticeship with the Department of Labor, wherein folks who are already working in childcare settings around the state are able to attend college classes, often at brick and mortar institutions. But they also have online options, you know, sort of in a nod to the very rural context in which some of their providers work. And they take this four semester course sequence that allows them at the end to basically get the state credential that they would need to be an assistant teacher in one of their state pre-K classrooms. And it also has built in a small sort of bonus or raise at the end, which doesn't, you know, get entirely at this question of compensation. But I think at least, you know, acknowledges the need to consider boosting compensation as you also ask for greater credentialing or degree attainment on behalf of your workforce. I also think a theme that I would just draw out of your comments Marjorie is realizing that really these states are all examples of places that have been able to make really multifaceted investments in the workforce. You know, it's obviously a very sort of challenging and complex landscape of looking at our early educator workforce and trying to make sure that their needs are met and that they, you know, sort of possess the skills and competencies that we would like to provide high quality care. And these are great examples of places that have again been very intentional and over time been able to build those systems to really support their workforce. And I would note that, you know, this report was published, but again, the work hasn't ended in these states and they're all continuing to think about ways to better support their workforce as they move forward as well. Right. Thank you, Maddie. I'm going to go on to the next question from Matthew Weyer who asks, is there much research or evaluation on K3 programs and practices for sustaining the pre-tay gains and how can policymakers work within the K-12 system? So that is an excellent question. And I think the answer is that there is ample research on the programs and practices that support high quality early instruction from kindergarten to third grade. There's also a revitalized movement to emphasize social emotional development in particular in those early years and really throughout school. And that I think is one place in particular where the connection and alignment between early instruction and preschool programs and instruction in early elementary school can really be strengthened and potentially help to really carry over the benefits of preschool into early elementary school and build upon them. Maddie, I wondered if you might want to also talk just briefly about the role of professional development and what we know about professional development in K through third grade as well as the role of principals and other school leaders. Yeah, of course. There's a couple of things actually I'd love to pick up on. So I think you're exactly right that professional development is an area where lots of states are thinking about taking action to help sort of reinforce this alignment or create some alignment between their early learning programs and the K-3 continuum. We've seen, for example, in state asset plans, a number of states have put at least a placeholder for thinking about this kind of joint professional development that can help not only educators, you know, teachers who are in the classroom, but also school leaders and center directors, folks who are really leading these programs, begin to, you know, deepen their understanding not only of what strong instruction looks like in elementary school, but also what it might look like in the early learning grades and being able to really build those bridges. You know, at L-P-I we did some research in California, for example, and we saw some very interesting examples of this, the Soprado Early and Academic Language, the SEAL program, was one example where there was an effort over the summer to have preschool teachers in a district and kindergarten teachers in that same district engaged in some joint professional development to help strengthen their skills around strategies to support multi-language learners, English-language learners. So I think we definitely are seeing some exciting work in the states there. I would also mention an example from West Virginia in thinking about this sort of pre-K onward alignment. We saw some very interesting work going on there. They actually created an Office of Early Learning that oversees not only their universal pre-K program, but also their elementary grades all the way up through grade five and have done a lot of thoughtful alignments of their standards, not only in sort of academic subjects, but also thinking about sort of holistic child development, social-emotional learning and how they can have standards for those that are not only applicable to pre-K, but also are aligned all the way into their K-12 system. So lots of really great examples, I think, and I'm sure folks on the line, either in their own states or in states that they work with, could also give us some great examples of how they're leveraging the K-12 system and professional development for their teachers to help strengthen those bridges between the early childhood system and their K-12 system. Okay. Well, thank you so much, Maddie. Thank you so much, Beth. We've come to the end of our time today. We want to thank you for your thoughtful questions, and we really look forward to additional conversation about this report in the coming days.