 My name is Christina Weiland, and I'm an associate professor at the University of Michigan. I direct the Equity and Early Learning Lab at the School of Education, and I'm a core faculty member at the Education Policy Initiative Center at the Board School of Public Policy. So thank you all for joining, and thank you especially for your patience with the tech issue today. We're all learning in this new format. It's our first virtual EPI event. We are really pleased today to welcome Dr. Yosha Kawa. He's a renowned expert in early childhood education. He's the Courtney Stale Roth Professor of Globalization and Education at NYU Steinhardt. He's co-director of the Global Ties for Children Center at NYU with Dr. Larry Aver. And most recently, he was also a member of the Biden Standards Education Unity Task Force. And we're extremely lucky to have him here today to hear about his really extraordinary work that he's leading to bring early education to young refugee children in parts of the world affected by humanitarian crises. So his project has been very generously funded by two $100 million grants, one from the MacArthur Foundation and one from the Lego Foundation. And it's in partnership with the Sesame Workshop and the IRC. I'm gonna hand it over to a hero momentarily, but just a few kind of administrative notes before that. I'm gonna acknowledge and thank the co-sponsors for the exciting talk today, the Education Policy Initiative here at the Ford School of Public Policy, the School of Education and the Equity and Early Learning Lab. We are recording the talk and it'll be posted on the website for those who would like to access it later. And Dr. Yoshikawa today is gonna talk for about 50 minutes. Hang on tight and stick around though. We'll have time about 20 minutes or so for Q and A from the audience. And please, as you have questions, type them into the Zoom's Q and A as opposed to the chat. That's the way that we'll be keeping track of them and consolidating them so that we can have a hopefully rich discussion at the end of the talk. So without further ado, hero, I'm happy to hand it over to you. And thank you again so much for being here today to share this important work with us. Thanks so much, Chris. It's a real pleasure to quote unquote be there. I really wish I was able to get there because I have lots of friends and colleagues both at the Ford School and the School of Education elsewhere at the University of Michigan. So thanks very much for this invite. So I look forward to your questions afterwards too. So I'm going to talk and I have to say I think when we arranged this, the date for this talk, it might have been just before COVID or around that time when we didn't quite realize that of course, a whole variety of plans, including everyday lives, livelihoods, routines, health and economic well-being would be drastically affected by COVID-19. So the title of this talk has shifted a little bit to reflect that. And so I'll be talking a little bit about parenting interventions. You may wonder why the shift from early childhood education to parenting. Well, I think many of you have experienced that or are experiencing that shift, which is that families are bearing the brunt of much of the pandemic when we think about education. So I'm going to talk first, a little bit very briefly about the challenges of COVID-19 globally for parenting and child development. I'm going to talk a little bit about, I'm going to be concentrating mainly on what an NGO partnership has done and is doing to pivot a little bit in terms of its ECD initiatives. But I also want to start with an innovative example because I think it is interesting at the national level by a ministry of education in a country that has greatly varying access to things like the internet. So I'll tell you a little bit about some innovative work to begin in Peru. And then I'll spend most of the talk talking about an initiative for the Syrian refugee response region in the Middle East and how two parenting programs have been developed in response to the COVID pandemic and how we're evaluating those. So the challenges of parenting and ECD during COVID-19 are things that I think many of you are experiencing directly. So and in low and middle income countries, I think these crises have been even in some cases further heightened. And these are crises of economic and food insecurity, job loss, forced migration, mental health and wellbeing challenges that are widespread, including evidence of increases in risk of domestic violence from a variety of countries around the world. Closing and disruptions of existing ECD programming, which include childcare and early education, but also in-person caregiver and parenting support and child protection services. And I would also include services for kids with disabilities here, which have been severely affected. There are, of course, widespread lack of access to the internet, not only in the United States, but certainly in many, many low and middle income countries for remote or distance interventions. And when programs open, the challenges of social distancing and a developmental period, when that is just not the norm for developing young kids. So I'm gonna start with a national government response, which has been quite innovative. And I think one of the models in Latin America, which is work by the Ministry of Education in Peru. And just to give some context, Peru is actually an upper-middle income country, but has a great deal of inequality. And so you see that very little internet access in rural areas, about 40% access in urban areas, and even in the capital of Lima, only half of the students have internet access. What the Ministry of Education developed was daily broadcasts with set times for different grades and age ranges. And this is on the national radio and TV networks. And these follow a curricula that the Ministry of Education has. And so they kind of meet some of the learning standards for the country. For pre-primary, they do include Plaza Sesamo, which is the Latin American version of Sesame Street. And they distinguish within the early childhood development period between learning activities within early childhood for the first two years of life, age three, four, five, and then pair grades later in primary education. This is also available in terms of the materials online. And any of you who are interested the program is called Aprendo en Casa by the Ministry of Education. And so here are some examples of a tweet about the first day of class for preschooler, these messages around COVID, around washing hands and in both kind of rural contexts and urban contexts and everyday play kinds of activities. Their schedule, importantly, has these virtual and remote models that they set up very, very quickly. And so they were on, as of, I believe, May. And they recognize that there will be a lot of variation in learning by the time of the next school year. And so they have planned individual child learning evaluations and efforts to provide instruction at the child's level of learning at the beginning of this current school year. So that's an example very briefly of what the Ministry of Education has done in a middle-income country context. I'm gonna spend most of the rest of the talk talking about a particular partnership where we are involved in the research and evaluation, which is the partnership of the International Rescue Committee, or IRC and Sesame Workshop in four countries in the Middle East. And as Chris mentioned, this was an awardee of a award mechanism at MacArthur Foundation called 100 and Change, which was a wide open competition to fund a single proposal promising real and measurable progress in solving a critical problem of our time. So there were many applicants. There were about 18 months of lots of work to put together this proposal after this partnership was identified as a semi-final, one of the eight semi-finalist proposals. And the argument for this solution was that there's a real gap. Despite the worldwide increase in investment in early childhood development that was driven by the strong economic and neuroscience and evaluation evidence of the past several decades, that had really not hit the humanitarian sector. So in an article with Katie Murphy and Alice Wormley, we looked at national refugee response plans for the 30 or so countries that had the largest humanitarian crises. And less than half mentioned early education. And less than a third mentioned anything about parenting or supporting parents. And so there had been a real gap in investment or consideration of early childhood development services beyond those for survival. And that, of course, on one level makes a lot of sense because the humanitarian sector is addressing crises of migration or manmade or armed conflict or natural disasters where the first priorities are survival. So their health, shelter, food, clothing, so those basics. And so early childhood development beyond that, which is really about learning and thriving and supporting children's development across domains beyond survival had really knocked in a priority. And it is still not widespread in the official refugee response plans, which are one indicator of policy level acknowledgement of priorities. So the rationale we made for an audience that was perhaps not familiar with the early childhood development field, it will be familiar to many of you and which are the Harvard Center on the Developing Child messages translating the science of early childhood development. Talking about how brain architecture is formed in the earliest years of life and is supported by the interaction of genes and experience. Talking about the critical role of responsive caregiving and serve and return interactions between adults and infants toddlers that build the connections that build brain architecture in the early years and how stress, such as the stress that can be experienced both before a flight, during flight and in the host's community as adjustment to after migration occurs. All of these can foster levels of stress that can affect early children's development. We also built in the rationale around the economic evidence from work by Jerry Bearman, Florencia Lopez and others. Talking about the societal cost of not investing in high quality pre primary education and parenting interventions and noting that these costs of not investing are higher in the context of the lowest access. And so therefore the comparison or the counterfactual matters for investments and therefore in the humanitarian context where there's been very, very low levels of investment that there may be a rationale for doing this. We also highlighted that there may be a whole variety of platforms and services on which to build early childhood development services and they include not just childcare and early education but social protection programs like cash transfer or cash grant kind of programs, nutrition programs, health, child protection programs and sanitation and hygiene programs all of which are part of the picture when it comes to humanitarian crises and services. So that was some of the rationale for the investment. So what was the actual solution? I'm going to present the video that we'll give a little bit of the kinds of services that we proposed at the time of the finalist presentation. So we'll see if this works and hopefully the sound will, and the video will, well, sorry. Sabah Alkhair, how are you? Our early childhood program begins in the home whether it's a shelter, a tent, a crowded apartment, it doesn't matter. How are you? The most important thing is that the children need to be with their parents. The first caregivers with whom they will build trusting relationships and learn new things in order for them to be able to build knowledge on the long run. We give them activities to promote, reading, learning the alphabet, counting, a lot of language skills. We empower the parents with skills to support their child's development. They can play with them using objects that they can find in their homes. We show them how to communicate with their children frequently in a way that promotes praise. Bravo! Most of the parents that I work with, when we first meet, they describe their role as shelter provider, food provider, as the one who's making sure that their children survive. Yet with time, they start engaging with their children and they would say, I used to do this in Syria, but I was not able to do it anymore with my kids. Thank you for helping me. So that gives you a little bit of a sense of where we are. So that gives you a little bit of a sense of where we are. Where the services were actually before the start of the initiative. So it gives you a sense that there was already a partnership between Sesame Workshop and the International Rescue Committee. And so in fact, the service model is both the development of a mass media program. That would be the first version of Sesame Street in essence. That would be tailored for the Syrian refugee response region in four countries in the Middle East. Then direct services that are both caregiver or parent-focused services and then child or early care and education-focused services. And then our research and policy agenda for partnerships. And I'm not going to talk very much about the policy partnerships, but there are extensive collaborations now that have been developed with the ministries of education, health, or social protection, depending on the country in this initiative. So these are the service areas as they existed just before the time of the proposal. And they're roughly still, they've expanded in Jordan in particular, but these are still roughly the service areas currently in these four areas of the Middle East, which is called the Levant area of the Middle East. So I'm going to quickly go through the mass media and then talk through a little bit of the parenting interventions. So Auckland-Simsum's first season launched in February of 2020. And it is set in a kind of a urban neighborhood that's a little bit of a conglomeration of kind of an area that might be a mid-sized city or town in Jordan with aspects of some of the other countries. Basma grew up in this neighborhood where his job is a newcomer to the neighborhood, but it's never really stated where he's from. But the dialects represented reflect a variety of dialects within the four countries. And there were lots and lots of discussions and input from language experts around how to represent the variety of dialects among migrant populations, host community populations, and diversity in the host community in these four countries. The other unusual decision that was made to really focus on one area of young children's development, which is emotions, coping, and self-regulation strategies for the first season, and more broadly around social and emotional development for the further seasons. And the second season is now on the air. And the third season is being developed for this next year. A multi-generational cast was surrounding these new characters. There's other new characters. There are live human actors. I didn't quite know this, but there are humanoids, animals, monsters. There's a whole categorization of sesame workshop kinds of characters. These were developed with a lot of input from local residents, experts, parental preferences, and formative research, but also an organization called Jordan Pioneers, which is a leading children's media organization in Jordan. The second half of every 26-minute episode is an actual variety show with a live audience of children and celebrities from the region and situations that explore the same emotions and coping strategies that are in the first part of each episode looks like a kind of traditional format with a mix of muppets and live actors and some animation. So in our impact evaluation, we considered a couple alternatives, one of which is to actually study naturalistic home viewing. And that would have been an ambitious project to actually develop an intervention to increase home viewing and use that to leverage the impact of home viewing on children's outcomes. Because of the uncertainty of those kinds of interventions, we went for the other option, which was to screen entire seasons in preschools with about 26 minutes of a preschool day devoted to particular episodes and to screen episodes daily over a three-month period. In this case, in government preschools in Jordan with relatively high proportions of refugee origin children. And we considered two potential control conditions and alternative mass media children's TV program, a whole set of 26-minute episodes, which we actually created with nature content. But the Jordanian Ministry of Education actually came back to us and said that they really preferred us to contrast it with their business as usual preschool curriculum without an additional alternative program. So that's what we are planning. We've done a lot of measure development around emotions, which are really complex and, of course, culturally specific. And so developing vignettes for emotion regulation and coping strategies and piloting those in Jordan. And then, of course, the preschool shut down. So we're delayed. And we've had to increase the sample size of schools to revise our effect size accounting for the fact that by the time the RCT starts, more people overall, more kids overall in Jordan will have seen the Auckland Simpson TV show. Auckland Simpson means welcome, Sesame. And so Sesame Seeds was the title of our initiative in the proposal, but it is called the Auckland Simpson. Then turning to the direct services, there are both programs for children, which are child care preschool. They can be relatively non-formal kinds of child care programs. Some might be satellites to multi-service centers like women's empowerment centers or health and nutrition based clinics. Or they can be full-fledged preschools, such as you saw in Lebanon. Those were in the Bacaw Valley in Lebanon in the video. But I'm going to concentrate a little bit on programs for caregivers, which also include both kind of high contact of what we call high contact and low contact programs. And the high contact ones are more in the area of home visiting, which is quite familiar to those in the US. And the model that was developed and had already been piloted at the time that the proposal was put into MacArthur was an adaptation by the Arab network on early childhood development in regional NGO based in Beirut of a program called Reach Up and Learn, which is focused on parenting responsiveness and stimulation and early development for six months old to 36 month old. It had been initially implemented in Jamaica probably starting about 30 years ago. And in a 20-year follow-up of a small-scale RCT, participants who had been infants and toddlers in the full program showed increased earnings, educational attainment, decreased depression, and crime. So the Reach Up and Learn program has been adapted for, for example, national use in a program called Kuna Masen through. And this version was adapted by this NGO. And it takes the form of bi-weekly visits in a 12-month program. And in the context of these countries, the IRC has implemented it based in different kinds of workforces. But in Jordan, added to health and nutrition messages. So we did a pilot study of participants about 200 caregivers and about 40 home visitors. And some of our qualitative quotes included they were uniformly positive once they got into the program, but they did admit these caregivers that this was initially somewhat of a bizarre program. That this idea of someone coming to the house and playing with a very young child was just not a norm in this context. But at this point, which was after about five months of home visits, there were some consistent themes. And this is work by Ifrah Magan and Salma El-Ka'uri. Ifrah is on the faculty of social work at NYU. And some consistent themes around the salience of and attention to child learning. But there were a lot of kind of aha moments about not realizing that children this age were doing the kinds of things that they were doing in the home visits and capable of learning from them. So this idea of being interested and finding out what are the hidden talents or skills of kids that I don't know of caring about that child more specifically. And many of these families had other children. Now I do. I sit with her and I teach her a song, for example, after a while she can sing it on her own. The caregivers also made some suggestions for the program, such as more visits or extending to children older than three. Leaving toys behind, because this was in the original model, was about rotating toys. And so toy making was added as a stronger emphasis in the revision of the program and some of the larger context around addressing financial and security. And for those of you who know the context of Jordan and this area about an hour away from the capital, it is a very expensive country. And so the cash grants, the limitations on employment for refugee populations made for high levels of salience of financial insecurity. The workforce supports that were raised by home visitors in implementation research included attention to the caseload, the stress of conducting home visits, the schedules of families versus home visitors, and the request for more in-service professional development to support their needs and for peer support. And so partly in response, starting in early 2019, the REC started implementing continuous quality improvement approaches, which those of you familiar with kind of Tony Breik and the Carnegie Foundation for the Advancement of Teaching, this is that very model applied for the first time to ECD in a humanitarian context. Now I'm going to talk a little bit about what happened after COVID. And that is that, of course, home visits stopped. And so with the lockdown in Jordan, this question of how could parenting supports be provided in a context where home visits are no longer possible. And so this week, we are training data collectors for our first program evaluation post-COVID, which is a RCT on the phone-based reach up and learn. And so this was created and adapted in a very fast pace of probably something like five or six weeks by a team from the IRC who specialized in early childhood development. And this kind of workforce of community health volunteers who are stipended and majority Syrian in background were trained now to provide a model on the phone, which includes features that I'm certainly now hearing are common features in multiple phone-based models in various parts of the world in Latin America, in South Asia, in the Middle East, which includes a check-in around COVID, providing COVID-related information and strategies for prevention and health and hygiene, a caregiver well-being check-in, and then getting to activities kind of third on the list, which is, in this case, the reach up and learn activities that are feasible to accomplish in the home. This is still targeted to the age range of the child. It is still targeted to the children's learning level. One feature of reach up and learn is that the sequence of activities and toys is targeted to what the child can do. And it's too easy for the child, they move on to the next level. And if it's too difficult, they move back. And there is a whole sequence of these. This is now a selection. The majority of the activities were retained, but they had to be adapted. And there are certainly limits, which I'm happy to talk more about on the phone, which is that you cannot actually do a lot of demonstration. You can't rely on, this is not a context where there's a lot of FaceTime or anything like that. And even with that, to hold a child, do a certain activity with the child, talk to the caller at the same time and get feedback. These are the kind of normal things that go on in a home visit. Those are just become nearly impossible when it's on the phone. But the activities themselves are retained, and this is currently being implemented in Jordan. And our impact evaluation will occur in one area at the very north end border of Jordan with about 80 home visitors and their caseloads randomized either to the health and nutrition kinds of content and the COVID information or health nutrition and reach up and learn content. And this gives you a little bit of a sense of some of the sample activities by visit that are within three month age buckets. And this is a sense of the theory of change. One of the things that we are going to be measuring is, which we think is one of the first times in the ECD field is to actually quote unquote observe quality, the observed quality of the calls. And of course, you're not observing, you're listening to them. So these are audio recordings. And we have a draft quality measure developed by members of our team at Global Ties. We are hypothesizing some parent well-being, so depressive mood, anxious mood, levels of stress, engagement in parenting activities related to responsive care, reductions in harsh discipline, engagement in the activities. And because of the phone-based low contact nature of this and the fact that certain aspects like feedback, live feedback are less prominent, our child development outcomes will measure them, but they're exploratory. And for those of you who are used to the kind of idea of registering clinical experimental evaluations, we will not be registering the child development outcomes as primary hypothesized outcomes, but mainly thinking about the parent process kinds of outcomes. We had done a pilot sample when this was planned to be an RCT of the in-person home visiting. And we did test a lot of the measures, which we are now going to be gathering through phone surveys. So we did a lot of kinds of measures, qualitative quotes that you already saw, observational measures of quality of home visits, direct child development and survey and a whole array of survey measures and did a whole variety of these validation analyses. And after iterative revisions, we were able to get to some pretty good. We're pretty happy with most of the survey measures. Here are some of our findings around parenting stress, efficacy, anxiety, depression, well-being, and sleep, which were related to each other in hypothesized directions with a pretty good fit. But some of the other areas required some revision. So our initial child development measure adapted for this context didn't quite capture some of the domains that we think were targeted in a kind of fine grained enough way. So we're moving to a longer child development measure when we get back into the field. But unfortunately, for a phone survey, we can't really do a direct assessment. And so for the phone survey, we'll be using a thing called the ages and stages questionnaire for parent-reported child outcomes. Again, those are exploratory in the phone-based version. So that's that evaluation. I also wanted to tell you a little bit about a text messaging intervention that was also developed since COVID for an even wider scale. And again, by the same team at the IRC. And this is designed to benefit not just IRC beneficiaries, but beneficiaries of a large array of other NGOs in the four countries of Atlanta and Simpson. And similarly, a mix. But these are text messages. So there's not much in here about a kind of interactive back and forth around well-being and coping, for example. But things like ECD awareness messages, COVID awareness messages, and then activities to do with their children at home that are then based on the child age, covering different areas of development, and some activities, including pictures or links to Ahlan Simpson videos. This is an example of what some of the messages might look like. And the idea that some might be accompanied by illustrations, some can be accompanied by audio messages, and some can be accompanied by short videos that demonstrate what an activity looks like. Our theory of change similarly here is that we are and we're probably a little more on the left of this theory of change, because it's even less intensive than a phone-based model where you're getting a phone call once every week or so from a community health worker. In this case, you're getting texts. And so our first question that the IRC was interested in was in thinking about different implementation factors, what would prompt the highest level of engagement with the text? And that is the first purple box in the middle set of intermediate outcomes. So it's kind of a first-order question. The intervention is still being optimized. And so the question that we're working with the IRC to answer is, before we get to formal impact evaluation, which types of messages maximize caregiver engagement? And we're measuring caregiver engagement by working with the texting platform NGO, called Bayamo, to split messages in two so that there is an introductory message about what the topic is. And today, we have information about this kind of activity. Would you like to see the activity? And then the caregiver or someone who has the phone and the family would have to actually press something on the phone to get the rest of the message. And so we think of that as a behavioral engagement indicator. And for those of you who know WhatsApp, the reason we're doing this is because on WhatsApp, you can get a first blue check, which is that the message was received or delivered. And then the second one, which is that it was opened. But beyond that, you have no idea whether a caregiver has actually engaged with it. So we have that measure of caregiver engagement, which is the first order outcome. And we have this question about whether that differs across different formats, whether a text is accompanied by an audio message, a text message only, a message with a picture, or a message with a video. Other questions like the time of day, and then, of course, qualitative questions about how these texts were and messages were received and what kind of meaning making happened, what were some of the barriers to perhaps engaging in it, did the activities make sense within their daily routine, those kinds of questions. But to optimize, and we're optimizing to this notion of behavioral engagement with a text, the IRC had a first set of questions, which was about the modality of the attachment. And also whether receiving awareness messages in addition to versus receiving an activity message only without a kind of ECD awareness message was another question. We're still considering some other questions, like messages in the morning versus the evening, and receiving reminder messages about the last activity. So to test this, we're using an approach called multi-phase optimization design, or the most design, which Linda Collins has written a couple of books about. She has just moved to NYU from Penn State. And this is an approach to test multiple intervention components and try to understand what is the optimal combination of, quote, unquote, active ingredients before you do a kind of summative impact evaluation. And what it basically is, is a multifactorial design with two condition factors. So each component is required to have a binary kind of on, off, or a that component versus an alternative component or comparison. And then you can test multiple ones simultaneously without this kind of traditional notion of needing a very large sample with a multi-arm experiment. And that's because it's a full factorial design. So to give you a little bit of a sense of this, this is what a four research question, two by two by two by two factorial design would look like. There would be 16 cells. And in this case, there's 25 per cell and a total sample size of 400. And you can see that right now the shaded versus the white is the comparison between a written text only versus a voice note accompanying the text. But you see the columns across incorporating three other research questions, which are video versus no video, images versus no images, and an ECD awareness message versus no ECD awareness message prior to the activity. So that in total gives you 16 cells with 25 per cell. And the comparison of this research question of written versus voice note of 200 versus 200. Then for this alternative condition moving over to the second experimental condition column for videos versus no video, you would do these blue cells versus the white cells. And that would again be 200 versus 200. So you can see that with a full factorial design like this that you have the equal power to test every single condition with a relatively modest sample size of 400 instead of something that might involve a five arm experiment, for example. So right now we are in the final stages of defining the components for the pilot to test the feasibility and the use of the BIAML platform for this most design. The main phase would be an end of 2000 per country in Lebanon, Jordan, Iraq, and Northeast and Northwest Syria. And for a phone survey follow up, we would be looking at outcomes beyond that one check versus two check versus a text back engagement measure to include some of these things like self-reported engagement in the activity. The most phase can be repeated to answer emerging or new implementation kinds of questions. So if the qualitative data show that there are some other factors that might be incorporated as active ingredients into a text-based series that could be done in another most phase. But then the idea is that per country there may be a quote unquote optimized program that could be the focus of a traditional RCT kind of impact evaluation, which is planned for 2021. So that gives you a sense of the two current distance or mobile interventions that we're evaluating currently. We do plan to get back into the field in both this initiative and one in Bangladesh for Rohingya refugees when the reopening permits and with our research partners on the ground and with Sesame, the IRC, and BRAC, which is the NGO we work with in Bangladesh. So just to wrap up, some cross-cutting implementation issues that we're observing from not just these initiatives but multiple country level efforts, which is a new integration of health, nutrition, protection, and education. And I think this is absolutely also true in the United States, which is that schools have a greater urgency to provide multi-service community school kinds of approaches, for example, when communities are varying so hugely in their patterns of stresses, of job loss in international contexts, migration, food insecurity, and the withdrawal of reduced kinds of services and community supports. There is an emerging quality measurement agenda, which we're participating in with work on evaluating phone-based interventions in the Middle East and India and Bangladesh. We're also coordinating with work in Latin America with Inter-American Development back on some of these same kinds of issues to compare notes around measures and how we think of research and evaluation, both on the monitoring and evaluation or implementation side and the impact evaluation side across countries. One theme that's come up a lot is the callers' well-being and stress. So first of all, there is rescheduling going on. During lockdown, I think during the most extreme lockdown, caregivers and parents were the most available as there is hybrid or partial reopening that changes. And callers' well-being and stress is a huge issue because everyone is under the situation of COVID. Everyone is dealing with this on top of the usual workforce stresses of working with families, but working with caregivers who have heightened levels of stress and basic needs themselves in many cases with callers themselves often from refugee backgrounds. For example, in a BRAC program called Besides You for the Rohingya and Cox's Bazaar, these are some of the themes that are emerging. So there's a lot of well-being of the workforce and support for the workforce that's required. There's a lot of variation in household control of who controls the phone, who answers the phone and the need to kind of think about fathers, grandparents, extended family members as households are facing the stresses of COVID together and often with kind of a new way to think about how the family unit is functioning in the context of COVID. All the complexities that we know from the United States and the variation of reopening levels and re-closing and then how to address perhaps these increases in the variation in learning levels when children return or children don't return. So these are some of the huge challenges. I want to thank you. Look forward to your questions and thank our partners and the Funders the Lego Foundation, the MacArthur Foundation and our partners BRAC, the IRC and my colleagues at the Global Ties for Children Center at NYU. Thanks so much, I think, rather than, OK, that's what I'm supposed to do. Oh, I think I'm muted. I'm not muted. OK, I was saying thank you, heroes, so much. This is the part where if everyone was unmuted, it would be applause. So we can impute that at the moment. But we want to remind everyone that we do have some time for Q&A for about 20 minutes or so. So please feel free to go ahead and enter any questions you might have into the box. And I will certainly kick us off to get us moving with a question that I had as I heard this terrific talk, which was around how in working with parents from such different cultural context, how do you think about this idea of what makes a good parent and how it differs across culture as you're thinking about how to reach them through phone and also the in-person work that was happening before COVID hit? Right, so that's a great question. And it is a theme that did come up in some of the qualitative work. And so I do think it is absolutely vital to do the kind of work to inquire about daily routines, about goals, values, and socialization practices. And so one area, for example, that comes up as a fairly consistent theme across some qualitative work in both Lebanon and in Jordan is the notion of faith and religion sources of coping and also of socialization and also of goals for children and goals for aspects of education that incorporate faith and religion, for example, are themes that come up. So I think that's just an example. But I think our approach in our qualitative work is to, in the traditional work on culture and human development in early childhood, to consider the daily routines but also the values and goals of parents. And so I think that is absolutely affects everything from the workforce and the selection of the workforce to the training and the content of interventions. So that's absolutely, I think, an important piece that emerges. And I think it's, yeah, so COVID has driven a kind of a widening of the sense of the kinds of supports that families need. But I think what you're raising is also the kind of cultural aspects. I didn't have time to talk about the Bangladesh program that Brock has developed, but that was really, their curricular and their approaches were based on pretty extensive qualitative work with the, in the early months of the Rohingya influx around the kinds of socialization practices with kids, the arts kinds of activities and the actual arts, the chants, the songs, the visual layout and aspects that these families were used to in their homes in Rakhine province in Myanmar. And so those were some of the early learning centers that Brock constructed, about 300 of them actually incorporate Rohingya arts and look more like Rohingya homes in Myanmar than there are, of course, these very, very crowded dwellings in the camps themselves. So these are some of the only places that actually look like traditional Rohingya kind of arts and those were kind of community members who did that and they've incorporated a lot of that into the content of their curriculum and the kinds of stories and the kind of moral lessons that lie behind the stories and the stories that are set in the kinds of landscapes and countryside of their origin. And so that's an interesting part of the kinds of story curricula that are in the pre-primary two to four and four to six year old programs that we're looking at in that context. So we are getting some Q&A. So I will read for our folks who have joined us. So this is from Professor Michelle Bolino at the University of Michigan School of Education. She's asking, can you talk a bit more about how you are supporting adults and these children's lives to work on their own coping strategies in such a difficult context? And so somewhat related to the last question as well. Right. So that's really important. The tele-counseling aspects of these programs, we're finding vary, but there's a near universal need kind of that comes up in calls for addressing the various levels of both basic needs and stress and kind of coping. And so the approaches that we've found ourselves kind of involved with partners are things like tele-counseling kinds of approaches. One of our research scientists founded the very first national crisis hotline in Bangladesh. And so she has provided input to BRAC on the approaches to training phone callers to integrate aspects of tele-counseling. Both in Bangladesh and she's doing some of that work in India and also translating some of that for the IRC as well. Because I think this was absolutely something that you could not go straight to activities. You have to actually think about how to do the open-ended questioning, some of the features of reflective listening and active listening that are really the bread and butter of both crisis intervention and how that would enrich these conversations. So yeah, so those are some of the aspects. But that does require its own sets of training and focus in the training. And Bradley, what are the biggest differences you're seeing in terms of putting an ECD program like this in place in a humanitarian context versus a more stable non-crisis setting? So thinking of setting aside COVID, but thinking of some of the other contexts that you've worked in low and middle income countries. So yeah, the Alhamban Simpson initiative started quite a few years after the beginning of the influx of Syrian refugees into these countries. So in 2011, some of the first big waves of fleeing the war and conflict in Syria, were driving large numbers of Syrian refugees across the borders. So at this point in the areas that the IRC is serving, that influx has lessened certainly in terms of numbers coming across the border. And so many of the children in these early childhood programs in Alhamban Simpson were actually born in the new country, right? So post-settlement. And so we are a little bit more in the more stable context of the humanitarian situation in that context. In Bangladesh, the Rohingya, the big influx started in the summer of 2017. So that's much more recent and very, very concentrated as far as the geographic space because this is really the land that is kind of right next to Rakhine province is this district of Cox's Bazaar in Bangladesh. And so these 34 refugee camps were set up in very close, they're pretty much contiguous. So it's like one great big refugee camp but they are divided. So there it is more recent but this distinction between kind of immediate humanitarian response and long-term settlement and drove on Alhamban Simpson this notion of both high contact and low contact interventions. So when you're, and the idea of the low contact intervention. So reach up and learn in the, this kind of idea of home visiting is actually relatively high contact intervention. 12 months of services. Similarly, a full year of preschool will be considered a high contact intervention even though in some other contexts it might be kind of considered fairly low intensity but within this sector, this notion of can we provide like a full year of preschool? Well, that's not really a certainty, right? So yeah, so the low dosage interventions are things where actually the total number of hours might be quite a bit less than 100 hours for example, of contact with the family and those were developed to be a little more responsive to the immediate crisis kind of situation where families might be on the move. We also think that these phone-based interventions are also appropriate for the phone being the lifeline in many, many cases during flight or when you're really not yet settled to try to keep in touch with caregivers and parents. And so to that and just a follow-up question what you're learning now from these interventions because of COVID, do you see them having lasting power? This is somewhat, I think Milagro's to your question. I think it kind of follows on here that in thinking about mass media strategies in middle income countries, do you see that some of what you might be learning now might last beyond the current crisis? That's the big question for the entire field, right? So we don't really know. We are just starting to see some of these patterns of convergence, but there's not enough studies in humanitarian context of follow-up. So in Ghana, work by Sharon Wolfe, following up teacher professional development in preschools in Ghana has shown and suggests that you really do need to kind of sustain investments or think about some of the other approaches to sustaining effects. And we have one other kind of pre-year follow-up of an early home visiting intervention based in a social protection program in Columbia. But we don't have many of these kind of medium term follow-up evaluations. So we know very little. And so we always have a sense that the counterfactual, the comparison matters. And that what you focus on during the intervention matters. We are just at the very, very beginning of starting to learn some of these things in the low and middle income countries, partly because we've been often restricted to immediate post-test kind of impact evaluations. We do cope and plan to start a long-term longitude no study in Bangladesh as part of the LEGO initiative. Okay, we have some questions from Gloria Yeoman Maldonado who's now at the University of Texas Health Services Center. She was a postdoc here before this. And so Gloria is asking a couple of questions. In your original media plus home visiting intervention was the media about socio-emotional development and the home intervention focused on more school readiness outcomes? First question. Oh, interesting, yeah. So the mass media program, just to be clear, it has its own separate impact evaluation just focused on the mass media. And what's interesting about the social-emotional aspect is that as the seasons, and there's going to be a total of four seasons, we think that starting with emotions does make some sense. And that then starting to build in some other domains of development, like social cognition or perspective taking and problem-solving, those kinds of things we think are going to raise some interesting questions about the breadth of social-emotional outcomes that might be targeted across multiple seasons of a program. And we're currently exploring even the idea of would there be a, would we think of the impact evaluation not focusing on a single season but on a kind of best of taking bits of multiple seasons that cover a broader range of social-emotional skills? So that's an interesting question around domain breadth in a mass media intervention. The home visiting program is, first of all, targeted to younger kids. The kind of sweet spot for Sesame programs traditionally has been the kind of four to six-year-olds, even up to seven-year-old age range. And so that's where most of the complexities of the plots, the kind of humor, well, there's lots of different kinds of humor as we all know in these programs. And for anybody who wants to watch this, you should just Google Aufwand Simpson. And I was trying to get a subtitled, there's not much subtitling in English, as you might imagine, but I was trying to get some to put into the presentation. And that's not widely available quite yet. But yeah, so you can take a look. But yeah, the home visiting programs are for younger kids and focus on these traditional areas, somewhat traditional areas of kind of responsiveness and learning activities that are meant to foster both outcomes on the social-emotional end, but also through some of these more warmth and responsiveness kinds of mechanisms, but also some of the cognitive aspects of development through this stimulation. I'm going to keep going with Gloria's question. I think she may be working on some interventions in Texas and have some questions that are targeted towards that. But for the phone-based intervention, you mentioned the activities were adapted to skills levels. Can you talk more about how those skills levels were assessed? They are assessed fairly informally, because on the phone, you can't really do direct child assessments. And so this is not the kind of independently assessed child outcomes. This is more reported by the parent and whether they were able to do it. Because again, the home visitor is not there to observe whether the child can do it or not. So that's a very good question for how we think about this in phone-based is like, how detailed do we have to get in terms of asking the parent about what a child was able to do or not able to do and why they were not able to do it or the context in which they were not able to do it or able to do it, right? Or how do you get to this notion of like it's kind of quote unquote too easy, like kid gets bored, kids can get bored for different reasons versus something that's difficult, right? And these are things that are trained, part of the training for home visitors, but I think are much more challenging for home-based interventions. Yeah, so that key part of the program is again, something that we're not sure is will be there in the phone-based. We can get a sense from the full recordings. We hope to get a sense of what these conversations are like and the probing around the context of learning. And I'm going to combine two, one from Gloria and one from Aditya Benjamin, which is around this scalability question, right? So this is around what are you hoping to learn in terms of cost and to inform the next stages of scalability? Yeah, so we're working with our NGO partners around the approaches to costing. And so those are things that we are just starting those costs studies, paying attention to both kind of startup and maintenance kinds of costs and the usual kind of range of costs that might occur. These are certainly in a lot of the NGO planning and their finance processes and budgeting processes and those kinds of things. The scaling effort is kind of going on simultaneously, which is that these are kind of contacts with NGOs and ministries in different countries. And it's a very important part of the initiative. And so one of our future impact evaluations might be in the context of scale. So as a program is scaling up and might be implemented by a ministry, for example, rather than another NGO or by the IRC, that there may be opportunities and expansion to kind of leverage that for an impact evaluation. So that is a thing we'll be exploring. As reopening happens, and of course, a lot of the scaling also got everything is paused because every NGO and every ministry is struggling right now with just keeping up with what they were doing before COVID hit. So the causal inference crowd, and because we only have another minute or two for questions, I'm going to ask a detailed one also from Aditya. How do you tackle the issue of attrition in the most design, especially with a smaller sample size compared to an RCT in general? So when you have like those ends of 25, are you seeing that? And what do you do? Because this is a new design to me in terms of the framing. Yeah, that's a good question. Of course, all of those issues of attrition still hold as a threat to the design. And just to be clear, that was just for an example to illustrate what that looks like. Our samples per country will be 2,000, roughly 2,000. So they'll be relatively large. And the intervention is relatively short. Yeah, so we're thinking of this is really something like three or four weeks of texts. And that's still not fully decided because it's partly whether the frequency of texts twice a week versus every day, every other day. These are still some of the questions. And that would also partly determine the ultimate length of the most phase. But these are not going to be long interventions. They're going to be nowhere near 12 months of texts. And then finally, the last question. Can you talk a little bit more about the process that you went through to contextualize the Sesame Street intervention to the local culture and the need and why Sesame Street? I know they were already doing some work, so that was part of it. Why is it particularly well suited for the needs of these communities? So Sesame Workshop had developed two programs for the Middle East. And Iftaya Simpson had been developed in Jordan. And so the nearly important pioneers had worked on that show. And so they had experience already in the country, but not specifically around issues of refugees or migration and some of the kinds of particular developmental focus that that might bring or content. So of course, this is not something that should displace locally developed media. And this is a partnership on the mass media side between Sesame Workshop and Jordan Pioneers, which is based in Amman. So to be clear, this isn't literally folks from New York coming in and translating a show that's already been developed. On the other hand, these are both INGOs. And I think in parts of this initiative, we're also working with much smaller local INGOs, for example, in the context of Bangladesh. And they do bring a different focus. And so there we are working on integrating, for example, into the Bangladesh study for adolescent mothers, a youth empowerment perspective and the development of youth organizing and leadership kinds of skills through a little bit more of a participatory action research model that might be embedded within our longitudinal study. There, we are potentially partnering with a different kind of NGO that's not coming from outside the country that really arose from within the Rohingya population, for example. So I think the humanitarian sector, of course, is often dominated by a large role for the international NGOs. But they are also often working with local NGOs in a particular context should they exist. And it turns out, for example, that in Cox's Bazaar, because of the prior waves of migration, going back 35 or 40 years, that there are NGOs that are local that arose within the context of those prior waves of migration. And then when the huge influx started happening in 2017, these NGOs also were playing a very important role with things like setting up new camps, setting up things like the network of health clinics and playing a very important role so that it wasn't just the so-called very large INGOs that were providing services. And that's still true there in the camps today. It's amazing work. I also just very quickly, how big is your team just to give us a size of do you have any idea? Like how many people are involved in all the work you're talking about? In New York, we have about, I think, 17 or 18 full-time folks working on these two initiatives on the early childhood side. And then we have a variety of partners, researchers, and consultants from working on site. And then in terms of data collection, the IRC actually hires the data collectors on the HansemSim initiative. And then Bangladesh Innovations for Poverty Action and their Bangladesh office is our data and research partner. But we're finding that for very complex issues like translation for the Rohingya language that we're kind of reaching out to further networks and resources and very local expertise. So we have a, it does feel like a small army. I'm not used to this scale. So it's been a lot of challenges. But we have a fantastic managing director. Alice Wormley is the co-PI Larry Abers involved. And we have a fantastic managing director and Lizzie Goodfriend, who has lots, years, and years of experience in international development project management. So that's been, yeah, she's been a lifesaver. Well, it really is extraordinary work. And we're so glad that we're able to sort of have you here for this talk and thank you for your flexibility as we invited you to come in person and to be willing to go with this virtual format. We're going to be all following your work closely from here onward. And we'll find a way to hopefully update the Michigan community as you learn more. And we want to thank all of those people who are able to participate today. You're patients with our technology issues at the beginning and for the questions. We will be having more events this year through EPI. We actually have one coming up on November 19th, in which we'll be hosting a panel of early literacy experts, including Dr. Nell Duke here at the University of Michigan School of Education. Pamela Mason from the Harvard Graduate School of Education, journalist Karen Chenoweth. And I'll leave it now with the executive director at Michigan Elementary and Middle School's Principal's Association. So really trying to blend policy, practice, and because this is a big topic in the media, right now, those perspect. So thank you again, everybody, for coming. And zero, stay safe. And we will look forward to hearing more about this work. Thanks, Chris, and everyone. This is great and great questions. Thanks so much. Bye.