 Hello everyone. Good afternoon, or good evening, or good morning, depending on where you're joining us from today. Welcome to Engineering for Change, or E4C for Short. Today, we're very pleased to bring you this month's installment of the E4C seminar series. The series is spearheaded by ASME's Engineering for Global Development Research Committee, and its purpose is to intellectually develop the field of engineering for global development. We host a new research institution monthly to learn about their work in advancing the United Nations Sustainable Development Goals. Today's seminar is presented by Dr. Erika Grala, who is an assistant professor of engineering management and systems engineering at George Washington University. Welcome Erika. And my name is Yanaranda. I am the director of engineering global development at ASME and the president of Engineering for Change. And I'll be one of the moderators for today's seminar, along with Dr. Jesse Austin-Brennan. The seminar you are participating in today will be archived on E4C site and our YouTube channel. Both of those URLs are listed on this slide. Information upcoming seminars is available on our webinars page. E4C members will receive invitations to upcoming seminars directly. If you have any questions, comments, and or recommendations for future topics and speakers, please contact the E4C team at research at engineeringforchange.org. And in addition to sharing your comments or suggestions, we also want to solicit your feedback more generally about the sector and what we can do to really improve our own strategy. So you'll see a big call for support here with this URL to our survey. I believe our admin will also add it to the chat. So we do really appreciate your feedback. If you're following us on Twitter today, I also invite you to join the conversation with our dedicated hashtag E4C seminar series. Now, before we move on to our presenters, I'd like to tell you a bit about Engineering for Change. E4C is an knowledge organization, digital platform, and global community of more than a million engineers, designers, development practitioners, and social scientists who are leveraging technology to solve quality of life challenges faced by underserved communities. Some of those challenges may include access to clean, moderate, and dignified sanitation, sustainable energy, improved agriculture, and more. We invite you to become a member. E4C membership is free and provides access to news and felt leaders, insights on hundreds of essential technologies in our solutions library, professional development resources, and current opportunities such as jobs, funding calls, research, and fellowships, and more. E4C members also receive exclusive invitations to online and regional events and access to resources online to their interests. We invite you to visit our website to learn more and sign up. E4C's research work cuts across geographies and sectors to deliver an ecosystem view of technology for good. Original research is conducted by E4C research fellows annually on behalf of our partners and sponsors and delivered as digestible reports with implementable insights. We invite you to visit our research page. The URL for that is listed on this slide. To export our field insights, research collaborations, and review the state of engineering for global development, a compilation of academic programs and institutions offering training in the sector. And just recently, we in fact published our state of EGD Latin America. So for those of you who are currently in Latin America and interested in seeing where there are opportunities to pursue graduate research, do take a look at that newly published report. Very exciting work that's there. If you have research questions or want to work with us on a research project as a research fellow or partner or sponsor, please contact us at researchatengineeringforchange.org. Now, I wanted to share also some research news that we published. In times that we are all experiencing right now, COVID-19 is obviously a really critical subject. And we have three research collaborations that we recently published around this. One is shown as an example here, engineering response to COVID-19. This is a comprehensive reference list for response aspects in low-resort settings, everything from PPE to social distancing protocols in informal settlements and more. In addition to that, the other topic of conversation, particularly here in the United States, is related to systemic racism and issues related to how we do engineering work, but that is inclusive. We recently published a really insightful article which featured a number of researchers who shared their experience with racism in international development publishing, something that may be new to some of you or may be familiar. And we do encourage you to take a look at that and share your feedback, share your experiences. And of course, let us know if there are additional topics that you would like to explore as part of our news. This is intended to release for the global conversation and help us all to do better in engineering at large. So with that on to a few housekeeping items, we want to take a moment to practice using our Zoom platform and invite you all to type in your location into the chat window, which should be on your screen. See where we have folks from. I see here folks from obviously Michigan. All right, we'll let you guys take a moment. If you're not seeing the chat window, it should be located bottom right of your screen. If the chat is not open, please do hit the chat icon in the middle of the slides. All right, so we have folks from Washington DC, from Colorado, from Toronto, Canada, from Brooklyn. I'm in Brooklyn as well. Boston and Arbor, Michigan, Washington DC, Chennai, South Africa, more coming in here. Oakland, California, welcome. Darfur, Sudan, Rwanda, Africa. We're so excited to have you here. Do continue, share where you are. Brazil and Nigeria, excellent to see you here. During the seminar, please kind of use the Q&A window, which is located just below the chat to type in your questions for our speaker so that we can keep those organized. Again, if you don't see that icon, please click the Q&A icon in the bottom of the screen in the middle of the slides and we'll be sure to share those with the presenter. Welcome from Rochester to Tempe, Arizona and the UK, we're thrilled to see you. And as I know that here are those instructions on where those windows are in case you're not seeing them. Again, really thrilled to have you all here and joining this global dialogue. All right, with this, it is my incredible pleasure to welcome Dr. Erica Gurala, who is the associate professor of engineering management and systems engineering at the George Washington University. She completed her PhD at MIT in the engineering systems division and her bachelor's at Princeton University in mechanical and aerospace engineering. She studies decision-making in real-world contexts to develop knowledge tools for better decisions in the design and operation of complex systems. And she'll be sharing that perspective, that work, and that expertise with us today. We're so incredibly excited to have you with us, Erica. I'm going to stop sharing my screen now and allow you to share your slides. Oh, you're on mute. Okay, now you can hear me, hopefully. It's really a pleasure to be here and thanks to E4C and to Jesse for inviting me to give this seminar and thanks to everyone out there for dialing in. I'm very excited to share all of this work with you. So let me just set up my screen for sharing. And hopefully, in a moment, you'll be able to see my slides. Okay. All right. So thank you for the introduction. I don't really need to say a whole lot more than that. What I'm going to talk to you about today is a particular approach that we developed in collaboration with USAID for trying to look at systems approaches in international development and particularly in agriculture in Uganda. And I'm going to illustrate those tools with an application in agricultural financing specifically, also in Uganda. And then take a step back and think about, all right, why was that useful or was it useful? Why and where do we go from here? And so I'm going to try to go a little bit quickly because I'd really rather hear what you think about this. But at the same time, this stuff is pretty complicated. And so I'm going to try to find that right balance. So to set the stage a little bit, the overarching challenge here is, as I think everyone here probably knows, achieving sustainable development goals is going to require adapting or redirecting some very complex systems. And USAID and other development organizations have begun to have recognized that certainly and have begun to try to come up with some solutions. One of those solutions is what they call systems approaches to development. And so this report is just one of the pieces of work that USAID has come up with. So what are systems approaches to development? So they mean a lot of different things. But the one that Uganda in particular has been looking at, let me illustrate that a little bit. So Uganda mission asks themselves, how do we support economic growth in Uganda? And as background, Uganda is composed of largely smallholder farmers with relative low agricultural productivity. And so in that context, how to support economic growth. So there's a farmer, but there's also a whole system surrounding the farmer. So a classic development approach would be, let's give the farmer improved seeds or let's train farmers how to use improved seeds in order to improve their productivity. So the systems approach or the market facilitation approach says, actually, let's not work with the farmers directly. Let's work with the system surrounding the farmers and to have that system then support the farmers. And so there are input dealers who sell farmer seeds and fertilizer. Let's work with the input dealers to help them see that it's actually a better value for them to sell improved seed non counterfeit seed at higher prices. And to help farmers see why they should be buying improved seed non counterfeit seed at those higher prices. Let's work with the traders and processors that buy the produce of the farmers and help them to see why paying a higher price for a higher quality product actually is beneficial to them in the long run and how they can help farmers learn how to create a higher quality product in order to meet their needs. And then let's help build capacity of the government and other actors in the system to create the right rules and conditions for this market to function in this way. And so the idea being let's intervene not with the ultimate beneficiary in this case the farmers, but with the actors around that farmer in the value chain and in the market. And that's the essence of this market facilitation approach which is one of the systems approaches that USAID is using. So it turns out that it's not as simple as this picture that I drew obviously. And so it's actually a much more complex system with many different kinds of things that influence each of those actors that I just talked about. So USAID needs to figure out where to intervene, right? What should they be doing in their next 20 million dollar activity for the next five years in Uganda? Where are the barriers that they need to alleviate or get around? And where are the leverage points where they can create the kind of positive change that that they're looking for? And then once they've invested that 20 million dollars in those five years in the course of that work they need to be asking is it working? Is the change happening that we anticipated? Because it's not as easy to say to how many farmers did you train. Now we're talking about working with people that are farther from the farmers is that work actually helping farmers? How do I know? Do I need to adapt my strategy? Okay so our work and this is a larger team who I'll mention in a moment has been developing approaches so methods and tools for finding these leverage points and barriers and also monitoring systemic change in agricultural market systems in Uganda specifically and using those approaches to support the work of USA and Uganda and that's been about four years now. And I have agricultural market in parentheses here because these tools are really not specific to agricultural market systems. Last month we saw a great seminar about water systems. Systems tools apply to water and sanitation. This is equally applicable there. We've talked about our approach with people from health democracy and governance energy and other areas at USAID as well and it's all applicable here and I do want to mention that this is not solely my work. This is the work of a much larger team led by myself and Jared Gensel at MIT and then we've had a number of researchers and students involved over the past few years and then many people at USAID have worked with us on this as well. So as long as I'm doing an aside I want to tell you a tiny bit about who I am because it's I find something interesting about this seminar is seeing the diverse backgrounds of people. So my bachelor's degree is actually in aerospace engineering and I started out doing systems engineering of physical systems like Mars rovers and then from my PhD I moved into operational systems and since then I've done quite a lot of work on supply chain management and information management for disaster response internationally. So there's an earthquake in Nepal. How do we set up our supply chains to be resilient to that? How do we set them up to be responsive to that? And underlying all of that is a lot of systems thinking and so the work that I'm talking about here is more on the development side and uses those systems approaches to think about these more amorphous human systems rather than these sort of physical product kind of Mars rover type systems. So that being able to use the same tools to span those different kinds of systems is something that I'm finding very interesting and I think this group could could help think through in the long term. Okay so now back to the thing that I want to talk to you about today. Agricultural financing in Uganda is the example I'm going to use here and so a little bit more background. I mentioned that a lot of farmers in Uganda are smallholder farmers working at or below the poverty line and for whatever reason they struggle to access loans. But loans can be a really important way of getting out of poverty because access to financing enables farmers to purchase inputs like seeds and fertilizer which enables them to increase their agricultural production which improves food security improves incomes and enables them to kind of get that cycle going. So USAID wants to figure out how to sort of jump start this cycle and enable more smallholder farmers to access loans. But just like I said before it's not as simple as this little picture that I've drawn here. There's this whole messy system around all of those all of those things that I've drawn and so we have those same questions. Where should I intervene? Where are the barriers and leverage points to getting that system going? And this map that I keep showing you as just a sort of example of the messiness is something that we developed over the last four years in collaboration with stakeholders in all parts of the agricultural market system in Uganda. And I'll talk a little bit about how we got there. So this is the process or the sort of set of tools we've been using. So the first thing is to map the system to understand what influences what in this system. The second step is to make sense of that map to find the pathways that lead to the outcomes that we care about. The third step is okay now I know what I think influences what okay what's the status? What's the status of the system? What's actually happening in the system? Let me measure what's happening in the system and then finally let me look at that map with a status and say where are the barriers? Where are the leverage points? So let me illustrate these steps briefly. Map the system first. So we have adapted basically a tool from from causal loop diagramming to be more appropriate for the development space. And so this is a map that allows you to visualize how the different elements of the system are linked to one another. And the elements are the important things that we care about in agricultural market systems and in many kinds of development systems. So we're often looking to see behavior changes by actors. We're often looking to see evolution of relationships among actors. And then we have some necessary conditions of the system that enable these changes to take place. So those are the kind of main building blocks of these maps. And then we would have interventions potentially by development actors and then key outcomes, things that we really want to see change in the system. So a map can show these relationships. So let me give you an example from agricultural financing. This is about how farmers get physical access to loans. So what is necessary for a farmer to get physical access to loans? And I should say actually that the arrows mean enables, not that it causes something, but it enables something. So how do farmers get physical access to loans? Well, there are two routes to that. One is they can go to a bank. They can go to a physical bank and ask for a loan. And the other is that they can do this through mobile money or through a mobile money agent and through a mobile phone. And so if you look at this map, which is probably hard to read on your screens, it's hard to read on my screen, you will see that for farmers to have physical access to a formal loan, they can either access it through mobile money or bank. If they're going to access it through mobile money, several things are necessary. There needs to be a mobile agent, mobile money agent nearby. Loans need to be available through mobile money. And the farmer has to have a mobile money account. And we can keep going back and say, okay, well, what does it take for loans to be accessible by mobile money? Well, the mobile money is the mobile money providers have to have some form of relationship with those financial institutions. So this is an example of the kind of math that we're talking about where, again, the elements are things that need to be true of the system and the arrows are what enables what. Okay, so we had this little piece of a map that I showed you here. Actually, many other things also influence whether farmers take out loans. It's not just whether they can physically access it. So there's a lot of other stuff going on in this map. And then even beyond that, lots of other stuff is happening outside of financing outside of loans in the market system. So you also have things about inputs importing things about farmer practices, commodity distribution. So the purchasing of produce and the selling of produce, the regulatory environment, the other kinds of services available. So you have a lot of influences on each of these things. Bottom line is map the system. Okay. So now we've mapped the system. I just summarized about, you know, three years of extensive work in one minute. Then we want to take that map and understand it because when you look at this thing that doesn't tell you anything at all, really, right? It just tells you it's complex. So how do I make sense of this? Well, we look for pathways within that map. So what do I mean by a pathway? You've already seen one example of a pathway. This pink bit here is a farmer's ability to physically access loans, which I've already told you has to do with physical banks or mobile money. But it turns out there's a lot more that goes into whether a farmer takes out a loan than whether he or she can physically access it. So here are some other pathways in many colors. I don't expect you to read these. But in yellow, sorry, in blue up here, we have whether a farmer meets the requirements for a loan, has an ID, for example, and has collateral. And then in purple over here, we have whether a farmer wants the loan in the first place. Are they interested in taking out a loan? We farmers need to also have information in order to be interested in taking out a loan in order to understand what that is. You need to be able to afford it. That's another pathway. So these pathways just group thematically relevant portions of the map so that we can look at each one individually. Okay, so now we've made sense of the map. We've broken it into tractable pieces. Now comes this really tricky part where we want to understand what is the status of the system. How can I measure what's happening on these pathways? So there were a number of challenges in this and not least of which is that there's not a lot of data that can span the entire Ugandan agricultural market system with one basis. So you're going to be pulling data from lots of different sources if you're going to have any hope of understanding what's happening across this whole system. And we needed to figure out a way to put all of that on one picture. So what we've done is a very simple approach. As we say, we're going to color the map according to how widespread is the adoption of this behavior. Okay, so financial institutions, sorry, farmers using mobile money, which you can't read again right here is this box. How many farmers are using mobile money? What percentage of rural Ugandans are using mobile money? And if it's low, it'll be red. If it's moderate, it'll be yellow. And if it's widespread, many people are using it at screen. So farmers using mobile money, it turns out about half of farmers use mobile money. Farmers that has access to a mobile phone. It turns out that almost 80% of rural Ugandans do have access to a mobile phone. So this one is green. And so now that we have these sort of data spread out on the map, then we can look for barriers and leverage points. So let's look back at that same pathway that we were just looking at. I suspect if I asked everybody, which I won't because this is an awkward format for audience participation. But if I asked you to identify the barrier on this map, I suspect you would point to the same place. Where is the barrier on this map? Give you a second to think about it. It's the red dot, right? It's the one that's red. Okay? The red one means very limited adoption. So the biggest barrier on this pathway is that hardly any farmers have a bank branch nearby. It's very hard for a farmer to get to a physical bank. Okay. So now we've been able to identify a major barrier to farmers physically accessing formal loans. Now can we come up with a way to solve that by using the same systems approach? And I'm just going to show you, this is again unreadable to you. But one of the interesting things about this is that farmers have a very hard time physically accessing a formal loan, meaning a loan from a bank. But it turns out that farmers don't have a hard time accessing an informal loan from a village savings and loan association or from family and friends. There's these informal financing networks that are functioning very well. And you can kind of see that down here with a lot of green and yellow at the bottom of the map on this yellow pathway. And so an interesting leverage point here would be to say, how can we expand access to formal loans by leveraging this informal existing network of financial service providers? And in fact, that's one of the things that's happening right now in Uganda is this agent banking model that draws on these informal networks to enable access to more formal loans. And so the systems approach allows you to kind of look across these, find these barriers and then look nearby or for leverage points that are functioning better and figure out how to leverage those. Let me give you another example. One of the things that USCID was very interested in is system help. How is my system doing at a high level? And so we said, well, a system health is kind of a weird concept. Like how do I say like, oh, we're doing fine, right? And so we said, all right, well, let's start with the elements. We now have element health, red, green and yellow. What about the pathway? How is this pathway doing? So I already showed you there's this big glaring red dot on this pathway. But let me do another fake audience participation question. How is the pathway doing? So hopefully you're all on screens where you can zoom in a bit. How is this pathway doing? Do you think the pathway as a whole is red? Do you think farmers really don't have physical access to loans? Or do they have another way to get physical access to loans other than the bank branch? So it turns out that there's an alternative, right? I said at the beginning that it's not just bank branches, you could also go buy mobile money. So how is mobile money doing? Well, this element is white. It means we couldn't measure it. We don't know, there's no data out there on whether farmers can access loans through mobile money. But we can look backwards and we can say, well, lots of loans are widely available via mobile money. There are many mobile money agents around and most farmers have one nearby. And some farmers use mobile money. So we can sort of infer that the status of this element here is yellow. It's about maybe around half of farmers could access a loan through mobile money. And then the next one here, farmer has physical access to formal loans. Well, it's an either or. They don't need both access to a bank branch and access through mobile money. They just need one or the other. So we could make this yellow and we could say, you know what? This is moderate. It's working moderately well. The access to formal loans is okay. Not great, but it's okay. So this is one way to sort of abstract up a level out of all this complexity and say, yeah, you know, one of these five things that we need for farmers to take out loans is okay. It's working okay. So if we then zoom out to this crazier picture, we say, all right, remember that the one we've just been looking at is the pink one. And its health is moderate. I said earlier that informal financing, the yellow one down in the lower right, that one's doing well, right? Most farmers have access to informal financing. And then to sort of foreshadow one of the other key takeaways, if you look over at demand, the health of that pathway is poor. So let's look at that. Very few farmers want to take out a loan for farming reasons in the first place. Very, very few. And one of the reasons for that is an unwillingness to take on the risk and a lack of trust in financial institutions. So the demand just simply isn't there whether or not they have the access. And that is one of the things that hasn't really been talked about. And I'll come back to that in a minute. So now we're at that last. We just finished identifying some barriers and leverage points in the system. And then there's a third thing we could do is look for data gaps where we don't know enough to even look for barriers and leverage points. And if you look at this map, everything in white and gray is something that we couldn't measure for one or the other reason. And so some of these big data gaps, we simply don't know enough about the affordability of loans. All of the data sources we looked at, we couldn't figure that out. We don't know a lot about whether there's a supply of appropriate loan products, meaning loans with good payment terms for farmers and the appropriate payment periods that span the harvest so they could actually make the money back to pay the loan. And we don't know a whole lot about information and how farmers access information about loans and whether they have access to information. So here we can't even find barriers and leverage points because we simply don't have the basis to do so. And that can be important information for a development organization to know where do they need to work on collecting data and information. So the point of this seminar is not necessarily to say like how is financing doing agricultural financing doing in Uganda. But these are some of the things we learned that formal loan access is limited, but improving informal loans are widely available. But the demand is low. Nobody wants these loans for agricultural purposes. They take loans for school fees. They take loans for many things, but not for agriculture. And no one is talking about this. And we don't really understand why that was missed. And then like I said, we know some things about where the data gaps are. And we can find these neat leverage points where we can say, well, let's leverage those informal financial networks to help with both access to and that trust in formal financing to get the demand back up. So let's now pause and take a step back from that whirlwind tour of complexity. How did we get here? So we said, first, if we want to understand this complex system, we need to map it and visualize it so that we can see what we think is influencing what. And I want to emphasize that's what we think, right? This is not an accurate map of the system. It's a map of what the people who made it think is happening in the system. Then we use that as a tool, we find pathways to key outcomes so that we can focus on tractable pieces of this. We measure those pathways and the system as a whole. And then we can look for barriers and leverage points. Okay. Deep breath. Reflections. Was this useful? Was this useful? So we did a series of workshops over the last four years in Uganda that were attended by a variety of stakeholders. And we were told that it was useful. And one of the reasons that people said it was useful is that it helped them to see the value of systems thinking. So those of you who are on this call, some of you may be engineers and you may be used to this idea that we need to think about the system. And in development, many people are thinking this way. But this is also new to a lot of people. And so just seeing the value of taking that big picture and thinking about the many influences was very valuable to some of the people we worked with. It was also useful in that it generated insights for USAID's program design. You can see some of our sort of ideas in the most recent set of activities that USAID has invested in in Uganda. And then we were also able to provide more specific insights that guided some of their decision making around their resilience programming in a particularly vulnerable region of Uganda, things around the seed industry where counterfeit seed is a big problem in agricultural finance like you just saw and in other areas. So we think it was useful. It certainly could have been more useful. And I'll talk about that in a bit. But we think that there were aspects of this that were definitely useful. Why? So as academics, we want to ask why was this useful? So one of the reasons is again, just visualizing the system and the pathways that enable or block change, it can be really eye opening. It sounds simple. It is not simple. So that visualization was very useful. And then particularly it was a way of engaging stakeholders around a sort of common understanding of how the market operates. One of the most interesting exercises we did in our workshops was we asked people to say, where are you working and draw themselves onto the map? And you can see that kind of in the lower left hand picture there. A lot of people figured out that they were working, a lot of different organizations figured out that they were actually working in the same space, thematically and geographically, and that they could coordinate their efforts a little bit because of this workshop. Then on the data side, I talked a little bit earlier about how hard it is to get a single data set that talks about all the aspects of this system. It's really, really helpful to put all those diverse data sources into one picture. And the map gives us a way to do that. And we developed some ways of trying to get all those differently sampled and differently scaled data sets to at least fit in a red, green, yellow color scheme. It's not perfect, but at least we can see everything in one place. And then based on this, of course, we can find all those barriers and leverage points and how to adapt our programming appropriately. And then finally, you know, another, a few more reasons why it was useful is it combats the sort of linear thinking that's relatively ingrained in more traditional development practice from results chains and log frames are mostly very linear. And this kind of says, all right, no, that's not enough. Let's look at the whole thing. I wrote, it's still simple sort of, there's really no way to call this simple, but we did try to adapt some of the systems engineering tools to at least focus on the right sorts of elements to be in the map to focus on those behavior changes, relationships, and the sort of market conditions. And then finally, something that has been really important and that we've been told has been really important is that it gets to the details. So a lot of the systems approaches in the development sector now are very high level. They say, we need to worry about these five things. And that's incredibly useful, but it doesn't get to the details of all these cause and effect chains that make things happen. So what we try to do is bring it down a level and get to more details, which led to, of course, this sort of very complex set of maps, but also gives you much more clear guidance for what to actually do. So then challenges, there's no way around it, these systems are complex and we're still figuring out what the right balance is between enough detail to make the right decision, but not so much detail that we're paralyzed and we don't know how to think about it. And so that's one of the challenges that we're still trying to navigate is how much do you really need to know? How much detail do you really need to get your development activity right? And we've been struggling with a few other challenges as well. I think I'll focus on the last one here, supporting real decisions. We have supported some real decisions, but there's also been a lot of opportunities that we weren't able to sort of support. And part of the challenge as an academic is how do we successfully engage with the practice of development from our sort of unique positions, right? So we weren't full time in Uganda doing all Uganda all the time. And that's a challenge, but that's also an advantage because we can step back and think big picture and come with a fresh perspective. So this is something we're still navigating. So thinking ahead, you know, thinking to what's next, I think everyone here probably understands sustainable development requires systemic change. So even if you're introducing, let's say, a new product or a new practice, so that could be farmers using high quality seeds or it could be a new cook stove. We've realized in recent years that it requires not just the cook stove but also the supply chain to get the parts and the fuel and the maintenance for that cook stove. But equally important is the behavior changes and relationships around the adoption of that new technology or that new practice, right? People have to change the way they cook potentially or the way that they get supplies in order to use this new technology. And so systems approaches like this one or like the ones you saw last week can help us to figure out where those leverage points and barriers are that can help with that behavior change piece. That's so important in the sort of engineering of global development. And so I'm going to stop there. I'm really interested to hear your questions and your feedback. And I just want to again acknowledge the larger team that was involved in this work. So thank you. Wow, Erica, you're blowing us away here. So just very excited to have you here as I told you at the beginning before we started. But every time I see one of these seminars, I'm further inspired, but yours was particularly near and dear to my heart with systems thinking. And I think that you really answered a lot of the questions I've been thinking about what goes in this system? What are the types of elements that are in a system? And I think that was really powerful in thinking about being able to support decisions, real decisions in organizations and make academic work useful is something that I strive to do and you are doing it. So I'm glad we had you, Doc, because that's really, really great stuff. So I'm going to ask the two questions that we have here. I have a whole bunch of questions. I want to encourage people, two things just as we end the sort of presentation part. Number one, if you're hearing something you want to follow up with another question, please put it in the Q&A, even if I don't get to it or try and synthesize it or dress in some way, we're going to be putting them linked to the video recording afterwards. So Erica is going to be able to answer those questions afterwards, even if we don't get to it. And so you'll be able to see a response to all of the questions when we put up the recording online. So please continue to ask questions in the Q&A, even if we don't really have time to get to it, although we have a lot of time. So this is great to have this discussion. So first, Alma asks this great question. What is different about using this for development? Sort of you have a lot of time, you have four years to get into this and sort of make this system map and really understand, let's do workshops with all these stakeholders, let's do it. You're saying just about the amount of time and resources required to even generate that map that's so rich in information. With your experience in disasters, how might this translate to a disaster or humanitarian relief situation where it's equally important, but maybe you don't have the same time scale? So if we're thinking about COVID-19 prevention or contact tracing, it's a rapidly evolving system, it's fluid. How can you do the same type of thing where perhaps the agricultural market system is pretty stable in a steady state situation? Yeah, that's a great question or several great questions. So let me try to remember I have two things that I want to say. So the first one is, yes, this is a lot of effort to do what we did, but we had a long time. One of the things I was most excited about with this particular research project is I never get four years to really learn a context. So it was really nice to be able to dive in. And we did do a lot of detailed rewrites of this map and we workshopped it over and over. That's not necessary, it's great, but it's not necessary to be able to do a map. We've also done system maps in an afternoon. It's a busy afternoon, but if you have the right people in the room and they're familiar with the approach, you can get pretty far in an afternoon and the added benefit of continuing to workshop it drops off. So you can do a light version of this. In fact, in the toolkits that we're writing up, we initially played with the idea of let's do an afternoon version and let's do a two-year version of this approach. So it can be done faster, but it still is a big lift for what would be maybe a disaster response context. So I would say a disaster response like responding to the earthquake in Nepal is very different than responding to COVID-19 because the earthquake in Nepal is sort of very fast paced, but also more familiar in the sense that like we've done earthquakes before, we've done Nepal before. Everything is unique, but we know at least where to start, right? We've rehearsed this before. And it goes too fast almost for this approach to be useful because we already know what questions to ask. We already kind of know which parts of the system are affected. With COVID, this is so new that we don't necessarily, and it's slower moving. We're not talking, you know, three days, we're talking months. And so that is a really good application of this approach is to look at COVID. And we've actually been doing that with Uganda, unsurprisingly. Because we already had this map, we were able to start with, okay, what's changed? Right? So what we're doing with COVID in Uganda is we're saying, let's look at the news, let's look at people who have been doing surveys, let's talk to people who know what's going on, let's talk to the seed distributors, let's talk to the people who sell maize across the border. And let's try to get a sense of the status of these map elements based on that. So here we're not doing as deep a dive into actual data, right? But we're doing more like a little bit of intuition combined with a little bit of data, but we're still using that same color, the elements. And we're saying what has been affected? What's not working now that used to be working? Where are the problem areas? And then we can, when we see, oh, there's a problem with importing seed, let's say, we can trace what are the effects, what are the ripple effects of that problem by looking at what that part points to on the map. So what we've been able to do with that is create these sort of reports that say, okay, here are the problems that, you know, the top three problems with inputs in Uganda with agricultural inputs, based on what we're seeing in the news and illustrated with these map pictures. So with COVID, the map has been really helpful. But again, we didn't have to build it from scratch. So I think it depends on what your goal is. If you want to do the really detailed analysis that I showed, that takes time. It's going to take, you know, it's going to take a while. It's not something you can do in an afternoon. But you can get really far in an afternoon if your facilitator, say me, builds a really bad strawman map first, right? But just gets you started, you say, no, that's wrong. And you get your stakeholders up there to say, no, this is wrong, this actually should go here. And this should be here. And we're missing this piece. And by the end of the afternoon, people have a common understanding of the system, at least as much as you can do in that amount of time. So there are, there are ways to apply it at different time scales. But I still don't know the answer to the specific question of sort of, would we want to use this in a disaster response? I don't know, would we want to use this in a slow moving, like a slow onset disaster, like a famine or COVID or something like that? I'd say yes. And we would be creative about the way to do it. Well, thank you for that insight. I think it's great to sort of identify the class of problems with which this approach might be very impactful. So that was really great. I'm going to ask the question from Nicholas and maybe just rephrase it a little bit. So he's asking about, you know, you used adoption status as that red, green, yellow, you know, indicator as to what is the health of this pathway or this type of thing? Is that the only indicator is that sort of trying to understand how did you decide to use that indicator? But also how you decided what the level of adoption had to be to be in the different red, yellow, green? I mean, I know that's qualitative, but you know, it does that mean the same thing to all the stakeholders to be not adopted to be red versus yellow. And then it seemed like there were feedback mechanisms. So you can just touch on the type of map that you were drawing in terms of it, because there are system dynamics maps, which I think are a little bit different than what you were drawing. And so maybe, maybe touch on that also. But starting with, what are these indicators you're using for each element in the system? And then how are you deciding how to classify them now? Yeah, so those are good questions. We put a lot of thought into what the colors mean and how to color an element. And the adoption status was actually kind of a big deal for us to pick, because you wouldn't necessarily, so let me back up, we wanted to be able to use the same things to color all the elements. So we didn't want to say like, quality of adoption or like, are they really good at using good business practices, or are they just using good business practices kind of badly or something, right? We didn't want to get into something that could be a different indicator for each element on the map. And adoption status is something very simple that also matches exactly what we're trying to do with saying, how widespread is change? How much change is happening? And when we looked at the map, almost every map and element is about adopting something, adopting a behavior, adopting a relationship, or the pervasiveness of a market condition or something, right? So we settled on this adoption status as a way of making everything uniform and simple. That misses a lot of nuance for sure. And if you're a development practitioner, that's not the only thing you're going to use to decide what you're going to do next, certainly. You would want to understand a lot more about what's going on in each element. But what we're doing is trying to build this sort of high level picture of what's happening in the system. And that was the easiest way that we could see to put it all on the same kind of scale, to be asking the same question about each map element. Now the thresholds for colors. So the way we talk about this is that is up to, if you're a development practitioner or if you're a researcher and you're asking a specific kind of question, you would basically do this based on a target. So if good change means you've got 20% of people to do something, then that's green, right? If good change means you need 80% to do something, that's green. So I'm not an epidemiologist, but COVID has some very clear things, right? There's some sort of threshold out there for herd immunity. We need X number of people to have immunity. And that would be your threshold for green, right? So it's sort of context dependent, what counts as green, yellow, and red. And a lot of times if you're doing development, that'll be based on whatever your target is for your development activity. So that's kind of how we got to the coloring scheme. And then sort of barriers and leverage points. We actually tried to do leverage points and barriers just without coloring the map, without measuring anything at first, because the measurement is a huge effort. And we could sort of guess, but we found that mostly people were saying what they already thought. Finance is a barrier, because I know finance is a barrier already. And so what's really nice about having the colors is it forces you to kind of prove your ideas about where the leverage points and barriers are. So it kind of forces the gathering of evidence to back up that assertion of a barrier or a leverage point. But certainly that wouldn't be, there's no reason you have to restrict yourself to only those. That's just what our approach facilitates is looking at the structure of the map, along with the colors in the map. So when we find something as a barrier and it's red, but you might look at the structure of the map like I showed and see that that's not the only path. If there's an alternative path, then maybe it's not a barrier. Maybe we just have to help people get on to that alternative path. So those are the two things in our approach that we would base that on, the structure of the system as embodied in the map and the colors on the map. I'm not sure if I'm understanding that question entirely. The part about feedback is interesting, so I want to address that. So those of you who know system dynamics and causal loop diagram will notice that there weren't a lot of loops in our map. And there really ought to be. There's a lot of loops in this system. And we found that was just overwhelming to people because they just added a lot of arrows, right? And so we ended up taking this interesting approach of kind of simplifying out a lot of this feedback and then but knowing that it's there. And then when we've turned a couple of these into system dynamics models like actual simulation models, and then of course we put all that feedback back in. But this was the level at which we could engage many of the people that we were working with. And I think that if we were focusing only on a single pathway, then we would put all that feedback into that pathway. But when we're trying to get this view of that entire system, it created more confusion than it allayed. But it's a real big problem for us because that's a fundamental truth about these systems is that, you know, how you can create systemic changes through feedback. So that's another piece we're working on figuring out. Yeah, thank you, Erica. You took that multi-part question just really unpacked it. So amazing response, horrible questioning on my part to put like a five-parter in, but you did an excellent job. So thank you for that saving me. What I would like to ask now is perhaps a follow-up. And James asked this question, I'm going to try and perhaps paraphrase it. What about, so you're talking now about, okay, we want to simplify it to be able to engage the stakeholder and understand these maps in a way that, you know, we can really grasp sort of that system level view. How do you account for where the edges of that map are, right? So James is asking about, okay, well, what about other interventions? Maybe if we had education, then this whole map would change like the relationship, the red, everything would change. If we had an intervention in some other area, you know, poverty is multi-dimensional, right? There's lots of aspects of it. And so, you know, perhaps your response or your adoption in a particular area, this seems to represent what we understand right now and we're identifying pathways within a particular system boundary. And in systems engineering, we spend a lot of time talking about where is the system boundary, right? So I wonder how you approach that with your stakeholders to say, look, we're just focusing on this. We, you know, how do we address, like maybe we need to also affect education before we can affect any of these things? Yeah, it's a really good question. And it's something we've wrestled with a bit. In our very first versions of the maps, we had put these like clouds for education and health and, you know, these other issues and said like, and drawn arrows, right? We said, we can't tackle this, it's outside our boundary, but we need to keep it on the map so that we don't forget about it. And that obviously fell off as the map grew bigger. But it's something that we found that we didn't have to think about that. I don't have a great answer. We found that having too much on the map made it too hard to manage. And so, we dropped the things that were outside our boundaries. And the drawing of the boundary is an art more than a science, as I'm sure many of you know. And for us, it was about balancing what is tractable for us and what is tractable for our stakeholders. Like what can we really conceivably do in a one day workshop? Can we tackle all of those things or not? And then we also had a sort of multi level approach. So the agricultural finance map that I showed you is more detailed than how we treat ag finance on the bigger market map. And so one could also imagine having a map that abstracts even a layer up and just says markets, education, health. And so when we were asked, and that's useful for a different kind of decision making. So in a way, it's about what kind of decision you're trying to make that day. Are you trying to decide the details of how you're going to improve agricultural financing? Or are you trying to write a sort of country level scope or country level guidance about what are the pieces I'm going to put in place, right? The country strategy basically. And then you want the right map to support the decision that you're trying to make. So the higher the higher level, the decision, the less detail you want on the map, but the more systems, right, the more pieces you want to be showing. And so we've tried to come up with sort of these different layers of maps where you can be zooming out and then zooming back in. We haven't solved that. And in fact, we're working now on a resilience map. And resilience has to do with all of those systems, education, technology, food security, nutrition, right, health, all of this. And we're struggling with how to get all of that on the map in a way that it's tractable. Okay, well, we have a lot of great questions here. So you're going to have your cut out for you, Erica, after our call, because we're going to send you this long list of questions. So you've really touched a nerve, I think, with a lot of people, just really excited about what you've presented in this approach, which I think is really great. I'm going to ask one last question, just because I'm the moderator, I get to make it my own personal question. And then we'll wrap up. My last question is, from a research perspective, in terms of these tools and approaches, what are sort of the big, unknown questions around the methodology? You sort of talked about them, okay, what's the way forward? We've talked about indicators, metering, operationalizing this approach. But you also sort of said it seems really simple, but it's really hard to do, right? So from like an academic perspective, I'm wondering from your experience, your view, what are the things that could really enable, like what question could I answer that would enable you to take this to the next level, right? Like what's the real barrier to applying this type of approach? So like, I'm less interested in say ag financing, I don't do ag financing. But if I wanted to apply your approach, what are things that we need to answer as a research community to say this is the right level of detail, maybe that's the important question. What are the unanswered questions for you from a practice approach as an academic that I need to answer, like, so that we could do this in practice across a lot of different contexts? Yeah, that's a really great question. And it's something that we've been struggling with maybe a slightly different version of that question is how do I make this accessible to development practitioners? The way that you put it as a research question, like what can I answer before I can do that is very interesting. And I think some of the things that we've struggled with the most is that that balance between complexity and simplicity. How much something I'm interested in, how much, how much of the detail, how much of the complexity do we need to make the right decision? This is, you know, something that I can think about how to answer with a decision theory kind of perspective, right? I need to know enough detail, but maybe I don't need to know all the detail in order to get that right, I can get it approximately right based on a higher level picture of the system. But I think that we need a little more detail than what we've seen used in practice in the past. And maybe we don't need quite as much as we're doing here, and that would be great news, right? So how much detail? And then the other thing I guess is how can we, and this one, you know, some people who have done more human factors work might be able to answer, figure out how to answer this, but how can we make this kind of map more accessible? Or how can we help people to access it better? And I know there's a lot of work on that, and particularly I've seen papers on it in sort of water and environmental areas. And, but a lot of those are sort of experimenting with bits and pieces of changes, like we did here, we changed the names that go in the boxes and we said, I think this works better. And not to belittle that work, that's super important, but I wonder if there's a more bigger way we could do that. How can we figure out how to help people access system mapping approaches without just tweaking the approach, but maybe something more creative than that, helping people to see them differently or something? I think those are two, two big questions that would help make this kind of work more accessible. Erica, thank you for setting our research agenda, so everyone on this call, let's get to work on those questions immediately. I certainly am going to be thinking about that, how we can build that into our work and try and answer that as a community. I want to just reiterate that all of the questions, we didn't get to get to all of the questions, hopefully part of that got answered in Erica's responses, but Erica is going to be receiving all of these questions and we will post responses from Erica on our website with the video recording and a link to that. So if you lost connection or anything else happened, you missed some of the great information that Erica has been sharing with us, you will be able to see that. And thank you all for all of your questions, really improved it. I'm looking forward to see you guys next month, two Wednesdays from now and I'll leave it to Yana to go ahead and wrap us up. Just thank you again, Erica, really inspiring stuff and thank you for joining us today. Yana. Thank you. Thank you, Jesse. Thank you, Erica. This has been such an enriching conversation. Definitely stay tuned for the recording. Give us about a week and change to get that up and some of the questions answered. And as Jesse mentioned, we will be back next month with Esther Adiembo-Abonio, who is Associate Professor of Engineering, Design, and Architectural Engineering at Penn State University. And we'll really have some phenomenal insights regarding the built environment. And with that, I want to thank you for attending. Please feel free to send your questions if you come up with them after so we can just add them to the pile for Erica to answer. And we of course encourage you to sign up as A4C members so you'll hear more about webinars, seminars, and all the other phenomenal opportunities that we have available to you. But I'd like to say good morning. Oh, Jesse, one more thing. Yeah, the survey. So there's a link in the chat. Yes. We will be, you'll maybe get an email message from us. We're really trying to collect information to how we can improve these seminars, but also we're trying to again make coherent research community. And this is the first step in that effort. And we have other initiatives that we're starting. And that survey is really going to help us target what are the sort of barriers to us becoming a single research community across these different disciplines. So please fill it out. It's super important to us. Link is in the chat. You might see it later. I just wanted to reiterate that publicly. Thank you, Jesse. Yes, filled out that survey. Thank you so much. And apologies for having the link being correct a little bit earlier. I believe our team rectified that. All right, with that, I wish you all a good day. And we will see you on the next A4C seminar. Thank you. Have a great day, everyone. Stay safe. Bye.