 So, thank you for the opportunity of having me as a discussant here. I see a lot of very illustrious faces in the audience and I'm sure we'd all love to hear their thoughts as well, so I'll keep this short. So thank you to the speakers for giving a really nice overview of where we stand in terms of microeconomics for development. As Carol also started the session that we've been talking about, the bigger picture, there have been some thoughts on how with the bigger picture we are, or rather with the focus on smaller analyses, we are maybe losing sight of the bigger picture and I think that this session sort of really got all those thoughts together. So, I'm just going to keep it a bit more general, so I think this is also going back to the idea of synthesis that Stefan Durkhan talked about two days ago. And I think this has also been raised very nicely by Alan in his opening presentation today. And I think as somebody who's starting out a career in academic and policy research, I think that with sort of focusing on the bigger picture, we also have, we also need to understand what are the mechanisms that make these bigger policies work. So I think that that's where the body of these smaller interventions, whether it's using experimental methods or observational data, we need to get sort of all the analysis together in order to be able to synthesize it to understand what works, what doesn't, where there are positive results, negative results before we can really come up with smart policy suggestions. In terms of sort of more specific comments, just from the reading of my reading of the literature, I'm not completely convinced that the fact that training has not worked for sort of enterprise performance is a very decided result yet given the sort of shortcomings of the existing literature that David and Chris highlight in their 2013 paper. There have been a lot of statistical problems with the papers that have done it largely on account of power in terms of sample size, the sort of time spans that these studies are considering, attrition issues, reporting bias and things like that. So I think until we can get a number of really good studies that are mitigating these concerns, we probably shouldn't sort of get to the thing to say that these training programs do not work. Moreover, they also have a recent NBER working paper, which is using a bunch of their own data from previous studies, which actually shows that these management practices that small firms adopt are positively correlated with firm performance. So I think that there's a sort of disconnect between what the observational data shows and what the experimental literature has shown. And I think that we need to sort of reconcile that. Also, there have been these spate of papers on training programs, whether that's at the more personal level. So we now have a bunch of papers that are looking at these skill building programs, cognitive skills, non-cognitive skills. But I think we still don't understand what components of the program work and what doesn't. And I think that we probably need more research in order to understand what aspects of these programs are more successful. And we also know from experimental literature in the lab that framing really matters. So we need to take that into account. And a lot of countries, particularly developing countries, are taking up skill building programs. But we need to know what works before they can be advised on how to take this forward. And I think that this also relates to Karen's point about measurement issues. And just another observation, it seems from what Alan spoke about and also what Chris mentioned, is that mitigation of risk appears to be one of the sort of key avenues for helping households as well as firms and farmers to undertake more productive investments. And I think that we need to think more about that, to think about what would incentivize these people to maybe even invest in these technologies when they have to pay a little price for it, even if it's not the full market price. So these are just a few of my thoughts. Thank you. Thank you once again for being here and thank you for the comments as well. So I'd just like to keep it short and mention a few points that I found exciting in this session and a couple of other sessions as well. And these are things that I would like to work on in the next few years. So the first one was that I think that we can spend some more time thinking about how field experiments can be used to understand spillover effects. So just as an example, we know that there's a tight two way relationship between agriculture and health. So interventions in the health domain would have effects for agricultural outcomes and vice versa. And I think we can still work further on understanding that. And then also from the point of view of availability of data, it's quite exciting to know that there are more carefully collected panel datasets over long periods of time. So I look forward to thinking more about intergenerational mobility and transmission of various effects. Finally, going back to more measurement issues. There was some talk yesterday two days ago in the conflict session as well that I think we need to move beyond just looking at the incidence of conflict at the level of the household and to more thinking about designing new questionnaires to look at the effect of conflict within the household. And finally, there's also been a movement towards measuring individual level, behavioral and personality traits. I think there's still a lot more work to be done in measuring them better and also understanding the role of aspirations and expectations. So thank you. So we've time for some questions. So I'll do, I'll take maybe three questions and then we'll see if we have time for a second round. So I'll start, I'll start at the back. But the question is that it seems that the methodology of RCT is partly inappropriate for long term effect due to attrition and due to spillover effect. So it's indeed a self contradictory statement. That's what we would like to do, but it's not the proper method because there are very, very specific conditions that we need to be met for the impact to be properly measured. Okay, we'll take one from the front. I think Alan's presentation put very clear background stories about development microeconomics and I entirely agree that the microeconomics economics work had to be put into macro context back when you finish doing micro works and we shouldn't forget RCT dominated and it is, it brought very good evaluation techniques, but we should understand the development, it's a long term process as Philippa mentioned and it is basically you cannot really do it RCT in the long, long term time dimensions. So we have to be aware whenever you make evaluation and conclusions not to be too decisive. But my question is very much to Woodruff. I'm not surprised that microfinance product is not good for scaling from micro to that because it's a standardized working capital and basically to get from micro to bigger scaling up enterprises, you need much more farm specific financial product. It's a very financial services, very information intensive things and you cannot do it with a standardized one. So it's not just grace period, two months changing it, but much more thinking about it. And in that sense, yeah, I will quickly, in that sense, also I have a question training. You mentioned that the actually industry specific sector, specific things are more important. So if training is too much general training, I'm not surprised it didn't have effect. But you know, if it is training, which is more geared to that particular sectors they are working, that might make quite different. So again, information. Okay, I think we'll take one more before I think I saw Louise's hand first. Thank you. I'm Louise Fox from the University of California at Berkeley. I first want to congratulate the panel members. This was a phenomenal panel, this was terrific, really good. I really want to focus a bit on Chris's presentation, but I'm going to pick up on what Alain said about the macro micro linkages. We used to do something called the micro foundations of macroeconomics and it seems maybe we need to go back to that at least for development. And I think if we went back to that, Chris, I would rephrase the question that you were answering. What do we know about creating wage employment? That's what your graph said that you put up is that there isn't enough wage employment and that was where Santiago Levitt left us at the end of the last session. And I think part of the answer is macroeconomic. It's Roderick's work about deindustrialization and the fact that the manufacturing sector is much more capital intensive due to globalization than it was years ago and that the service sector is a sector of smaller firms and lower productivity. And so that leaves us then. I'm not sure the right question is, can we get self-employed and small enterprises to grow? And I think that when you talk about the marginal returns to capital in small enterprises, it's important to note that most of those grants may have increased profits, but they did not increase employment fantastically. And I think we need to remember that about your whole discussion about self-employment and these graphs and the shortage of capital. And finally, I would like to, and that's the problem with generalizing, by the way, from these RCTs to the broader generalizing beyond an RCT is that giving a grant is fundamentally different than giving a loan. And the people's behavior to a grant is fundamentally different than a loan because, I mean, Stiglitz has explained this to us many years ago about people's attitude toward risk. Finally, yeah, I'd just like to say on Chris, I think one question I would pose and maybe add to your agenda. Do you agree that we have underestimated in survey research on firms? Most of the survey research has been on households and not really on firms. Okay, so I think I'll ask the panel to respond and then we might have time for another round. Yeah, the issue of attrition and the kind of short term, long term is not specific to RCTs. We are going to whichever entry point we take, whatever measure we have, whatever way we have used to manage this entry point, we're going to be affected by how do we keep track of the longer term impacts. I think there are interesting ways of combining natural experiments with RCTs. For example, think of an experiment where you do an RCT, a progressor type, right? And then you want to follow the long term consequences. You're going to have a lot of attrition to follow specific individuals, but maybe you need to project it at the higher unit of analysis at the municipality or at the regional level. So you're going to have a kind of dilution of tracking particular individuals, but they are going to be, of course, there's migration and there's going to be attrition across municipalities if you like. But you can recoup some of those spillover effects and some of those longer term effects by kind of redefining, projecting onto a natural experiment following what you have done at the level of an RCT initially. And then you can do the opposite in terms of what you are proposing, which is start with a natural experiment, which allows you to kind of track the broader longer term impact. And then it raises issues of design, issues of incidents, which then you take on via specific RCT. So you go the other way around. We did, for example, work on the credit bureaus in Guatemala. You look at first the rollout and the impact that the credit bureaus have in the general sense, but then it asks very specific questions as to who are the beneficiaries, who are the ones who are hurt. And then you do maybe an encouragement design, you do an RCT. So I think we have to be careful not to put the method before the question. And once we have the question, it eventually suggests ways by which we can reconcile being precise in terms of measuring impact using RCTs, but at the same time following of the broader implications of what those interventions have using other kinds of data. Chris, do you want to? Yeah. So let me let me let me quickly go through a couple things here. So first let me start with Luis, so fantastic. First of all, I'll point you to the petal website, the program I run for DFID, where there are four research themes, one of which is micro founded macro model. So we are trying to push in that direction. I can think of, you know, a handful of people who are I think are doing fantastic work in that area, but it's a very small handful and I think it's not enough. And we've had trouble getting good proposals on that theme in particular that I think is is particularly challenging. But I agree with you completely that that's an area. Second, I would say, I don't think I wanted to say that that scaling up micro enterprises is the way to move from the top left to the bottom right. In fact, by bringing up John Sutton's work, I think I'm saying that, you know, the evidence we have on large terms, a lot of them start large. There's there are other reasons to be interested in profit and and and and income from micro enterprises. That's that's tends to be people from the lower half of the income distribution. Even if they don't grow, if you can increase their incomes, there are reasons there are reasons to think that that might be a good thing. Clearly grants work differently than than than than loans. I think that's again, I think I see this as part of the learning process of trying to understand then what is what we need to do about the loans to make them work more like grants. And and I think that's a that's a you know, I think the research itself sort of illuminates an area. Now, the other thing I'd say is it illuminates an area. This is this is very much the the the the stereotypical. Why are you at? Why are you looking there? Because the light, that's where the lights better. You know, we can do we can do RCTs on on micro enterprises. The samples are big. You know, small grants are enough to get to get a shock. We can't do that very easily on large firms. And I think that's that's a problem. And that's the problem I want to sort of highlight in this, that I think the focus has been too much on on small firms because the methods that have been used have been focused on small firms. We need to think about how we how we how we how we do something in in in in larger firms. And I think we're making some progress in that area, but it's it's come later. And so I can't point you to a lot of things yet, except things that are that are that are in progress. And the last thing I want to say is is that you know, to say that we also I think David and I are not saying that training for micro enterprises doesn't work. We're saying that we don't know anything that we haven't learned because of and again, I guess when I reflect on that, I want to say maybe we haven't learned because this is just the sort of problem that RCTs are not so good at at at at solving. But but that's but but but perhaps there's there's a design out there that that would that would actually make more more progress in that in that area. Just to just to compliment what I'm saying on on the long term. So I mean, so I completely agree the kind of the the concerns about attrition and general equilibrium are mythological concerns that can be addressed within our city or without. No, it can be costly. And so that kind of requires that when I mean, in a certain way, the question is more when do we do that extra effort? I think that's one part of the question. And the other one is thinking, if we're interested in the long term, we should be thinking about that when we design those studies, and and go from there. So when we these when we design the studies, and we know we want to know the long term, then the certain types of information we need. And if the general equilibrium is a concern either on the short or on the long term, we need randomization at a higher level to take that into account. Now, and there is a tradeoff there between the short term and the long term potentially, but that that's something that we can consider. Now, the other part though, I think is and this is coming back to kind of the part that I skipped over. I think kind of understanding the context and the house of decision making and the theory can also help us understand which parts of the population we expect a long term impact on. And that then allows to so in a certain way, going away a little bit from the average treatment effect on the entire population and thinking about, okay, where where should we focus or efforts? And then because then it becomes easier if in the sense of if we or if we track the population that is of the highest interest, and that's, you know, a more defined population, the ones where we have seen short term impacts, for instance, to kind of because if there's no short term impacts, it makes sense that there won't be much long term impacts, although I guess you can tell a story where that's the case. So I think it's an issue of being of being more innovative in a certain way. And so that's kind of the I didn't explain the methodology behind to the graduate results I presented, but that's kind of what we did. We focused on the cohort that should have had the highest impact based on the drop out rates, etc. And then we did a lot of effort to find them no matter where they went. And then we and then we used kind of attrition methodology to bound to whatever we find subsequently. So I'm we started a bit late, so I'm going to take two more two more questions and John maybe. Yes, John Rand University of Copenhagen. It's a question for Chris following up on Louise's comment and referring to the sheer work with the clino. They also did some work or he did some work with Ben Alton as well about the missing middle. But what I took from that work is actually what you also said now that we may be focusing a little bit too much also on research on the small and medium scale enterprises. This has led policymakers a little bit to focus their credit schemes on the small and medium scale sector because researchers doing a lot of effort in documenting that you have big effects of actually granting capital grants to these firms. However, the sheer and ultimate work shows that large firms may be more relatively more capital constrained and they are more job creating as Louisa maybe is you referred to. So are we doing something are we biasing policymakers as well by the research that we are doing on focusing on these experiments on the small and medium scale enterprises. Thanks John. And then one last question over here if you can make it quick please. I'm from the University of Gothenburg. I have a question for Professor Guthrub very short. You mentioned about capital labour and technology but as I come through your work there's also one more the surrounding institution of firms may impede their growing bigger because moving bigger may become more visible to the public and you have done a job in Vietnam showing that firm relying on informal arrangement to set to dial the contract and I've done the same dishes in SME in Vietnam and I felt that paying more bribe likely to have in compensatory industry likely to have environmental certificate and this paying bribe firm is also likely to is it through surveying route. So my question is is this a a venue for more such on why cannot grow in big. Yeah okay so I'm missing so the last question is a great question or by biasing policymakers by focusing too much attention on this. I have to say I think in most contexts the large firm owners are better connected politically and have better access to capital than small enterprises do. So maybe there's a sense in which we're offsetting we're offsetting it but that's not the right answer because clearly we do need to look at at at what's happening in large firms as well. You know this is a place where I think there's some there's some work out there. Ester and Obj would have a paper that's been a working paper forever and I think now is finally published using changes in policy for sort of slightly larger middle-sized firms in in India that looks at returns to margin returns to capital investments where set-aside programs were changed the the thresholds for set-aside programs were changed that allows them to look at access to capital for larger larger enterprises. That's the kind of stuff we're going to have to do because you're not going to give grants to to firms where it's a hundred thousand you know there's a hundred thousand dollars. It's going to have to be things things like that. I know several I tried myself in in Mexico Santiago is still gone but so I tried myself in Mexico not not not Santiago was involved but but to do something with a program on credit guarantees that you know so this is the government spend hundreds of millions of dollars a year in subsidies for large-scale firms through through credit guarantees large-scale medium at least medium you know medium size much larger than that than the small so we have no idea whether these work or not whether it's all inframarginal or whether there's anything that happens that extends the margin. I tried to put something together in Mexico I know there's a Antoinette Shore and Russell Tosa try to put something together in in Indonesia I think that one's still alive but but barely. These are really hard programs to try to to try to assess and I think we we have to kind of go back to to thinking about how we use the data that's available to answer those those questions. I agree completely we shouldn't we shouldn't be ignoring that I think they're much more challenging and I think that's where we where we need to move. Surrounding institutions I also agree completely I think these the micro enterprises I'll worry less about that than I do once firms get to a certain a certain size and and the the training the things that have been done on training a few of them go to slightly larger firms and you're going to run into to that they're going to be constraints from from being visible they're going to be constraints from running past your known network of suppliers and and and customers that make it hard to that hard to expand more and so forth. Again I can point to a couple of things that are ongoing Dan Keneson has a has a project part of his BRIC project where he's wants to do something that's going to shift the cost curve that's going to tell us a lot about about the expansion of firms that are slightly larger and and what the demand curve for that at the firm level looks like which is I think one of the big research issues now is I can think about industry demand curves but I want to know about the firm to anchor that look at a place where there are 100 people doing exactly the same thing textbook tells us they all face flat demand curve they don't they don't for a variety of reasons including institutional ones you mentioned and I think we don't have a good sense of how much friction there is in those markets and I think we need to we need to think that out I can think of some projects that are that are ongoing in agricultural markets and are ongoing in you know Dan's work in the BRIC sector and some others that that are that are doing that but I think that's a that's that's a that's one way to kind of make progress in that in that area okay so I like Karen and I maybe to sum up and if you have any last comments I mean we have been working with large firms the you can have access to administrative data you can work with large banks it's a different set of contacts there are less degrees of freedom but you have a larger database as well right so I think the working style is different but I don't see why a priory there's such an impediment to asking questions about large firms that you could ask about small firms okay so I'm afraid that's all we have time for so I mean I agree that it's been one of a very interesting session and we have come a long way but it's a lot further to go in relation to looking at the microeconomics of development so I'll let you all go to your lunch now and I won't say any more so thank you thank you everybody