 Good afternoon. Welcome, everybody, to this side event session, which is entitled, Missed Realities and Solutions Towards High Integrity Forest Carbon Credits. My name is Sven Wunder. I'm a principal scientist with the European Forest Institute, and also I am a senior associate of CIFOR, of the Center for International Forestry Research. And let's start with some housekeeping, which has now disappeared from my screen. Sorry. So maybe I can say first that there will be interaction with the public in this session. On the one hand, there's a chat box with is reserved for technical issues, and there's a question and answer box, which will allow for questions to be posed to the panelists. Then there is communication regarding the recording of this session, and I quote, we wish to inform you that the event will be recorded to facilitate note-taking after the meeting. Data will be destroyed once this purpose is served. Please let us know if you have any questions or concerns. Good. So as to our panel, I will briefly introduce Kevin Brown from WCS. We have a online, also, Aded Angelsen and Aaron Sills. And we have Thuy from CIFOR eCraft, also as a commentator. We are trying to, it's an exciting issue. We're trying to span perspectives going from the academic to the more applied and practitioner's perspective. We've seen a remarkable growth of the carbon market over the recent years, and that is, in principle, a good thing. But we've also seen quite a lot of critique, as with the Guardian article in January. And a recent editorial from the Guardian starts off by saying, the emerging carbon of this market is chaotic and dysfunctional, full stop. So there's a lot to discuss. I hope we're going to have some interesting viewpoints from our panelists and from you here in the audience and people online. And with that, let me start with Aded Angelsen. Aded, could you share your screen and start your presentation? Aded Angelsen is from the Agriculture University in Norway. Has been working on deforestation issues for many years. Yeah. You have to stop sharing before I can share. OK. So good morning or no afternoon. Well, it depends on where you are. So you see my screen OK now? Yes. It's not full screen yet. Here we go. That full screen. So good afternoon. Thanks for the invitation. Sorry for not being enrolled, but I saved a few carbon emissions, although the plane would probably have flown anyway. I'm going to give a brief introduction to the title is Records and Carbon Market. And just some of the basic issue that hopefully can provide some framework for the discussions and presentations also that we are going to have this afternoon. So originally, when red was consumed, COP15, maybe I'm mixing two dates here on the screen, COP15, that was 2009, but also two years before in 2007. The main idea was to quote the Bali declaration on red was policy approaches and positive incentives. So positive incentive is UNFCCC speak for result-based payment, payment for environmental slash ecosystem services, hash on delivery, whatever you want to call the baby. The core idea was you pay based on verified or certified emission reductions. That's the basic idea. The implementation has been slow. And up to now, I would say red has mainly not been about paying for results. It has been worked on Phase 1 on Phase 2 of red, which is Phase 1 being kind of the preparatory, laying the ground, building up the MRV system, Phase 2 being more the first part policy implementation, and then a third phase as payment of results. But we are increasingly doing so. So when I teach a bit on red to students, I start very basic and I thought should do the same. What does it take to create a market? In this case, a carbon market. And note that I use market in a kind of a broad sense because it is include any type of result-based funding really paying someone for delivering a particular kind of service and emission reductions in our case, whether the money comes from private investors in a carbon market or from public firms. So here are the four elements that you need. First, you need a commodity or service and the emission reduction, which is not a very tangible thing. So you need a couple of things. You need an MRV system for measuring, report, and verify that the change in say, forest carbon stocks over a period or the emissions. But this has to be measured against the reference level. And these are kind of the two elements of defining the commodity. You need someone to sell it or providers in a past system, which is about carbon rights. It's about benefit sharing. I'm coming back to this in half a minute. You need a set of buyers, basically three sources, compliance market, voluntary market, or public funds, for example, development aid. And finally, you need some rules of the game, some institutions, some marketplace, and standards. So for any type of market or market-like mechanism, this is what you need, the four key elements of any type of market. And when we talk about red credit credibility, it's linked to getting all these four elements of a market in place and having credibility on all this, whether it's benefit sharing, how you have defined emission reduction, et cetera. So for the buyers, you have essentially three sources. Originally, when we talked about red in Bali in 2007, in Copenhagen two years later, when we didn't get the Copenhagen protocol or Copenhagen agreement, the original idea was cap and trade, model after the Kyoto protocol, where countries take on commitments. And you can meet those commitments, as you did under the Kyoto and the clean development mechanism, by buying credit in other countries, or trading among, say, among annex one countries, as you also could in Kyoto. That was the idea. Compliance market where red can be used as an offset. It's a compliance because there are quotas set, caps being set by some authority and national government, typically, or it could be a state government, as California said. This is a mixed bank, basically because of the reluctance to take on commitments by countries' legally binding commissions, such that you have to reach a certain. And if you don't, you can buy your credits offsetting. So the second is the voluntary market. And as Sven alluded to, there's been a remarkable growth in this. See from 2000 to 2001, where this increased by almost fourfold up to $2 billion US dollars here. Was a slight dip in last year and 2022 for reasons that we all can't think of. It's mainly the companies, I would say, the big change. The companies are committing to net zero emissions and they realize we cannot do everything on our plant, in our upstream and downstream supply chains. We have to get some offsets to do that. And it's also very much a project focus because you want to have something that is more tangible and concrete and show your shareholders the general public that this is our project, this is how we have offset it. And it need to be concrete rather than saying we bought emission reductions from Peru or Indonesia and said this is our reductions. The third one is the public funding. And this has been up to now 90% from ODA or official Development Aid. You can see the graph here how the share of climate funding, both adaptation and mitigation, as you can see that the mitigation part has increased quite a bit. And it's now a third of Development Aid. This is an increase in concern among many poor countries that climate has kind of grown too big and taken resources away from more directly poverty reducing. Although, of course, a stable climate is also necessary for fighting poverty in the future. So who are the sellers? And this is a long debate. Just touch on some few basic questions. Who owns an emission reduction in the forest? And Cecilia Luttrell, what's it say for? And a few others. We wrote an article in 10 years ago among the different rationales for the benefit share in Hustel. It can be those who do their mission reduction taking actions. It can be some based on cost, can be on the facilitators. Those are the legal rights, the stewardship, those that are protecting the forest, indigenous groups, for example. Or it can be a particular pro-poor benefit that you're using their proceeds from carbon credits to sell. In practice, a compromise between very different consideration effect on efficiency in particular one side and equity on the other side. Third element is to define this emission reduction. And in theory, it's very simple. You have an emission reduction is there using this IPCC framework. You have an activity, which is the reduction in the area of being deforested. That's the activity. And you multiply it by emission factors, carbon per hectare. And it's critical because you define the impact. Did it work? You define success, the reputation. Will you be reelected because of you have delivered in the environmental area? Or most importantly, it defines the payments in the result-based system. So setting reference level is very tricky, as we'll hear about more. It's something hypothetical because you have to define the counterfactual. What would happen without the policy? And we'll never see it. And you can never prove that this is wrong because we are defining a scenario that did not happen, namely the development in emissions without red-red intervention. It's strong economic and political interests related to that. And it's also some conceptual confusion. I've been through that when I got involved in some reports just after Copenhagen. We had the main interpretation of reference level was a business as usual scenario, kind of a more technical prediction. And that's what you use in impact assessment. Increasingly in at least the UNFCCC documents, it has come to refer to something, as it may call a crediting baseline, something that is like an emission quota. And that's two very different things to what should be the quota, from which point would you like to get paid? And a prediction of what will happen in the absence, the normal use of the reference level. And now in the submissions to the UNFCCC, it's mainly using in terms of a crediting baseline. So my favorite quote about the reference level is this, reference level is a benchmark set so low that success is guaranteed. In an early of the red books we published, we kind of made this distinction on some clarification because in COP 15, in Copenhagen, when they have some decisions on how to develop forest reference emission levels or forest reference levels, you take into account historical data and adjust for national circumstances. National circumstances, some negotiations described to me, not as a black box, but a can of worms so that you want to open it, but it's so hard that you close it. So it's up to the countries to kind of define what's national circumstances. So that's for the baseline, but then there may be other national circumstances relevant for the more crediting baseline from at which point you should be paid for emission reductions, depending on say capabilities, a climate depth and maybe some other same certainty that you would like to take into account when setting that baseline. But at least make a distinction between the two. And then the final list, the institutional rules. You need some independent verification certification. And I often compare it to coffee. And carbon is not coffee. We cannot have emission reductions for breakfast. So the kind of trivial point is that when a buyer gets an emission reduction, it's used as offset. It's a paper that you would like to use in your accounting. You don't have it for breakfast. You don't have a profit from it. You just want to have it approved that somewhere else emissions have been reduced and you can use it in your accounts. Whereas coffee, when I buy coffee, I would like good Arabica that tastes well. That is a good dark roast, which I prefer. And I have a very strong interest in the quality of that. The same kind of quality control, at least the incentives for that does not exist. Of course, there is a reputation risk. And if you look at the, at least then, the UNFCCC process, it says in the Warsaw framework on red to offer a facility of non-intrusive technical exchange of information, which is like leaving to the countries to suggest from what point would you like to get paid with the obvious kind of incentive problems it creates. In the volunteer market, you have taken a similar approach but of historical plus some adjustments. And we'll hear more about exactly how exactly that can be done. Again, I should stress that there's no kind of correct formula to be done. And you can never read it or you can somewhat test it, but never fully test it whether you got the accurate reference level. So finally, the contribution of science. I think one will hear in particular, Erin will talk about how we can learn from lessons in other fields because setting reference level is actually key in science. We normally presented under other names like impact assessment, but impact assessment is to define a counterfactual and then look at the present situation compared to that counterfactual, establishing causality. So it's very much a core and I think one of those involved in this can also learn a lot just from general scientific principles. And the second is this on a good day, I like to think that research and academics are more independent than others. So we have this. And I'd like to finish with this quote, a few years old, 1892 from a statistician. Whenever the struggle resurfaces between the champions of the general interest and that of private interest, you'll find us a statistician at our post-Armed and ready to march. So thank you. Thank you very much, Arit. Kevin, you are next and you have been working much more with the applied side also of all of this. I look forward to your presentation. Great, thank you, Sven. So I'm Kevin Brown, here representing the Wildlife Conservation Society. And thank you to C4ECraft for sponsoring me on this trip. I speak on behalf of Wildlife Conservation Society wearing a few different hats. On the one side, we're an NGO, but we do act as a Red Plus project developer in the voluntary carbon market. Excuse me one second, can I spend my, please have the slide, thank you, no worries. Sorry, one moment, let me see if I can advance the slides here. Okay, I got it. Okay, sorry about that. So we're involved in developing avoided deforestation and reforestation projects around the world. We have 15 years of experience in this. We led the development of two of the most highly regarded avoided deforestation projects in the Kira project in Madagascar and the Kiosema project in Cambodia, you may have heard of. We're field led with staff in over 50 countries. And we have staff, including myself who have authored existing and forthcoming methodologies in the voluntary carbon market. So I speak from kind of the project developer side from the standards development side. And then I come from a remote sensing GIS background and I've done enough capacity building assistance to governments on developing their NFMS, their National Forest Monitoring Systems to kind of understand what are some of the practical trade-offs of how do you ensure that monitoring is done and baselines are set in such a way that you're accurately accounting for the carbon and emission reductions, but not placing an undue burden on the process. So a quick note, why does WCS still support the RED Plus mechanism? It's really the only game out there that provides long-term sustainable financing for natural forests. People have talked about other things. They've piloted other things at smaller scales, but it's the only mechanism that's been tried and that has a large potential proven pool of financing behind it. So for the near term, it's really the only game in town for us. And it makes the carbon owners participants in the market. They're not recipients of aid. You're actually engaging them and producing a global product and you're empowering them to take ownership of their resource. So that's really important to us. At the outset of this session, Sven mentioned something on the top of everyone's mind. We've been hearing a lot about integrity in RED Plus. We wanna ensure that projects are producing credits that have integrity. We've heard integrity come under question from different angles. And I just wanna pick that apart a little bit and what do people really mean when they're talking about that? And I can at least speak to how we think of integrity at WCS. So much of the discussion that we've heard of the criticisms has been around the atmospheric integrity. So in particularly, additionality, but we also have to think about leakage and we have to think about permanence. But this is kind of around the atmospheric integrity. Is a carbon offset credit actually doing harm to the atmosphere? But there's other parts of integrity we have to think about and of course, biodiversity, are we leaving the planet in a better shape for the next generation than we found it? And equity. How do we ensure that the maximal amount of the benefits of this financial mechanism are going back into the people that actually live and depend on these resources, that it's not just some kind of profit mechanism for outsiders? So I like to think about, there was a comment earlier today in the plenary about the perfect not being the enemy of the good and that's my mindset here with integrity and red plus. We hear a lot about atmospheric integrity. You hear a lot about having equitable outcomes and these are absolutely critical. But if we have red plus that has atmospheric integrity and equitable outcomes, but is not a financially viable mechanism because it's too hard to implement, the requirements are impossible for anybody to meet, then it's never gonna scale and it's never gonna really reach the potential to actually serve as a tool to address the climate crisis. So having atmospheric integrity and equitable outcomes without financial viability is a global failure. Just wanna put that provocation out there. And I wanna delve a little bit into that putting on my project developer hat and talking to the private sector actors that really wanna get involved in this space. They really wanna get in and develop projects and make use of those offsets but some of the challenges that they face. I was interested to hear I'll already say earlier today that earlier in the session that ODA funding is actually increasing for climate mitigation. I'd love to follow up on that but I think what we found in our space that even if maybe it is increasing, it's still far insufficient to meet the demand needed for the financial, the amount of money that's required to set up a red project is easily millions of dollars and oftentimes these investments don't produce any emission reductions that result in like a net profit. You can't repay that investment for sometimes five, 10 or more years. So the private sector is looking at this like an investment. They would need to sink a lot of money into one of these projects. There needs to be some kind of assurances from their side that if they perform to a certain standard that's known ahead of time that's outlined in a carbon offsetting methodology that if they perform to those standards that those credits will be issued and that they can sell them. And there's so much uncertainty in this space that they already have to cope with. And when I say investors I really mean what I mean is anyone who has an economic stake in the project so it's not just those that put in the financial capital but it's the carbon owners too who are paying in the opportunity cost of not using that forest for something else. Because that's often a quite substantial cost. You're asking them not to get some other kind of economic benefit from that use of that forest. So there's so much uncertainty already. You have to ensure that your project actually performs against the baseline, right? You have to deal with the volatility in the price of carbon credits. You have to deal with changing regulatory environment around carbon offsetting. All of these things can be mitigated to a certain degree but the one that's new that's been real challenge is with the these discussions about high integrity and these ex post assessments of additionality are the risk that after you've gone through all of the investment and all of the work and issued these credits after how many years that you then find out that well actually the global scientific community actually doesn't think that that carbon credit is a real thing so you didn't actually produce anything. So that strikes fear into the heart of the whole community and it really has put a chilling effect on some of these actors who might be otherwise wanting to get more involved. So this event where mostly come from a forest monitoring background and thinking about modeling, forest change. So I wanna kind of tie together why are we talking about financial viability of projects to do the evolutions in the voluntary carbon market standards because the decisions that we make and how these standards are set up have implications for financial viability and for atmospheric integrity. So it's a really important balance to get right. I wanna frame for a quick second how the voluntary carbon speaking here just about the avoided deforestation methodologies were typically set up over the past 10 years. This is a reference region approach. You identify a proxy for your project somewhere else in the country. You develop a historical deforestation rate from that and then you apply that to your project area and assume that that's your future rate of loss. And then of course that's validated against using a third party audit. And for a quick second, why was this approach set up this way? Well, there were practical reasons. It wasn't because Vera or anyone wanted to allow project developers to game anything. There were practical reasons. There was no Google Earth Engine 15 years ago. There was no University of Maryland deforestation dataset. People were still classifying a single Landsat image at a time. So the idea that a project developer could go and assess an entire baseline drawing on information from an entire jurisdiction, it was just unrealistic. So hence why the reference region approach was developed. Moving to what I'm calling red 2.0, which is not an official term or anything, is that the way that Vera has been moving in a new avoided deforestation methodology that's gone out for public comment a few times, that's going under final validation as we speak, is that now all projects are gonna get a baseline that their activity data, their deforestation rate allocated from a jurisdictional pool so that no projects can be over allocated such that it adds up to more than the reference level for the country. There's no more use of trends, only historical averages that will reduce volatility in the baselines. And then the baseline projection period has been shortened to six years. So it allows more frequent updating of the baseline to follow trends, changes in forest trends. And some of the big advantages to this methodology are one, it dramatically reduces the amount of resources that have to be spent on analysis for projects to get up and running. It allows project developers and countries to understand where are the high potential places in their country to invest in. It's just known ahead of time, they don't have to go through a bunch of feasibility studies. And most importantly, by developing these data sets at a jurisdiction, it allows the voluntary carbon market to leverage existing capacities in government MRV and NFMS offices, bring them into the process and start to bring revenue into those offices and get them prepared for jurisdictional red, which is the next phase. I know I'm getting over time, so I just want to touch on two really important areas, I think that haven't, that are, we're still grappling at how to address in the best way possible. A huge issue is the risk map. The way that these methodology works is that baseline is allocated to projects using a jurisdictional risk map. There's been a lot of work that's gone into creating some metrics for how to identify which is the best risk map of all the different options, total operating characteristic and area under curve, but it's really an area that I think needs more discussion and bringing in more experts. So I would definitely advise, those with a background in that to contribute. And then secondly, I won't go through all of these points, but leakage is a big area of uncertainty. There are several different types of leakage risk that are accounted for under the vera methodologies, and all of them to some degree or another are based on an understanding of human behavior. And I can say that these, how leakage is assessed and how it's monitored has not received nearly as much attention from the MRV community as has some of the forest carbon sides. So it's an area of, I think, that has huge implications for additionality, but hasn't received as much attention. Okay, well, I think I'm over time, so we'll just end there. Thank you very much. Thank you, Kevin. So we are shifting back online now to Aaron Sills from North Carolina State University. Aaron, please go ahead. Great, thank you. Can you hear me? Yes. And now let's see if you'll be able to see my screen. We can see the file, just need a presentation mode, if possible. I have it in presentation mode now. Is that what you see? Yes, now, yep. Perfect. Okay, well, good afternoon in Rome to everyone. I'm going to continue on the theme of the first two speakers, that is comparing the methods that we use for MRV or for accounting for carbon credits to the methods that we use for evaluating the impact of RED Plus interventions. And so just to run through this comparison, again, I think we're all reinforcing the same point here. For both of these, you need the MRV piece. So you need to identify where the intervention is being implemented, the RED Plus area. And you need to monitor forest change in that area so that you can estimate the emissions associated with any deforestation or forest degradation. That gives us a measure of what happens with RED. So the difference comes in when we start comparing that measure to something else. In the first case, we're going to compare it to an ex-anti-forecast of what forest change was expected under business as usual. That is what sets up what Arl called the crediting baseline. In contrast for impact evaluation, we're going to compare the same measure to ex-post observations, contemporaneous observations of forest change in comparable areas that are also under business as usual. So there's no RED projects there and that gives us our counterfactual. So two different types of comparisons for really two fundamentally different purposes. So the first ex-anti-forecast establishes essentially a performance benchmark. So Arl suggested maybe a low performance benchmark, but nevertheless it's a benchmark and carbon credits are generated when either forest loss or emissions fall below that benchmark or what's typically called a reference level. Switching to impact evaluation, we're going to take ex-post observations that establish what would have happened in the context of other ongoing policy and economic changes, right? So this is what the world would have looked like given all the other changes going on but without a RED project. That provides an estimate of the impact of different types of red interventions, whether and how much impact they have on reducing forest loss, under what conditions, and serves as really important feedback to both investors, potential investors in RED and to policy makers. Okay, so this sets up a challenge moving on to the second part. So that's the comparison. The challenge is that conceptually this ex-anti-forecast and ex-post observation that we have in the two different approaches are trying to get at the same thing. They're trying to get at a prediction of what would have happened without RED, but in practice they're really not interchangeable, right? So this ex-anti-forecast can't substitute in to causal impact evaluation because fundamentally the forecaster is never going to know all of the other changing political and economic and social conditions that affect outcomes under the counterfactual, right? So that doesn't work. Likewise, the ex-post observation, so observations of forest change, forest loss and comparison areas, isn't a good substitute for our accounting or especially our contracting system because when you set up essentially the contracts to tell someone you will pay them if they deliver reduced forest loss and reduced emissions, you don't know this ex-post outcome, right? And further, the people who are involved in the contracting don't have control over it. They don't have property rights over the control areas or the comparison areas. So again, that doesn't make sense as part of payment upon results type contract. Okay, so we've got two different systems, one to generate carbon credits and the other to generate feedback. So I'll come back to in the end, we do want to use that feedback to improve our system for carbon credits, but that's different than compounding the two, right? So the two are not exactly the same. And just to continue to hammer this home, we can look at this graphically, which is what I'll do next if we take sort of the simplest version of the accounting approach or the MRV approach, which is to say, let's look at what forest change was going on, what was the deforestation rate before a red project came in in the intervention area? And then we're gonna compare that to what happens after our intervention, right? And if deforestation rate or rate of forest degradation goes down, then you get credit for that, right? And so reference levels are actually more complex than this, but you could think of this as really the most simplest approach to in order to do an impact evaluation of whether our red project actually had an impact and a causal effect on the rate of deforestation, we need to bring in comparison areas. And then we're gonna compare the rate of change in deforestation or degradation in the intervention area and the comparison area. And that difference is going to give us our estimate of the causal impact of the red intervention in what's called the Baki approach. So before, after control intervention. So to illustrate this, I'm gonna draw on Sephora's global comparative study on red, which was set up using the Baki approach. So we've looked at these 23 different sites where red projects were being implemented across the global south. And in most of these sites, we selected villages both inside the red area and outside the red area, selected so that they had similar underlying drivers of forest change, that is they were pre-matched. And then we collected really loads of data from before the red project started in roughly 2010. And then after the red project started, so we first went back in 2013. For today, though, I'm just gonna focus on remote sensing information about forest cover in these areas. So we did use the Hansen Global Forest Change data. And what I'm showing you here is taking the before-after approach, we looked at all of the 23 initiatives and we calculated for each of them, what was the average annual deforestation before the project and the average annual deforestation after the project? And if the average annual deforestation went up, then we say that's a poor outcome in this before-after comparison. If it went down more than 0.1, because the range we're using here, we say it was a good outcome. And then if it was about zero, neutral. So that's that simple sort of before-after approach. You set a reference level based on what was happening before, and then you look at what happens. So not great outcomes. Nine projects had increasing deforestation, but of course, you should immediately be thinking, maybe that's because the background level of deforestation was increasing. And so you actually want to compare these project sites to the rest of the landscape. And that's what we do here. So we compare the deforestation and the project sites to deforestation and the jurisdiction that includes those red sites. And we find that fewer of them increase deforestation relative to the background increase in deforestation in that jurisdiction. And then next we move down to look at the specific villages we selected. Again, we selected villages inside and outside most of these sites to find comparable villages. And what's most notable here is when we compare villages that were matched, that have similar underlying drivers of deforestation, we do find that a much smaller fraction of these sites look like they have poor performance and a much larger fraction look like they have good performance. So the conclusion is that careful selection of control areas in this case is actually revealing that these red projects had an impact. They reduce deforestation relative to what happened in the matched villages. And just one more illustration here and then I will wrap up. So that suggests that the controls are really critical to select carefully. So again, what I did in that example or what we did in that example was we picked matching villages. But another approach that was referenced in the Guardian coverage that Sven talked about is to look for a weighted combination of potential controls that are both matched in terms of the underlying drivers of forest change and that are following a similar trajectory of forest change historically. Okay, so that's an alternative way to find a control set or a comparison set. We applied that to 12 red projects in the Brazilian Amazon. This is work led by Tali's West and published in PNAS in 2017. So in this case in these 12 projects, what we did is in red here, this shows the cumulative deforestation in hectares from 2000 to 2018. So shows cumulative deforestation going up as you might expect in most of these 12 projects. And then in blue, what we're doing, let's look at the top left here. Before the project's implemented, we're looking for a combination, a synthetic combination of other sites in the Brazilian Amazon that follow the same path of deforestation. And then we look at what actually happens to deforestation in that synthetic control after the project was implemented. And in this top left case, we see that in red, the synthetic, or in blue, sorry, the synthetic control had higher deforestation than the red project in red. So this is what you would hope if you were running a red project is that your deforestation is lower than the deforestation in the control. With this method, we can actually test for statistical significance. And what we find in this study is that in only one of these sites is there a statistically significant impact deforestation. So only one site manages to reduce deforestation in a statistically significant way. All of these projects, however, are claiming credits for reducing deforestation and reducing forest carbon emissions. So again, I wanna come back to the main point here. How is that possible? Well, it's possible because for evaluating the impact, we're looking at what happened in comparison areas after the projects were implemented. We're tracking those outcomes over time. Whereas what the projects are doing when they set up is they're looking at the information they have in hand at that point. That's forest change in their project site and in comparison areas. And they're using that to build a forecast. Okay, so I think what we need to remember to do is sort of to use the information from the impact evaluation to help improve the system for forecasting the counterfactual. So back to the theme of not letting the perfect be the enemy of the good. We don't need to expect or we shouldn't expect that these are going to be identical because they're based on fundamentally different information. But we should fully expect that we're gonna use the information from impact evaluation to design better accounting systems that maximize the incentives to maximize reductions in deforestation. Thank you. Thank you, Erin. Great. So we're turning back to Glennery and I can give you a short presentation on a meta-study of how Red Plus projects have in fact, what impacts they have created. And we start out from a theory of change of what Red Plus is trying to achieve with some funding flows from bilaterals and privates, et cetera and some information about forest stocks and their development in the past. We typically have a mix of measures, some incentives, some disincentives and some enabling measures that are treatments. And then we wanna see some outputs. We wanna see that people actually understand that intervention that they participate to a certain extent on the ground. And this then should have some reach some outcomes in terms of reduced deforestation on the ground and perhaps also some improved incomes. And in terms of impact, obviously our end goal is to reach sort of mitigated global greenhouse gas emissions and some side objectives regarding biodiversity and human wellbeing, et cetera. But often our impact relations will actually be at the outcome stage in terms of measuring land use impacts. So in this meta-study, we have to somehow delimit what we wanna take on board in terms of the actions that are taken that is number one to four here and in terms of the publications that we want to include. And in terms of the actions, for instance, I'm not gonna go through every detail here but we excluded pure deforestation projects. We did include as a several category national past schemes that had a carbon that also had a carbon focus. In terms of literature, we are including we are including both gray and peer-reviewed literature but we wanna focus on the end part of the theory of change that I've just shown you on outcomes and on impacts. And we wanna taking a rigorous approach in terms of counterfactual approaches that are being used in quasi-experimental methods. Okay, so what do we get in terms of global coverage? This shows like the density of red projects. So it's the areas enrolled in red plus projects over the forest area which is interesting in the sense that you actually have quite a lot of red projects relatively speaking and for instance, it's part of East Africa and then the Indian countries compared to Brazil which has absolutely many red projects but actually not so much in compared to the huge forest areas that the country holds. And our sample is quite well, that's the red part red-colored part which is fairly well distributed across the tropics with some areas the Congo basin for instance is not well represented also some South Asian red projects, et cetera. So what are our results? A, we split that up into the sort of the red projects and programs and then these public PS programs that have also a carbon focus. And you can see for both of them we find a significant impacts but they're quite small and they're quite variable across different projects. But there is at least some causal effects to call for. What about the permanence part? That's a particular interest for forest carbons you know and in principle you could imagine if you look at the diagram in the lower part here you could imagine that after a project has stopped which is the second vertical line you could either it has managed in the red part to reduce deforestation it could either continued on that reduced part like S1 that would be great. It could continue on the old path that is S2. It could also continue towards the old part but pick up really eat up that forest conservation gain that you would have reached during the project or you could at the extreme have a really accelerated worse than before kind of scenario of accelerated deforestation. There are few of those assessments after project available but what we can see is that it's kind of the S2 part is the one that seems to be dominating you get a parallel trend but you keep kind of the transitory gains you have made during project implementation you were able to keep those. We also assess the welfare part and what we find is even smaller positive outcomes in terms of impact sizes. Typically people don't get worse off in terms of incomes and assets and consumption that's the outcome stage but at the impact stage which would be the self perception of how people their subjective well-being there you sometimes do get some negative results and I think it's both a result of in some cases equity outcomes you may not be worse off but you feel worse off because some other people got better off in your village for instance and also some expectations created around the implementation of these projects that are not always satisfied for instance when projects are not being able to be continued. So we also looked at what makes a difference in terms of within sample variation in terms of what factors seem to be boosting effect size and clearly if there is a real differentiation that's threat to start with in the place where the project is being undertaken that helps in terms of that's quite intuitive that if you really fight against something then you also more likely to get a result. So that's important is spatial targeting towards high forest carbon density, high threat within your site for instance road near areas, et cetera. Those that did spatial targeting also achieved better results and on the welfare side perhaps a bit surprisingly when there are benefits are being differentiated that which is often used to squeeze rents, et cetera. Then sort of the welfare effect is larger and that's maybe because you were able to customize then two different people's needs, those benefits generated. So how well does Red Class do compared to other kind of conservation tools that are out there? It seems to be that this is the kind of just a coinciding effect, you know, the higher it is, the more impact you would have and you can see compared to other enabling measures or disincentives like command and control or other incentives, then not significant differences really, there's a lot of variation within these categories but at least it does as well, at least as well or as poorly as other conservation tools. Still it is the case that many of the conservation things we do do not really sufficiently large scale change behavior on the ground. We still have things to learn about this. Of course, this does not analyze cost effectiveness. Some of these projects are expensive in terms of having like a boutique-like character that shows off in various ways. So this is not integrated in the assessment. And then I wanted to just briefly come back to also the discussion that all the previous speakers have taken up about the additionality, about baselines and about the impact evaluation. So these are similar figures, illustrations as those shows shown by Erin, but for Thales' West's newest research where the approach of synthetic control is being extended to a global sample. And this has also been used as background material for the Guardian article in January. And obviously I don't wanna go into details, but you have here both the defect to observed deforestation in red. You have some deforestation synthetic control sites that in blue, and you have the extended deforestation baselines that the projects have been using. And I think what you can see even at a quick glance is that you see project starts with the vertical lines and you can see a lot of these projects have yellow lines that compared to the past deforestation move up like hockey sticks, right? So they assume that after the project starts that you would get a really large pickup in deforestation. And that's problematic, of course, when you get that at such a disseminated scale of different projects. So my point here is that those sort of critical results we have received like in this study that's more than 90% of the carbon credits were non-performing, they are only in part due to the relatively modest impacts on forests on the ground that these projects have had. They are obviously to a larger scale due to some of the assumptions that initially have been made about the baselines adopted by the projects. So five quick takeaway messages. First, a couple of contextual ones in, I think if we look at the performance of Red Plus a lot of these things have remained under finance. They did not have sufficient security about future resources being available. And a lot of the projects have really been a drop in the sea compared to the overall target of mitigation. And the things that they have done on the ground is really a mixed back. It's in a way, Red Plus has been to carbon, what integrated conservation development projects has been to biodiversity. A sort of an umbrella for quite heterogeneous mix of on the ground interventions. As to the impacts that we do find in this study, they are typically statistically significant. They are modestly in size. They are only permanent in the sense of the transitory gains made that I mentioned. And in terms of the record is just as good or bad as so many other conservation tools. We believe that Red Plus could have more of a forest conservation impact. If these interventions were more spatially targeted in the sense of choosing to start with areas where the deforestation is high rather than low. And also then within your site giving priority to those areas that are really the most threatened to deforestation make sure that those are enrolled. So this may sound intuitive. I see people nodding in the audience, but it's not exactly what characterizes those projects that we are looking at. Not for all of them at least. And I think for the integrity of carbon credits, I also think we really need to look into how the baselines are being constructed in the first place. And Kevin mentioned now there's a reform process going on where we're trying to also learn from these impact evaluations that science has brought forward and perhaps find some methods that are better equipped. Thank you very much. So we now turn to the discussion part. We have a short intervention from Tui as a commentator. Please, Tui. Thank you so much. Thanks to the presenter for a very impressive presentation. I think that I just wanted to pick up one of the things which is Board Arrow and also Kevin mentioned in their presentation which is looking at the social integrity aspect and the fact that we need to have an equitable sharing of benefits as well as the involvement of local communities to ensure the equities and the Venice are taken into account. And just based on that few things that are the results from our 14 years of research under CFO Global Comparative Study on Red Plus, looking at both national policies on price carbon market as well as looking at the project level. I think that why we see a lot of improvement and a lot of discussion on how to advance the methodology to evaluating the carbon credit. I think that the progress in terms of non-carbon benefit has been much lower compared to the other aspect. We have seen in many countries some of the key enabling conditions that Arrow put into his presentation such as institutional framework to clarify who's on the carbon rise and who should be the beneficiary to receive the payment. This in pending in many countries after 15 years of implementing Red Plus are still in the process of designing beneficial mechanism and the countries have a lot of difficulties in this. The other thing is when we talk about the cost and benefit in terms of Venice. I think that we have heard from Kevin in terms of a lot of improvement and advancement in terms of methodology to reduce the cost to analyze the data and collect the data. With our Global Comparative Study on Red Plus in 2019 now where we're documenting the administrative cost of the government, both national and sub-national government. Most of the time the sub-national government as well as indigenous local community have subsidized many costs of carbon project but these costs are not taken into a cow, not recorded and causing quite a challenges in terms of discussion in terms of contract arrangement between government agency with the indigenous people and the private sector. So this cost has to be accounted and taken into a cow. The other thing is Sven mentioned earlier in his slides about some of the projects are very expensive but in many cases the cost to do a proper free prior in forming consent involving with the local communities require a significant financial resources and unfortunately if you're looking at many national Red Plus policies and the first carbon project based on our global database the cost given to gender mainstreaming or to implementing social safeguard receive a very modest amount compared to other things which brought our attention if you wanted to commit it to social safeguard and also equitable outcome you also need to provide sufficient financial resources to it. The other thing that we wanted to also highlight in our research is perhaps with the forest carbon market we always think that it could be one of the policy instrument leading to transformation of chain but in fact we also need to have transformation of chains in many things to enable forest carbon market particularly with the land tenure system with the social safeguard which has seen I think that underdevelop compared to the other advancement when other presenter talk about the methodology. Yeah, so just a quick comment from the equity perspective. Thanks Ben. Sorry, thank you Tui. Excellent. So please think about any questions here in the public or put them into the Q&A box while I'm giving the word briefly to briefly to Aril for also equally brief comment. Aril are you there? Yes, I am. Okay, thanks. Very nice for between interventions and just a couple of points I would like to I think Kevin you mentioned this that you have a situation where project developer has a good project as delivered and then comes Sven and colleagues and publish an article telling maybe there were no emission reductions after all and used inflated baselines and so first they have to say that this is not new I remember the first no camp project in Bolivia where Greenpeace 2009 I think it was came out and very strongly criticized the TNC for having exaggerated the impact which was I guess the first forest offsetting project for American utilities so it has been a debate but I do see the problem and there are a lot of risks and one is to find yeah you see that it doesn't hold I think the thing that Aril when you compared the exposed kind of that you normally do an impact assessment versus the ex-ante before the project that that's something we can learn and it was also in 2009 I have to check the article was speaking that comes which have this successful compensated successful efforts so there is you set a reference level but you may say have a formula where you adjust it when you see what happened one example is this what happened these days it's much higher commodity prices that that may put a pressure on forests so if you have say the reference level the business as usual scenario depends on how commodity prices that would actually reduce the risk of an investor because if you have a corona situation with high commodity prices putting a pressure on deforestation you're more like sorry you are less likely to achieve a certain emission reduction based on a predetermined on ex-ante reference level if you could adjust and say that okay there were increased pressure on forests so therefore we we adjust upwards the reference level so you compensate for the effort rather than the actual reductions when we have the information afterwards so this should actually be a risk reducing thing and should be welcomed by project proponents I think the problem is don't make it too complex keep it sensible sensible simple kiss the kiss principle also applies here but it would be interesting to hear from Kevin also whether you know you're not with Vera anymore but but that and private investor whether they would think of some simple formula for some exposed adjustment of the baseline that actually will reduce the risk hope for a project developer thank you also great inputs I will now turn it to the public because we have been talking a lot of ourselves so we want some some inputs from the rest of you and I will take those questions that are now and then the panel can can react I have the lady in red over there to start with oh very awesome great and I've got in also yes thank thank you thank you for the really impressive presentation we have a lot of about the history of red and the impact of the red the implementation of red plus project but what I feel is that perhaps we feel a little little piece on the context because from Bali and Copenhagen the context has totally changed because we had we have had the Paris agreement but more than this we have the national transparency framework that set clear requirements for all parties so all parties now have they are national determined contributions so climate target in terms of emission reduction the most of them and all parties have to submit at the latest by next year so we are the we are there have to submit greenhouse gas inventory with categories so no more red we have Lulu CF categories of forest land or land converted to forest land and whatever and then more than this we have a space for article six so for market-based mechanism and so for project to be run at local level or national level or sub-national or whatever but each crediting activities should be reconducted a reconciled a national level with the with a tabular format to be submitted by each country parties every two years and this could be taken to account because it's really nice this kind of reflection because we learn a lot from the red plus implementation but we have to change in my in my view we have to change our idea in order to adapt to the new context thank you yes thank you so much thanks for the presentations and the discussion and my comment are sort of related to the lady to my right here and it's really going back to so we have heard sort of one version around red plus which is very much about establishing separate isolated projects where there is an investor that has a need for a return on that investment and we have heard and learned a lot about the problematic sides of that with actually establishing an effect with the accounting and so on but sort of going back to the original thinking around red the way we perceived it and I'm sorry I forgot to say I'm with Norway's international climate and forest initiative we have been working on this for 15 years and our starting point has always been that you need to approach this at the national level so a national level approach to red plus means having a national reference level it is linked to the policy implementation which is necessary for the transformation that we are really seeking to make happen through the financial incentives of course on the ground those the actual overall outcome in terms of reduced deforestation will be the result of a lot of different projects but having them all being part of one national strategy under the same reference level is extremely important to avoid the issues around leakage that have been described and that we all know to give the basis for benefit sharing and so on and within that context red plus is much more an instrument or a mechanism to support forest countries towards low emission development to changing the trajectory of economies more dependent on deforestation than on forest protection and sustainable forest management and I miss that perspective and I think just as was said by my friend over there linking red plus to the overall national determined contribution as one area where you can have international support because the race system established in place is also an effective and easier way I should say of implementing red plus. I would last my last point is regarding the issue of reference levels and additionality and it's just a very simple approach in saying that historical emission levels are well established as a straightforward approach to how you start off with the unambitioned level for a climate target and so it's really treating the forest sector the way we treat other sectors. It would be interesting to hear reflections around this discrepancy between the national level red which we have been working on and we see now is really picking up with some very impressive results from Indonesia and several other countries compared to what has been tested and where we've got important insights from but which really enters into a lot of more difficult issues. Thank you. Thank you. I think I don't know who was first actually maybe since you were closer by and then and then you yes. Thank you. Murray Suarez from the MRV unit of Mozambique. So first I think the the issue with these some of these articles criticizing the results of red projects in my opinion has a lot to do with the distinction between avoided deforestation and reduced deforestation so I think a lot of these projects that go into an area that doesn't have deforestation and predict deforestation are a lot more prone to these biases than a project that aims to reduce deforestation over over a large area and I think scale has a big has a big relation to it because you we are I think transitioning to a phase where we go from you know lots of small projects more jurisdictional projects and that's the approach that that we've had in in Mozambique and so I think with regard to to the the the presenter Aaron Sill that talked about the control sites I was wondering how how she sees this this working when countries are going towards jurisdictional programs where you may have the entire country covered by jurisdictional red programs and you then may may have difficulty finding these control sites because as it stands now if you have a very small project area then it's easy to find a control site but imagine if every province of a country or you know large areas are covered by red projects then then you may you may have difficulty finding these these sites so yeah thank you thank you I have you here and I'm going to ask a question as well and then we're going to respond yeah thank you very much for the presentations I have to say that I ended up a little bit confused I was trying to to find a common denominator like a wrap-up message to the to the talks and I I have to say that I got mixed messages no on the one on the one hand I will I saw a serious advocacy towards how projects have been done and how they have or have not delivered right but then for example from Sven's and Erin's presentation I see reference to okay to compare the performance of projects we need synthetic comparisons I imagine that it's by statistical matching and these type of things that include a bunch of parameters we have used but I don't think that but it entails that there's a underlying message that the way the reference levels for carbon products has been established up until now it's probably not the right one okay so I'm making sure that I understood that correctly so that's that's that's that's a good one the other thing is that there's a lot of things to connect here what was mentioned by jurisdiction I think that there's a lot of things to connect here what was mentioned by jurisdiction and how carbon products fit in overarching national or subnational strategies I definitely think that projects should be part of the implementation strategy over broader strategy basically of the broader area of intervention but these also as Sven was pointing out need to be in the room that perhaps touches what Caving was was advocating for which is the fact that we're missing the conservation of carbon stocks in the equation of the products and recognition for those and this is perhaps something that needs to be to be brought up because and that's why we see a lot of conservation projects being labeled as avoided deforestation projects and that's why we have these issues with the baselines but also because there's a lot of things that we need to think about and think about ourselves for when tackling deforestation is successful right because then we will move into okay we stopped it but now we need to maintain these results and then there is some there needs to be some some element of a maintenance fee or an insurance or something that will sustain these in the future so I didn't so I was thinking about these types of mechanisms and things and you can't mention that here yeah and then the final one the other element in the room is the fact that we need to pay for implementation that's clear implementation is costly and then of course it's easier to implement where nothing is happening if although it's where it's actually happening. So that becomes a little bit of a perverse incentive, right? So yeah, some ideas there. Thanks. Thank you. But are there any more questions in the room? OK, last one. Yeah, more than a question. It is a comment. I think there have been a very good point here. There has been a change in paradigm since the Paris Agreement, because now every single country has commitments. And that was not the case when red was designed. And to fulfill those commitments, countries have to increase their ambition over time every five years. And regarding that, they may look this sector, because it's very attractive for themselves. So I think it's a change in paradigm. We don't need to think that projects or results-based payments for red coming from outside of the countries is going to be the main avenue for increasing ambition on the sector. The second comment is, when red was defined, the idea was national. And we had long discussions about that. And it was pointed out behind me as well. And we are only been talking about incentives here. But red plus is about policy and incentives. And for example, when you deal with your reference areas for the counterfactual, if you have managed to establish a policy enabling conditions, these may affect also these counterfactual areas. Particularly because these policy approaches are national mostly. So I think we have to reflect on what the discussion was at the beginning of red plus, which was incentives are not only monitor incentives, and policy approaches are equally important. And the fact that the Paris Agreement also changed the setting of the rules, which doesn't mean that projects may not be useful. But maybe we have to rethink about the objective of projects. I think that's what lesson learned is. And despite of the very interesting presentations I learned a lot today, what it led us to go is towards more complexity. So I wonder if this won't deviate the attention to where it needs to be put, and with this in changing and transforming this sector. That's, I think, my reflection. Good. Briefly, online comment. Yes, question from Askari, Indonesia. Indonesia have done restoration on peatland hydrological function through construction of rewetting infrastructure, such as canal blocking, canal backfilling to decrease the peat decomposition. Therefore, it reduced the CO2 emission. The restore area about 3.7 million hectare in concession or company area, and about 1 million in community area. Can this achievement be counted or claimed in red? And how to get the funding or claim for this achievement? Good. Thank you. We have a large pool of questions, and very little time to respond. I will just start out by saying very briefly that the results I represented in the meta-study did, I mean, was to talk to you on national versus project level. So they did actually include some jurisdictional approaches and also some country programs, including the NIC fee interventions in Guyana and Indonesia, were impact-related, and in both cases, found a modest but positively significant impact. The thing is that not a lot of these things is that the research will always be backward-looking because it takes some time to analyze the data. What really worries me for the future is that I don't see the jurisdictional approaches as they're being rolled out now in practice, being accompanied by efforts to do serious impact evaluation, because often the donors would have to ask for that these things are being put in place so that they'd be in five, six years can actually get some result of this type for the jurisdictional approaches. That is not happening today. I turn it over to the others. Great, thanks so many great comments. I think I agree with all of them. But I just want to touch on two things I heard that I think are super important, and I apologize for not coming out more in my presentation, that I completely reject a dichotomy between projects and programs. I think what's happening is that you're starting to see them merging together. We're always going to need location and site specific interventions to achieve results at a jurisdictional scale. You still need actors who are working in particular places, working with particular investors to actually get those results that you can then package in a jurisdiction. So I think what's so exciting about the new VARA methodology is that it does actually link together now the project accounting with jurisdictional accounting. So if you can get that up and running, you can start to generate revenue to finance the jurisdictional program and then help to ease that transition to a fully jurisdictional standard in the future. And then secondly, the question about red was supposed to be jurisdictional. Completely agree with that. But red is a mechanism that targets developing countries. And I think it's not totally appreciated, two things. One, how much money it would cost to actually reduce deforestation at a jurisdictional scale? Billions of dollars. Billions of dollars. And countries don't have those types of resources. And the private sector doesn't want to work through national governments usually. It's just a hard truth. But they want to work through site-based interventions because they feel like they have more control over the process. So I think I'll leave it at that. Thank you. Erin, do you have any comments on what has been said? I want to give everybody the chance to react. Sure. So first, I agree all great comments and raising important points. I agree that certainly there is an institutional drive and probably a need to shift from the project to the jurisdictional approach or to merge, as Kevin said, to my mind that doesn't eliminate any of the challenges we talked about. It just moves them to a different level. As Ben said, we still need impact evaluation at the jurisdictional level. We still need to think about how to set reference levels that provide incentives to generate impact. And even more challenging, we need to think about as we transition there, how do we attribute any reductions in deforestation to site-specific projects versus jurisdictions. So I think you can translate everything we talked about to the jurisdictional level. Second point I wanted to make is in response to the question clarifying whether we are in fact saying that our impact evaluation shows that the way reference levels has been set doesn't accurately demonstrate impact or measure impact. And I would say that is true. Our impact evaluations have shown that reference levels were inflated measures of what would have happened in terms of deforestation without red interventions. My point is that it doesn't necessarily mean that we need to substitute or that we can substitute setting crediting baselines with quasi-experimental counterfactuals. What it tells us is that we need to think carefully about how to set the parameters around methodologies for setting those reference levels so that they are set in a way that really incentivizes maximum impact in terms of reducing deforestation. The problem with inflated reference levels is then it sets the bar to blow, right? And you don't have good incentives to reduce deforestation. So I think I'll leave it at those two points. Thanks. Ariel, do you have one minute to really come with any gold nuggets? Gold nuggets, okay. You will decide. I think Maria Sanchez, you said we just added more complexity and simple answer. Of course, there is the saying that there are no good answers, only good questions. But I think red, like life in general, is a set of dilemmas. And we have to deal with that and then try to, facing all dilemmas, try to find reasonable compromise. When we have had some about national, jurisdictional and project level, that red was supposed to be both the national and but also projects. And that's where the more the appetite for private sector, as Kevin just said. I think it's also this fact that good policies are contaminating the reference levels or baselines that now it's every six years. And of course, the business as you showed is or should be now with the red policies and you are to reward based on that. Another dilemma is kind of red is rewarding those who have high emissions, whereas the forest stewards who have low deforestation, how can they be rewarded in circumstances? I think these are dilemmas and there is simply no good answer to that. But I think this one referred to that in a previous session, don't let the perfect be the enemy of the good. And that we get it approximately right. We don't need to fine tune it. And I'm increasingly also saying that while we should take a lot of into account on livelihoods and biodiversity, a good rule is that if you have too many objectives and try to kill, sorry for the expression too many birds with one stone, you are likely to underachieve it. So I'm telling from the beginning, being an advocate for keeping it relatively simple and try to get something to work. I know that we need to also look at the socioeconomic benefits, but think more in that's the whole package of development intervention that are to ensure that. And for red, the climate crisis kind of sufficiently grave and serious to deserve a kind of a key focus on that and keep it as simple as possible, but not simpler. That was Einstein quote. Good. It is five o'clock and one minute. It's been a very rich session. And I think we've all, and that includes myself, have learned a lot about this topic and about different aspects of its complexity. Thank you all for assisting and please don't hesitate to contact. I'll just present us for further discussion. Thank you. Hey, hello. Just a final announcement, a practical announcement. Okay, we would like to remind everyone that we have an ice cream session and here in the tree of life area, so close to the atrium. We also would like to remind participants, so everyone is welcome. We will also would like to remind participants to the tour tomorrow, that we are going directly after the exhibition. So please come with comfortable shoes and light bags, not too heavy because we will go together from here. Thank you very much and enjoy the exhibition and the reception.