 Good afternoon. Welcome, everybody, to this side event session, which is entitled, Myths, Realities, and Solutions Towards High Integrity Forest Carbon Credits. My name is Sven Wunder. I'm a principal scientist with the European Forest Institute, and also I am a senior associate of CIFOR, of the Center for International Forestry Research. And let's start with some housekeeping, which has now disappeared from my screen. Sorry. So maybe I can say first that there will be interaction with the public in this session. On the one hand, there's a chat box with reserved for technical issues. And there's a question and answer box, which will allow for questions to be posed to the panelists. Then there is communication regarding the recording of this session. And I quote, we wish to inform you that the event will be recorded to facilitate note taking after the meeting. Data will be destroyed once this purpose is served. Please let us know if you have any questions or concerns. OK, good. So as to our panel, I will briefly introduce Kevin Brown from WCS. We have a online, also, Aded Angelsen and Aaron Sills. And we have Thuy from CIFOR eCraft, also as a commentator. We are trying to, it's an exciting issues. We're trying to span perspectives going from the academic to the more applied and practitioner's perspective. We've seen a remarkable growth of the carbon market over the recent years. And that is, in principle, a good thing. But we've also seen quite a lot of critique, as with the Guardian article in January. And a recent editorial from the Guardian starts off by saying, the emerging carbon offsets market is chaotic and dysfunctional, full stop. So there's a lot to discuss. I hope we're going to have some interesting viewpoints from our panelists and from you here in the audience and people online. And with that, let me start with Aded Angelsen. Aded, could you share your screen and start your presentation? Aded Angelsen is from the Agriculture University in Norway. He has been working on deforestation issues for many years. Yeah, you have to stop sharing before I can share. OK, so good morning or no afternoon. Well, it depends on where you are. So you see my screen, OK, now? Yes, it's not full screen yet. Here we go, that full screen. So good afternoon. Thanks for the invitation. Sorry for not being enrolled, but I saved a few carbon emissions, although the plane would probably have flown anyway. I'm going to give a brief introduction to the title is Records and Carbon Market and just some of the basic issue that hopefully can provide some framework for the discussions and presentations also that we are going to have this afternoon. So originally, when red was conceived COP 15, maybe I'm mixing two dates here on the screen, COP 15, that was 2009, but also two years before in 2007. The main idea was to quote the Bali declaration on red was policy approaches and positive incentives. So positive incentive is UNFCCC speak for result based payment, payment for environmental slash ecosystem services, cash on delivery, whatever you want to call the baby. The core idea was you pay based on verified or certified emission reductions. That's the basic idea. The implementation has been slow and up to now, I would say red has mainly not been about paying for results. It has been more phase one on phase two of red, which is phase one being kind of the preparatory laying the ground, building up the MRV system, phase two being more the first part, policy implementation and then a third phase as payment of results. But we are increasingly doing so. So when I teach a bit on red to students, I start very basic and I thought should do the same. What does it take to create a market, in this case a carbon market? And now that I use marketing in a kind of a broad sense because it is include any type of result based funding, repaying someone for delivering a particular kind of service and emission reductions in our case, whether the money comes from from private investors in a carbon market or from public firms. So here are the four elements that you need. First, you need a commodity or service and emission reduction, which is not a very tangible thing. So you need a couple of things. You need an MRV system for measuring, report and verify. The change in say forest carbon stocks over a period or the emissions. But this has to be measured against the reference level. And these are kind of the two elements of defining the commodity. You need someone to sell it or providers in a past system, which is about carbon rights. It's about benefit sharing. I'm coming back to this in half a minute. You need a set of buyers, basically three sources, compliance market, voluntary market or public funds, for example, development aid. And finally, you need some rules of the game, some institutions, the marketplace and standards. So these, for any type of market or market like mechanism, this is what you need, the four key elements of any type of market. And when we talk about red credit credibility, it's linked to getting all these four elements of a market in place and having credibility on all these, whether it's benefit sharing, how you have defined emission reduction, et cetera. So for the buyers, you have essentially three sources. Originally, when we talked about red in Bali in 2007 in Copenhagen two years later, when we didn't get the Copenhagen protocol or Copenhagen agreement, the original idea was a cap and trade model after the Kyoto protocol where countries take on commitments and you can meet those commitments as you did under the Kyoto and the clean development mechanism by buying credit in other countries or trading among, say among annex one countries as you also could in Kyoto. That was the idea, compliance market where red can be used as an offset. So compliance because there are quotas set caps being set by some authority and national government typically or it could be a state government as California stuff. This is a mixed bank basically because of the reluctance to take on commitments by countries legally binding commissions such that you have to reach a certain and if you don't, you can buy your credits offsetting. So the second is the voluntary market and as Sven alluded to, there's been a remarkable growth in this. See from 2000 to 2001 where this increase is by almost fourfold up to two billion US dollars here. It was a slight dip in last year and 22 for reasons that we all can't think of. It's mainly the companies I would say the big change. The companies are committing to net zero emissions and they realize we cannot do everything on our plant in our upstream and downstream supply chains. We have to get some offsets to do that. It's also very much a project focus because you want to have something that is more tangible and concrete and show your shareholders the general public that this is our project. This is how we have offset it and it need to be concrete rather than saying we bought emission reductions from Peru or Indonesia and said this is our reductions. The third one is the public funding and this has been up to now 50% from ODA or official development aid. You can see the graph here how the share of climate funding, both adaptation and mitigation, as you can see that the mitigation part has increased quite a bit and it's now a third of development aid. This is an increase in concern among many poor countries that climate has kind of grown too big and taken resources away from more directly poverty reducing, although of course a stable climate is also necessary for fighting poverty in the future. Who are the sellers? This is a long debate, just touch on some basic question. Who owns an emission reduction in the forest? Cecilia Luttrell was in C4 and a few others. We wrote an article in 10 years ago among the different rationales for the benefit share in Hueshdaal. It can be those who do their emission reduction taking actions. It can be some based on cost, it can be on the facilitators, those are the legal rights, the stewardship, those that are protecting the forest, indigenous groups for example, or it can be a particular pro-poor benefit that you're using that proceeds from carbon credits to sell. In practice, a compromise between very different consideration, effectiveness, efficiency in particular one side and equity on the other side. Third element is to define this emission reduction. In theory, it's very simple. You have an emission reduction. It's using this IPCC framework, you have an activity, which is the reduction in the area of being deforested, that's the activity and you multiply it by emission factors, carbon per hectare. It's critical because you define the impact, did it work? You define success, the reputation, will you be re-elected because of you have delivered in the environmental area? Or most importantly, it defines the payments in a result-based system. Setting reference level is very tricky. As we'll hear about more, it's something hypothetical because you have to define the counterfactual. What would happen without the policy? We'll never see it and you can never prove that this is wrong because we are defining a scenario that did not happen, namely the development in emissions without red-red intervention. It's strong economic and political interest related to that. It's also some conceptual confusion. I've been through that when I got involved in some reports just after Copenhagen, we had the main interpretation of reference level was a business as usual scenario, kind of a more technical prediction and that's what you use in impact assessment. Increasingly in at least the UN FCCC documents, it has come to refer to something as you may call a crediting baseline, something that is like an emission quota. That's two very different things to what should be the quota, from which point would you like to get paid and a prediction of what will happen in the absence, the normal use of the reference level. Now, in the submissions to the UN FCCC, I'm using in terms of a crediting baseline. My favorite quote about reference level is this, reference level is a benchmark set so low that success is guaranteed. In the early of the red books we published, we kind of made this distinction and some clarification because in COP 15, in Copenhagen, when they had some decisions on how to develop forest reference emission levels or forest reference levels, you take into account historical data and adjust for national circumstances. National circumstances, some negotiations described to me, not as a black box but a can of worms so that you want to open it, but it's so hard that you close it, so it's up to the countries to kind of define what's national circumstances. So that's for the baseline, but then there may be other national circumstances relevant for the more crediting baseline from at which point you should be paid for emission reductions depending on, say, capabilities or climate depth and maybe some other same certainty that you would like to take into account when setting that baseline, but at least make a distinction between the two. And then the final list, the institutional rules. You need some independent verification certification and I often compare it to coffee. Carbon is not coffee. We cannot have emission reductions for breakfast. So the kind of trivial point is that when a buyer gets an emission reduction, it's used as offset. It's a paper that you would like to use in your accounting. You don't have it for breakfast. You don't have a profit from it. You just want to have it approved that somewhere else emissions have been reduced and you can use it in your accounts. Whereas coffee, when I buy coffee, I would like good arabica that tastes well, that is a good dark roast, which I prefer and I have a very strong interest in the quality of that. The same kind of quality control, at least the incentives for that does not exist. Of course, there is a reputation risk. And if you look at the, at least then the UNFCCC process, it says in the Warsaw framework on red to offer a facility of non-intrusive technical exchange of information, which is like leaving to the countries to suggest from what point would you like to get paid with the obvious kind of incentive problems it creates. In the volunteer market, you have taken a similar approach of historical plus some adjustments. And we'll hear more about exactly how exactly that can be done. Again, I should stress that there's no kind of correct formula to be done and you can never read it or you can somewhat test it, but never fully test it whether you got the accurate reference level. So finally, the contribution of science. I think one will hear particularly Aaron will talk about how we can learn from lessons in other fields because setting reference level is actually key in science. We normally presented under other names like impact assessment, but the impact assessment is to define a counterfactual and then look at the present situation compared to that counterfactual established in causality. So it's very much a core and I think one of those involved in this can also learn a lot just from general scientific principles. And the second is this on a good day. I like to think that research and academics are more independent than others. So we have this and I like to finish with this quote a few years old, 1892 from a statistician. Whenever the struggle resurfaces between the champions of the general interest and that of private interest, you'll find us the statistician at our post arm it and ready to march. So thank you. Thank you very much. Kevin, you are next and you have been working much more with the applied side also of all of this. I look forward to your presentation. Great. Thank you, Sven. So I'm Kevin Brown here representing the Wildlife Conservation Society and thank you to C4ECraft for sponsoring me on this trip. I speak on behalf of Wildlife Conservation Society wearing a few different hats. On the one side, we're an NGO, but we do act as a Red Plus project developer in the voluntary carbon market. Excuse me one second. Can I spend my, please have the slide. Thank you. Sorry. No worries. Sorry one moment. Let me see if I can advance the slides here. Okay, I got it. Okay, sorry about that. So we're involved in developing avoided deforestation and reforestation projects around the world. We have 15 years of experience in this. We led the development of two of the most highly regarded avoided deforestation projects in the Kira Project in Madagascar and the Kiyosema Project in Cambodia you may have heard of. We're field led with staff in over 50 countries. And we have staff including myself who have authored existing and forthcoming methodologies in the voluntary carbon market. So I speak from kind of the project developer side from the standards development side. And then I come from a remote sensing GIS background. And I've done enough capacity building assistance to governments on developing their NFMS, their national forest monitoring systems to kind of understand what are some of the practical tradeoffs of how do you ensure that monitoring is done and baselines are set in such a way that you're accurately accounting for the carbon and emission reductions but not placing an undue burden on the process. So a quick note, why does WCS still support the Red Plus mechanism? It's really the only game out there that provides long term sustainable financing for natural forests. People have talked about other things. They've piloted other things at smaller scales, but it's the only mechanism that's been tried and that has a large potential proven pool of financing behind it. So for the near term, it's really the only game in town for us. And it makes the carbon owners participants in the market. They're not recipients of aid. You're actually engaging them and producing a global product and you're empowering them to take ownership of their resource. So that's really important to us. At the outset of this session, Sven mentioned something on the top of everyone's mind. We've been hearing a lot about integrity in Red Plus. We want to ensure that projects are producing credits that have integrity. We've heard integrity come under question from different angles. And I just want to pick that apart a little bit and what do people really mean when they're talking about that? And I can at least speak to how we think of integrity at WCS. So much of the discussion that we've heard of the criticisms has been around the atmospheric integrity. So in particularly, additionality. But we also have to think about leakage and we have to think about permanence. But this is kind of around the atmospheric integrity. Is a carbon offset credit actually doing harm to the atmosphere? But there's other parts of integrity we have to think about. And of course, biodiversity. Are we leaving the planet in a better shape for the next generation than we found it? And equity. How do we ensure that the maximal amount of the benefits of this financial mechanism are going back into the people that actually live and depend on these resources? That it's not just some kind of profit mechanism for outsiders? So I like to think about that there was a comment earlier today in the plenary about the perfect not being the enemy of the good. And that's my mindset here with integrity and red plus. We hear a lot about atmospheric integrity. You hear a lot about having equitable outcomes. And these are absolutely critical. But if we have red plus that has atmospheric integrity and equitable outcomes but is not a financially viable mechanism because it's too hard to implement the requirements are impossible for anybody to meet, then it's never going to scale and it's never going to really reach the potential to actually serve as a tool to address the climate crisis. So having atmospheric integrity and equitable outcomes without financial viability is a global failure. I just want to put that provocation out there. And I want to delve a little bit into putting on my project developer hat and talking to the private sector actors that really want to get involved in this space. They really want to get in and develop projects and make use of those offsets but some of the challenges that they face. I was interested to hear, I'll already say earlier today earlier in the session that ODA funding is actually increasing for climate mitigation. I'd love to follow up on that. But I think what we found in our space that even if maybe it is increasing it's still far insufficient to meet the demand needed for the financial the amount of money that's required to set up a red project is easily millions of dollars. And oftentimes these investments don't produce any emission reductions that result in like a net profit. You can't repay that investment for sometimes five, ten or more years. So the private sector is looking at this like an investment. They would need to sink a lot of money into one of these projects. There needs to be some kind of assurances from their side that if they perform to a certain standard that's known ahead of time that's outlined in a carbon offsetting methodology that if they perform to those standards that those credits will be issued and that they can sell them. And there's so much uncertainty in this space that they already have to cope with. And when I say investors I really mean what I mean is anyone who has an economic stake in the project so it's not just those that put in the financial capital but it's the carbon owners too who are paying in the opportunity cost of not using that forest for something else. Because that's often a quite substantial cost. You're asking them not to get some other kind of economic benefit from that from the use of that forest. So there's so much uncertainty already. You have to ensure that your project actually performs against the baseline. You have to deal with the volatility in the price of carbon credits. You have to deal with changing regulatory environment around carbon offsetting. All of these things can be mitigated to a certain degree but the one that's new that's been real challenge is with these discussions about high integrity and these ex post assessments of additionality are the risk that after you've gone through all of the investment and all of the work and issued these credits after how many years that you then find out that well actually the global scientific community actually doesn't think that that carbon credit is a real thing so you didn't actually produce anything. So that strikes fear into the heart of the whole community and it really has put a chilling effect on some of these actors who might be otherwise wanting to get more involved. So this event where mostly come from a forest monitoring background and thinking about modeling, forest change. So I want to kind of tie together why are we talking about financial viability of projects to do the evolutions in the voluntary carbon market standards because the decisions that we make and how these standards are set up have implications for financial viability and for atmospheric integrity. So it's a really important balance to get right. I want to frame for a quick second how the voluntary carbon speaking here just about the avoided deforestation methodologies were typically set up over the past 10 years. It uses a reference region approach. You identify a proxy for your project somewhere else in the country. You develop a historical deforestation rate from that and then you apply that to your project area and assume that that's your future rate of loss. And then of course that's validated against using a third party audit. And for a quick second why was this approach set up this way? Well there were practical reasons. It wasn't because Vera or anyone wanted to allow project developers to game anything. There were practical reasons. There was no Google Earth Engine 15 years ago. There was no University of Maryland deforestation data set. People were still classifying a single Landsat image at a time. So the idea that a project developer could go and assess an entire baseline drawing on information from an entire jurisdiction, it was just unrealistic. So hence why the reference region approach was developed. Moving to what I'm calling red 2.0 which is not an official term or anything is that the way that Vera has been moving in a new avoided deforestation methodology that's gone out for public comment a few times that's going under final validation as we speak is that now all projects are going to get a baseline that their activity data, their deforestation rate allocated from a jurisdictional pool so that no projects can be over allocated such that it adds up to more than the reference level for the country. There's no more use of trends, only historical averages that will reduce volatility in the baselines. And then the baseline projection period has been shortened to six years so it allows more frequent updating of the baseline to follow trends, changes in forest trends. And some of the big advantages to this methodology are one, it dramatically reduces the amount of resources that have to be spent on analysis for projects to get up and running. It allows project developers and countries to understand where are the high potential places in their country to invest in. It's just known ahead of time. They don't have to go through a bunch of feasibility studies. And most importantly, by developing these data sets at a jurisdiction, it allows the voluntary carbon market to leverage existing capacities in government MRV and NFMS offices, bring them into the process and start to bring revenue into those offices and get them prepared for jurisdictional red, which is the next phase. I know I'm getting over time so I just want to touch on two really important areas, I think, that we're still grappling at how to address in the best way possible. A huge issue is the risk map. The way that these methodology works is that baseline is allocated to projects using a jurisdictional risk map. There's been a lot of work that's gone into creating some metrics for how to identify which is the best risk map of all the different options. Total operating characteristic and area under curve. But it's really an area that I think needs more discussion and bringing in more experts. So I would definitely advise those with a background in that to contribute. And then secondly, I won't go through all of these points, but leakage is a big area of uncertainty. There are several different types of leakage risk that are accounted for under the Vera methodologies and all of them to some degree or another are based on an understanding of human behavior. And I can say that these, how leakage is assessed and how it's monitored has not received nearly as much attention from the MRV community as has some of the forest carbon sides. So it's an area of I think that has huge implications for additionality, but hasn't received as much attention. Okay. Well, I think I'm over time, so we'll just stand there. Thank you very much. Thank you. Kevin, so we are shifting back online now to Aaron Sills from North Carolina State University. Aaron, please go ahead. Great. Thank you. Can you hear me? Yes. And now let's see if you'll be able to see my screen. We can see the file just need a presentation mode, if possible. I have it in presentation mode now. Is that what you see? Yes, now. Yeah. Perfect. Okay. Well, good afternoon in Rome to everyone. I'm going to continue on the theme of the first two speakers. One is comparing the methods that we use for MRV or for accounting for carbon credits to the methods that we use for evaluating the impact of red plus interventions. And so just to run through this comparison again, I think we're all reinforcing the same point here for both of these, you need the MRV piece. So you need to identify where the intervention is being implemented, the red plus area. And you need to monitor forest change in that area so that you can estimate the emissions associated with any deforestation or forest degradation. That gives us a measure of what happens with red. So the difference comes in when we start comparing that measure to something else. In the first case, we're going to compare it to an X anti forecast of what forest change was expected under business as usual. That is what sets up what Arl called the crediting baseline. In contrast for impact evaluation, we're going to compare the same measure to X post observations, contemporaneous observations of forest change in comparable areas that are also under business as usual. So there's no red projects there and that gives us our counterfactual. So two different types of comparisons for really two fundamentally different purposes. So the first X anti forecast establishes essentially a performance benchmark. So Arl suggested maybe a low performance benchmark, but nevertheless it's a benchmark and carbon credits are generated when either forest loss or emissions fall below that benchmark or what's typically called a reference level. Switching to impact evaluation, we're going to take X post observations that establish what would have happened in the context of other ongoing policy and economic changes. So this is what the world would have looked like given all the other changes going on but without a red project. That provides an estimate of the impact of different types of red interventions, whether and how much impact they have on reducing forest loss under what conditions. And serves as really important feedback to both investors, potential investors in red and to policymakers. Okay, so this sets up a challenge moving on to the second part. So that's the comparison. The challenge is that conceptually this X anti forecast and X post observation that we have in the two different approaches are trying to get it the same thing. They're trying to get it a prediction of what would have happened without red. But in practice, they're really not interchangeable. So this X anti forecast can't substitute in to causal impact evaluation because fundamentally the forecaster is never going to know all of the other changing political and economic and social conditions that affect outcomes under the counterfactual. So that doesn't work. Likewise, the X post observation so observations of forest change forest loss and comparison areas isn't a good substitute for our accounting or especially our contracting system. Because when you set up the, essentially the contracts to tell someone you will pay them if they deliver reduced forest loss and reduce emissions, you don't, you don't know this X post outcome, right. And further, the people who are involved in the contracting don't have control over it. They don't have property rights over the control areas or the comparison areas. So again, that doesn't make sense as part of payment upon results type contract. Okay, so we've got two different systems one to generate carbon credits and the other to generate feedback. So come back to in the end, we do want to use that feedback to improve our system for carbon credits. But that's different than compounding the two right so the two are not exactly the same. And just to continue to hammer this home we can look at this graphically which is what I'll do next if we take sort of the simplest version of the accounting approach or the MRV approach, which is to say, let's look at what forest change was going on what was the deforestation rate before a red project came in in the intervention area. And then we're going to compare that to what happens after our intervention. Right. And if deforestation deforestation rate or rate of forest degradation goes down, then you get credit for that. Right. And so reference levels are actually more complex than this but you could think of this as really the most simplest approach to in order to do an impact evaluation of whether our red project actually had an impact and a causal impact on the rate of deforestation. We need to bring in comparison areas. And then we're going to compare the rate of change in deforestation or degradation in the intervention area and the comparison area. And that difference is going to give us our estimate of the causal impact of the red intervention in what's called the Baki approach so before after control intervention. So to illustrate this I'm going to draw on Sephora's global comparative study on red, which was set up using the Baki approach. So we've looked at these 23 different sites where red projects were being implemented across the global south. And in most of these sites we selected villages both inside the red area and outside the red area selected so that they had similar underlying drivers of forest change that is they were pre matched. So we collected really loads of data from before the red project started in roughly 2010, and then after the red project started so we first went back in 2013. For today though I'm just going to focus on remote sensing information about forest cover in these areas. So we did use the Hansen global forest change data. What I'm showing you here is taking the before after approach, we looked at all of 23 initiatives, and we calculated for each of them what was the average annual deforestation before the project and the average annual deforestation after the project. So if the average annual deforestation went up, then we say that's a poor outcome, and this before after comparison. If it went down more than point one, because the range we're using here, we say it was a good outcome. And then it was about zero neutral. So that's that simple sort of before after approach you set a reference level based on what was happening before and then you look at what happens. So in terms nine projects had increasing deforestation. But of course you should immediately be thinking maybe that's because the background level of deforestation was increasing. And so you actually want to compare these project sites to to the rest of the landscape. And that's what we do here so we compare the deforestation the project sites to deforestation and the jurisdiction that includes the for those red sites. And we find that fewer of them increased deforestation relative to the background increase in deforestation in that jurisdiction. And then next we move down to look at the specific villages we selected again we selected villages inside and outside most of these sites to find comparable villages. And what's most notable here is when we compare villages that were matched that have similar similar underlying drivers of deforestation. We find that a much smaller fraction of these sites look like they have poor performance and a much larger fraction look like they have good performance. So the conclusion is that careful selection of control areas in this case is actually revealing that these red projects had an impact, they reduce deforestation relative to what happened in the matched villages. And just one more illustration here and then I will wrap up. So that suggests that the controls are really critical to select carefully. So again, what I did in that example or what we did in that example was we picked matching villages. But another approach that was referenced in the Guardian coverage that's been talked about is to look for a weighted combination of potential controls that are both matched in terms of the underlying drivers of forest change, and that are following a similar trajectory of forest change historically. Okay, so that's an alternative way to find a control set or a comparison set. We applied that to 12 red projects in the Brazilian Amazon. This is work led by Tali's West and published in PNAS in 2017. So in this case in these 12 projects. In red here this shows the cumulative deforestation in hectares from 2000 to 2018. So shows cumulative deforestation going up as you might expect in most of these 12 projects. And then in blue, what we're doing, let's look at the top left here. Before the projects implemented, we're looking for a combination, a synthetic combination of other sites in the Brazilian Amazon that follow the same path of deforestation. So what actually happens to deforestation in that synthetic control after the project was implemented, and this top left case we see that in red, the synthetic, or in blue sorry the synthetic control had higher deforestation than the red project in red. So this is what you would hope if you were running a red project is that your deforestation is lower than the deforestation in the control. So with this method we can actually test for statistical significance. And what we find in this study is that in only one of these sites is there a statistically significant impact for a station. So only one site manages to reduce deforestation in a statistically significant way. All of these projects however claiming credits for reducing deforestation and reducing forest carbon emissions. So again I want to come back to the main point here how is that possible. Well it's possible because for evaluating the impact, we're looking at what happened in comparison areas after the projects were implemented we're tracking those outcomes over time. Whereas what the projects are doing when they set up is they're looking at the information they have in hand at that point. That's forest change in their project site and in comparison areas and they're using that to build a forecast. So what we what we need to remember to do is sort of to use the information from the impact evaluation to help improve the system for forecasting the counterfactual. So back to the theme of not letting the perfect be the enemy of the good. We don't need to expect or we shouldn't expect that these are going to be identical because they're based on fundamentally different information. But we should fully expect that we're going to use the information from impact evaluation to design better accounting systems that maximize the incentives to maximize reductions in deforestation. Thank you. Thank you Aaron. So we're turning back to Glennery and I have can give you an our presentation a short presentation on a meta study of of how red plus project have in fact what impacts they have they have created. And we start out from a theory of change of what we're plus is trying to achieve with some some funding flows from bilaterals and privates etc. And some information of a about if forest stocks and their development in the past know we we typically have a mix of mesh of measures, you know some incentives some disincentives and some enabling measures that are treatments. And then we want to see some some outputs we want to see that that people actually actually understand that intervention that they participate to certain extent on the ground. And this then should have, you know, some reach some outcomes in terms of reduced deforestation on the ground and perhaps also some improved incomes. And in terms of impact, obviously, our end goal is to to reach sort of mitigated global greenhouse gas emissions and some side objectives regarding biodiversity and human well being etc. But often our impact relations will actually be at the at the outcome stage, in terms of measuring land use impacts. So in this matter study, we have to somehow to limit what we what we want to take on board in terms of the actions that are taken that is number one to four year. And in terms of the publications and that we that we want to to to include. And in terms of the actions, for instance, you're not going to go through every detail here but we excluded pure effort station reforestation projects. We did include as a several category, a national past schemes that had a carbon that also had a carbon focus. And in terms of literature, you know, we are a we are including a we are cloning both gray and peer reviewed literature. But we want to focus on a on the end part of the theory of change that I've just shown you on on outcomes and on impacts. We're taking a rigorous approach in terms of counterfactual approaches that are being used in quasi experimental methods. Okay, so what do we get in terms of global coverage? This shows like the density of red projects. It's the the areas enrolled in red plus projects over the forest area, which is interesting in the sense that you actually have quite a lot of red projects. Relatively speaking in, for instance, in part of East Africa and, and then the Indian countries compared to Brazil, which has absolutely many red projects but but actually not so much in compared to the huge forest areas that they that the country holds. And our sample is quite well, that's the red part, red colored part, which is very well distributed across a across the the tropics on with some areas Congo Congo basin for instance is not well represented also some South Asian red projects, etc. So what are our results? We split that up into the sort of the red projects and programs, and then these public PS programs that have also a carbon focus. You can see for both of them. We find a significant impacts. But they're quite small and they're quite variable across across different projects. But there is some, at least some causal effects to to to call for. What about the permanence part that's a particular interest for forest carbons, you know, and in principle, you could imagine if you look at the diagram in the lower part here, you could imagine that after a project has stopped, which is the second vertical line. You could either it has managed in the red part to reduce deforestation, it could either continued on that reduced part like as one that would be great. It could continue on the old path that is as to it could also continue towards the old part but in the pickup really eat up that forest conservation gain that you would have reached during the project. Or you could at the extreme have a really accelerated worse than before kind of scenario of of accelerated deforestation. We have few of those assessments after project available, but what we can see is that is kind of the the as two parties is is the one that is seems to be dominating you get a parallel trend, but you have you keep kind of the the transitory gains you have made during project implementation, you were able to keep those. And that's the welfare part. And what we find is, you know, even smaller positive outcomes in terms of impact sizes. Typically, people don't get worse off in the terms of incomes and assets and consumption. That's the outcome stage, but at the impact stage which would be the self perception of how people their subjective well being. I do get some negative results and I think it's both a result of in some cases equity outcomes, you may not be worse off but you feel worse off because some other people got better off in your village for instance, and also some expectations created around the implementation of these projects that are not always satisfied. For instance, when projects are not being able to be continued. We also looked for at, you know, what makes the difference in terms of within sample variation in terms of what what what factors seem to be boosting effect sizes. And clearly, if there is a real different stations that's set to start with in the place where the project is is being is being undertaken. That helps in terms of that's quite intuitive that if you really fight against something then you also more likely to to get a to get a result. Another important is spatial targeting towards high forest carbon density, high threat within your within your site, for instance, road near areas, etc. Those that did spatial targeting also achieved better results. And on the welfare side, perhaps a bit surprisingly. There are benefits are being differentiated that which is often used to squeeze rents, etc. Then, then is sort of the other the welfare effect is larger. And that's maybe because you were able to customize them to different people's needs. Those benefits generated. So how well does red plus do compared to other kind of conservation tools that are out there. It seems to be that this is the kind of the just the coins D effect, you know the higher it is the more, the more impact you would have. And you can see compared to other enabling measures or a disincentives like command and control or other incentives. Then no, not significant differences really, there's a lot of variation within these categories, but at least it does a as well, at least as well, or as poorly as as other conservation tools. And this is the case that we that many of the conservation things we do to not really sufficiently large scale change behavior on the ground. We still have things to learn about about this. Of course, this does not analyze cost effectiveness. Some of these projects are expensive in terms of having like a boutique like character that shows off in various ways. So this is not integrated in the assessment. And then I wanted to just briefly come back to to also the discussion that all the previous speakers have taken up a about the additionality about baselines and about a about the impact elation. So these are similar figures. Illustrations as those shows shown by Aaron, but for a tireless is West newest research with the approach of synthetic control is being extended to to a global sample. And this has also been used as background material for the for the Guardian article in January. And obviously, I don't want to go into into details, but you have here, you have here both the defect to observe deforestation in red. You have some deforestation synthetic control sites that in blue, and you have the extended deforestation baselines that the projects have been using. And what you can see even at a quick glance is that a you see project starts with the vertical lines, and you can see a lot of these projects have a yellow lines that compared to to the past deforestation move up like hockey sticks right. So they assume that after the project starts that you will get a really large pickup in deforestation. And that's that's problematic, of course, when you get that at such a disseminated scale of, of the of different different projects. So, my point here is that, you know, those sort of critical results we have received like in in this study that's more than 90% of the carbon credits were non performing. They are only in part due to the to the relatively modest impacts on on forests on the ground that these projects have had. They are obviously to to a larger scale due to to some of the assumptions that initially have been made about about the baselines adopted by the projects. So, five quick takeaway messages first a couple of of contextual ones in a I think we look at the performance of red plus a lot of these things have remained under finance they did not have a sufficient security about a future a future resources being available. And a lot of the projects have really been a drop in the sea compared to the overall target of mitigation. One of the things that they have done on the ground is it's it's really a mixed back. It's in a way a red plus has been to carbon what what integrated conservation development projects has been to biodiversity, sort of an umbrella for quite heterogeneous of on the ground interventions. As to the impacts that we do find in the study, you know, they are typically statistically significant. They are modestly in size. They are only permanent in the sense of the transitory gains made that I mentioned. And in terms of the the record is just as good or bad as so many other conservation tools. We believe that red plus could have more of an of of a forest conservation impact. If these interventions were more spatially targeted in the sense of choosing to start with areas where the deforestation is high, rather than low. And also then within your site, giving priority to those areas that are really in the most threatened to deforestation, make sure that those are enrolled. So this may sound intuitive. I see people nodding in the audience, but it's not exactly what characterizes those projects that we are that we are looking at, not for a lot of them at least are. And I think for for the integrity of the of carbon credits. I also think we really need to look into to how the baselines are being constructed in the first place and and Kevin mentioned now there's a reform process going on where we're trying to sort of also learn from these impact evaluations that science has brought forward and perhaps find some some methods that are that are better equipped. Thank you very much. So we now turn to the discussion part we have a short intervention from to a as a commentator. Please. Thank you so much. Thanks over the presenter for a very impressive presentation. I just wanted to pick up one of the things which is both arrow and also Kevin mentioned in their presentation, which is looking at the social integrity aspect. And the fact that we need to have an equitable sharing of benefits as well as the involvement of local communities to ensure the equities and the Venice are taken into account. I think that's based on that few things that are the results from our 14 years of research under C4 global comparative study on red plus looking at both national policies on price carbon market as well as looking at the project level. I think that why we see a lot of improvement and a lot of discussion on how to advance the methodology to evaluating the carbon credit. The progress in terms of non carbon benefit has been much slower compared to the other aspect. We have seen in many countries some of the key enabling condition that I will put into the presentation, such as institutional framework to clarify who's on the carbon rise and who should be the beneficiary to receive the payment. I think in pending and many countries after 15 years of implementing red plus are still in the process of designing beneficial mechanism and the countries have a lot of difficulties in this. The other thing is when we talk about the cost and benefit in terms of fairness. I think that like we have her from Kevin in terms of a lot of improvement and advancement in terms of methodology to reduce the cost to analyze the data and collect the data. Our global comparative study on red plus in 2019 now where we documenting the administrative cost from the government both national and subnational government. Most of the time the subnational government as well as indigenous local community have subsidized many costs of carbon project. These costs are not taken into account not recorded and causing quite a challenges in term of discussion in term of a contract arrangement between government agency or with the indigenous people and the private sector. So this cost has to be accounted and taken into account. The other thing is Sven mentioned earlier in his slides about some of the project are very expensive. But in many cases the cost to do a proper free prior informant consent involving with the local communities require a significant financial resources. And unfortunately if you're looking at many national red plus policies and the vice carbon project based on our global database. So given to gender mainstreaming or to implementing social safeguard receive a very modest amount compared to other thing which brought our attention. If you wanted to commit it to social safeguard and also equitable outcome. You also need to provide sufficient financial resources to it. The other thing that we wanted to also highlight in our research is perhaps with the forest carbon market. We always think that it could be one of the policy instrument leading to transformation of chain. But in fact we also need to have transformation of chains in many things to enable forest carbon market particularly with the land tenure system with the social safeguard which has seen. So I think that under develop compared to the other advancement when other presenter talk about the methodology. Yeah. So just a quick comment from the equity perspective. Thanks Ben. Sorry. Thank you to excellent. So please think about any questions here in the public or put them into the Q&A box. I'm giving the word briefly to you briefly to add it for also equally brief comment. Are you there. I am. Okay, thanks. Very nice for the three interventions and just a couple of points I would like to, I think Kevin, you mentioned this that you have a situation where project developer has a good project as delivered and then comes Sven and colleagues and publish an article telling well, maybe there were no emission reductions after all and use inflated baselines and so first they have to say that this is not new. I remember the first no camp project in Bolivia where Greenpeace 2009 I think it was came out and very strongly criticized the TNC for having exaggerated the impact which was, I guess the first offsetting project for American utilities. So it has been a debate, but I do see the problem and there are a lot of risks and one is to find, yeah, you see that it doesn't hold. I think that the thing that Erin, when you compare the exposed kind of that you normally do an impact assessment versus the X Santa before the project that that's something we can learn. And it was also in 2009 I had to check the article was speaking that comes from tell another which had this successful compensated successful efforts. So there is, you set a reference level, but you may say have a formula where you adjusted when you see what happened. One example is this what happened these days it's a much higher commodity prices that that may put a pressure on forest. So if you have say the reference level, the business as usual scenario depends on how commodity prices that would actually reduce the risk of an investor because if you have a corona situation with high commodity prices putting a pressure on the reference level, you're, you're more like, sorry, you're less likely to achieve a certain emission reduction based on like predetermined on X and reference level. If you could adjust and say that okay they were increased pressure on forests. So therefore, we, we adjust upwards the reference level so you compensate for the effort, rather than the actual reductions when we have the information afterwards. There should be a risk reducing thing and and should be welcomed by project proponents. I think the problem is, don't make it too complicated keep it sensible, sensible simple kiss, the kiss principle also applies here. Yeah, it would be interesting to hear from Kevin also whether you know, you know you're not with Vera anymore but but that and private investor whether they would think of some simple formula for some exposed adjustment of the baseline that actually will reduce the risk for for a project developer. Also great inputs. I will now turn it to the public because we have been talking a lot of ourselves. So we want some some inputs from the rest of you, and I will take those questions that are now and then the panel can can react. Thank you to the lady in red over there to start with. Oh, very awesome. Great. And I've got in also yes. Thank you. Thank you for the really impressive presentation. We have a lot of questions about the quality of red and the impact of the red implementation of red plus project. But what I feel is that perhaps we feel a little, a little piece on the context because from Bali and Copenhagen. The context has totally changed because we we had, we have had the Paris agreement, but more than this we have the national transparency framework that set clear requirements for all parties. So all parties now have their national determined contributions so climate target in terms of emission reduction, the most of them, and all parties have to submit at the latest by next year. Okay, we are there have to submit greenhouse gas inventory with categories. So no more red. We have Lulu CF categories of forest land or land converted to forest land and whatever. More than this, we have a space for article six so for market based mechanism. And so for project to be run at local level or national level or sub national or whatever, but each crediting activities should be reconducted a reconciled a national level with with a tabular format to be submitted by each country parties every two years. And this could be taken to account because it's really nice. This kind of reflection because we learn a lot from the red plus implementation, but we have to change in my in my view, we have to change our idea in order to adapt to the new context. Thank you. Yes. Thank you so much. Thanks for the presentations and the discussion and my comment are sort of related to the lady to my right here and it's really going back to so we have heard sort of one version around red plus which is very much about establishing separate isolated projects where there is an investor that has a need for a return on that investment. And we have heard and learned a lot about the problematic sides of that with actually establishing an effect with the accounting and so on. But sort of going back to the original thinking around red the way we perceived it and I'm sorry I forgot to say I'm with Norway's international climate and forest initiative. We have been working on this for 15 years and our starting point has always been that you need to approach this at the national level. So a national level approach to red plus means having a national reference level. It is linked to the policy implementation which is necessary for the transformation that we are really seeking to make happen through the financial incentives. Of course on the ground those the actual overall outcome in terms of reduced deforestation will be the result of a lot of different projects but having them all being part of one national strategy under the same reference level is extremely important to avoid the issues around leakage that have been described and that we all know to give the basis for benefit sharing. And so on and within that context the red plus is much more an instrument or a mechanism to support forest countries towards low emission development to changing the trajectory of economies more dependent on deforestation than on forest protection and sustainable forest management. And I miss that perspective and I think just as was said by my friend over there linking red plus to the overall nationally determined contribution as one area where you can have international support because the race system established in place is also an effective and easier way. I should say of of implementing red plus. I would. Last my last point is regarding the issue of reference levels and additionality and it's just a very simple approach in saying that historical emission levels are well established in a straightforward approach to how you start off with the unambitioned level for a climate target and and so it's really treating the first sector the way we treat other sectors. It would be interesting to hear reflections around this discrepancy between the national level red which we've been working on and we see now is really picking up with some very impressive results from Indonesia and several other countries. Compared to what has been tested and where we've got important insights from but which really enters into a lot of more difficult issues. Thank you. Thank you. I think I don't know who was first actually. Maybe since you were closer by and then and then yes. Thank you. We have a question from the M. R. V. Unit of Mozambique. So first I think the issue with these some of these articles criticizing the results of red projects. In my opinion has a lot to do with the distinction between avoided deforestation and reduced deforestation. I think a lot of these projects that go into an area that doesn't have deforestation and predict deforestation are a lot more prone to these biases than a project that aims to reduce deforestation over a large area. And I think scale has a big relation to it, because we are, I think, transitioning to a phase where we go from lots of small projects to more jurisdictional projects. And that's the approach that we've had in Mozambique. And so I think with regard to the presenter, Erin Sill, that talked about the control sites, I was wondering how she sees this working when countries are going towards jurisdictional programs where you may have the entire country covered by jurisdictional red programs, and you then may have difficulty finding these control sites. Because as it stands now, if you have a very small project area, then it's easy to find a control site. But imagine if every province of a country or large areas are covered by red projects, then you may have difficulty finding these sites. So yeah, thank you. Thank you. You here, and then I have one online question as well, and then we're going to respond. Yeah. Thank you very much for the presentations. I have to say that I ended up a little bit confused. I was trying to find a common denominator, like a wrap-up message to the talks. And I have to say that I got mixed messages. On the one hand, I saw a series of advocacy towards how projects have been working and how they have or have not been delivered. But then, for example, from Sven's and Erin's presentation, I see reference to OK, to compare the performance of projects when it's synthetic comparisons. I imagine that it's by statistical matching and these type of things that include a bunch of parameters. We have used those to assess, for example, the impact of protect areas and things like that. But it entails that there's an underlying message that the way the reference levels for carbon products has been established up until now is probably not the right one. OK, so I'm making sure that I understood that correctly. So that's a good one. The other thing is that there's a lot of things to connect here. What was mentioned about jurisdiction, I think, there was an elephant in the room that there was no mention during the talks about this element of nesting and how carbon products fit in overarching national or subnational strategies. I definitely think that projects should be part of the implementation strategy over broader strategy, basically over a broader area of intervention. But these also, as Sven was pointing out, need to go and happen in the places where there actually is forestation happening. Otherwise, but this brings another element in the elephant in the room that perhaps touches what Caving was advocating for, which is the fact that we're missing the conservation of carbon stocks in the equation of the products and the recognition for those. And this is perhaps something that needs to be brought up because, and that's why we see a lot of conservation projects being labeled as avoided deforestation projects. And that's why we have these issues with the baselines. But also because we need to prepare ourselves for when tackling deforestation is successful, because then we will move into, OK, we stopped it. But now we need to maintain these results. And then there needs to be some element of a maintenance fee or an insurance or something that will sustain these in the future. So I didn't see that brought up here. And I think that it's something that I would invite you guys to explore, particularly because while our conservation society is also advocating for this type of mechanism and things like that. And you can see Kevin mentioning that here. Yeah. And then the final one. The other elephant in the room is the fact that we need to pay for implementation. That's clear. Implementation is costly. And then, of course, it's easier to implement where nothing is happening. Although it's expensive, it is much more expensive to tackle deforestation where it's actually happening. So that becomes a little bit of a perverse incentive, right? So yeah, some ideas there. Thanks. Thank you. Were there any more questions in the room? OK, last one. Yeah, more than a question. It is a common. I think there have been a very good point here. There has been a change in paradigm since the Paris Agreement, because now every single country has commitments. And that was not the case when red was designed. And to fulfill those commitments, countries have to increase their ambition over time every five years. And regarding that, they may look this sector, because it's very attractive for themselves. So I think it's a change in paradigm. We don't need to think that projects or results-based payments for red coming from outside of the countries is going to be the main avenue for increasing ambition on the sector. The second comment is, when red was defined, the idea was national. And we had long discussions about that, and it was pointed out behind me as well. And we are only been talking about incentives here. But red plus is about policy and incentives. And for example, when you deal with your reference areas for the counterfactual, if you have managed to establish a policy enabling conditions, these may affect also these counterfactual areas, particularly because these policy approaches are national, mostly. So I think we have to reflect on what the discussion was at the beginning of red plus, which was incentives are not only monitoring incentives, and policy approaches are equally important, and the fact that the Paris Agreement also changed the setting of the rules, which doesn't mean that projects may not be useful. But maybe we have to rethink about the objective of projects. I think that's what lesson-learn is. And despite of the very interesting presentations I learned a lot today, what it led us to go is towards more complexity. So I wonder if this won't deviate the attention to what needs to be put with this in changing and transforming the sector. So that's, I think, my reflection. Good. Briefly, an online comment. Yes, question from Askari, Indonesia. Indonesia have done restoration on peatland hydrological functions through construction of rewetting infrastructure, such as canal blocking, canal backfilling to decrease the peat, the composition. Therefore, it reduced the CO2 emission. The restore area about 3.7 million hectares in concession or company area, and about 1 million in community area. Can this achievement be counted or claimed in red? And how to get the funding or claim for this achievement? Good. Thank you. We have a large pool of questions and very little time to respond. I will just start out by saying very briefly that the results I represented in the meta-study did, I mean, we talked here, national versus project level. So they did actually include some jurisdictional approaches and also some country programs, including the NICFI interventions in Guyana and Indonesia were impact-related, and in both cases, found a modest but positively significant impact. The thing is that not a lot of these things is that the research will always be backward-looking, because it takes some time to analyze the data. What really worries me for the future is that I don't see the jurisdictional approaches as they're being rolled out now in practice, being accompanied by efforts to do serious impact evaluation, because often the donors would have to ask for that these things are being put in place so that we in five, six years can actually get some results of this type for the jurisdictional approaches. That is not happening today. I turn it over to the others. Great, thanks so many great comments. I think I agree with all of them, but I just want to touch on two things I heard that I think are super important, and I apologize for it not coming out more in my presentation, that I completely reject a dichotomy between projects and programs. I think what's happening is that you're starting to see them merging together. We're always gonna need location and site-specific interventions to achieve results at a jurisdictional scale. You still need actors who are working in particular places, working with particular investors to actually get those results that you can then package in a jurisdiction. So I think what's so exciting about the new Vera methodology is that it does actually link together now the project accounting with jurisdictional accounting. So if you can get that up and running, you can start to generate revenue to finance the jurisdictional program and then help to ease that transition to a fully jurisdictional standard in the future. And then secondly, yeah, the question about, red was supposed to be jurisdictional. I completely agree with that. But red is a mechanism that targets developing countries and I think it's not totally appreciated. Two things, one, how much money it will cost to actually reduce deforestation at a jurisdictional scale? Billions of dollars, billions of dollars and countries don't have those types of resources and the private sector doesn't wanna work through national governments usually. It's just a hard truth, but they wanna work through site-based interventions because they feel like they have more control over the process. So I think I'll leave it at that. Thank you. Erin, do you have any comments on what has been said? I wanna give everybody the chance to react. Sure, so first I agree all great comments and raising important points. I agree that certainly there is a sort of an institutional drive and probably a need to shift from the project to the jurisdictional approach or to merge, as Kevin said, to my mind that doesn't eliminate any of the challenges we talked about, it just moves them to a different level, right? As Ben said, we still need impact evaluation at the jurisdictional level. We still need to think about how to set reference levels that provide incentives to generate impact. And even more challenging, we need to think about as we transition there, how do we attribute any reductions in deforestation to site-specific projects versus jurisdictions? So I think you can translate everything we talked about to the jurisdictional level. Second point I wanted to make is in response to the question clarifying whether we are in fact saying that our impact evaluation shows that the way reference levels has been set doesn't accurately demonstrate impact or measure impact. And I would say that is true. Our impact evaluations have shown that reference levels were inflated measures of what would have happened in terms of deforestation without red interventions. My point is that that doesn't necessarily mean that we need to substitute or that we can substitute or setting crediting baselines with quasi-experimental counterfactuals. What it tells us is that we need to think carefully about how to set the parameters around methodologies for setting those reference levels so that they are set in a way that really incentivizes maximum impact in terms of reducing deforestation. The problem with inflated reference levels is then it sets the bar to blow, right? And you don't have good incentives to reduce deforestation. So I think I'll leave it at those two points. Thanks. Ariel, do you have one minute to really come with any gold nuggets? Gold nuggets. OK. You will decide. I think, Maria Sanchez, you said that we just added more complexity and simple answer. Of course, there is the saying that there are no good answers, only good questions. But I think red, like life in general, is a set of dilemmas. And we have to deal with that and then try to, facing all dilemmas, try to find reasonable compromise. I mean, we have had some about national, jurisdictional, and project level. That red was supposed to be both the national and but also projects. And that's where the more the appetite for private sectors Kevin just said. I think it's also this fact that good policies are contaminating the reference levels or baselines that now it's every six years. And of course, the business as usual is or should be now with the red policies. And you are to reward based on that. Another dilemma is kind of red is rewarding those who have high emissions. Whereas the first year to have low deforestation, how can they be rewarding circumstances? I think these are dilemmas. And there is simply no good answer to that. But I think this one referred to that in a previous session, don't let the perfect be the enemy of the good. And that we get it approximately right. We don't need to fine tune it. And I'm increasingly also saying that while we should take a lot of into account on livelihoods and biodiversity, a good rule is that if you have too many objectives and try to kill, sorry for the expression, too many birds with one stone, you are likely to underachieve it. So I'm from the beginning been an advocate for keeping it relatively simple and try to get something to work. I know that we need to also look at the socioeconomic benefits. But think more and that's the whole package of development intervention that are to ensure that. And for red, the climate crisis kind of sufficiently grave and serious to deserve a kind of a key focus on that. And keep it as simple as possible, but not simpler. That was Einstein quote. Good. It is five o'clock and one minute. It's been a very rich session. And I think we've all and that includes myself have learned a lot about this topic and about different aspects of its complexity. Thank you all for assisting. And please don't hesitate to contact us as presenters for further discussion. Thank you. Hello. Just a final announcement, a practical announcement. Okay, we would like to remind everyone that we have a nice cream session and here in the in a tree of life area, so close to the atrium. We also would like to remind participants. So everyone is welcome. We will also would like to remind participants to the tour tomorrow that we are going directly after the exhibition. So we please come with comfortable shoes and light bags, not too heavy because we will go together from here. Thank you very much and enjoy the exhibition and the reception. Thank you.