 So, hello and welcome to today's TEC, SPCA, European webinar. My name is Henrik Andersson. I'm an associate professor here at the University of economics and the current president of the Society for Ben Kelsen. So, I will provide a short introduction to the SPCA and the background for this webinar that we are going to host today before I give the floor to today's moderator and the speakers. So, the Society for Ben Kelsen is relatively young, about 15 years old, and it was established at University of Washington in 2007 with a generous grant from MacArthur Foundation. And this is a society that aims to make sure that we use economics in policy decision makers. So, it's a society with aim to provide a bridge or platform for people from research and policy sector to meet and work together. And society has organized its conference in the U.S., Washington, D.C., since 2008. So every year, the conference has been organized in Washington, D.C., and we saw an interest from Europe. So in 2019, we decided to organize the first European conference, it was organized here, as I say, since I'm in Toulouse, at Toulouse Economics, and then we organized a conference in Stockholm in 2021, it was postponed due to COVID from 2020, and organized by VGI. And last year, Paris Community Conference organized the conference in Paris. And for those interested, I'm very happy to inform you about the conference in Milan in September, September 11, 2012, so it's the fourth European conference that we organized. Call fast, I think, closed, but registration has recently been open, and we had a record number of submissions this year, so that would be a great conference. But I will not keep you too long from listening to our excellent speakers. So for more information, please visit the web page of the society, the link here on the slide, where you can also find information about our journal, the journal about cost analysis. About the European webinar, it's a joint event by the two school economics and society about cost analysis. This year is a third webinar, the first one took place in 2021, where we have professors who are part of the skip test speaker, taught by the university here in Washington as the discussant. And last year, Maal Weiguard talked about infrastructure and energy. And you can find the recordings to those presentation on the link provided here on the web page. So this year, I'm very happy to announce the moderator, Dr. Doramas Georgi Kallevon at the European Investment Bank, and our panelist professors, Susanna Marta at LSE, Massimo Florio, Fiumo Silesica di Milano, and Emil Kiné at the Paris School of Economics. So a huge thank you to all of you for taking time off to participate in this event, and of course also a special thank to Stephanie, Lisa here at TSD and the Foundation, to make sure that we can actually have this event. We need someone behind the scenes to actually make that thing happen. But with that very brief introduction, I give the floor to Doramas to introduce himself and the speakers. Thanks, Doramas. Thank you very much, Henrik. Hello, everyone. My name is Doramas Georgi Kallevon, and I'm an economist at the European Investment Bank. The EIB is the project financing arm of the European Union, and we are heavy users of CBA. You will be able to find a guide to our economic appraisal, how we perform CBAs and cost effectiveness analysis, and sometimes multi-currency analysis that we use to appraise our projects. And you can find it in Google, on the website. This is an open document. I specialized in the appraisal of transport and particularly aviation projects. And in the EIB, we do projects pretty much across all sectors of the economy, with exceptional defense and so on. Today, we have a very distinguished set of speakers. Each one of them is going to speak for about 20 minutes, and we're going to leave the Q&A to the end. The duration of the seminar is one and a half hours, starting at three CET. We will start, we will follow an order, which is bioalphabetical order for first name in Bursley. So we start with Susana Murato, then we'll follow with Massimo Florio, and finally with Emil Kine. Susana Murato will be the first one to speak, and she's going to be talking to us about including well-being measures in CBA, a very innovative area of CBA practice. She's Vice President for Research and Professor of Environmental Economics at the London School of Economics, where she leads the efforts and the development of research at the institution. Her research interests are into social value measurement, particularly when applied to environmental health and cultural heritage goods and services. She specializes in areas of subjective well-being and behavioral economics, and she has over 120 scientific publications and her books, as well as long track record of advising institutions across the world. With that, Susana, the floor is yours. Thank you, Doramas. Thank you very much for the introduction, and thank you everyone for coming. I'm going to share my screen and just let me know if it's working. Is it sharing? Can you see it? Okay, brilliant. So I'm going to talk about the use of well-being measures and methods in cost-benefit analysis in the UK. I mean, this is quite an innovative thing to do, a pioneering thing to do, and I'll leave it up to you to decide whether it's a good thing or a bad thing to do after you see what we've been doing in the UK. So I'm going to talk a little bit about the background for the use of well-being measures in the UK and for our green book, and then I'm going to talk about well-being in the green book, well-being and valuation, and current and future challenges associated with this approach. So well-being in UK policy. The UK government and various UK governments have had welfare or well-being of the population in mind. We had that since, you know, David Cameron, conservative government a few years ago, and even our Labour politicians, at the moment, they said that well-being is essential for how they want to judge the impacts of projects in society. So it is across parties, and well-being has been officially measured in the UK since 1991 through our British Household Panel Survey, and importantly, since 2011, our Office for National Statistics has started measuring well-being on a regular basis. So we now have a lot of data on well-being in the UK. And by well-being, I mean, there's many definitions of well-being, but the Office for National Statistics defines well-being as how we are doing as individuals, communities, and as a nation, and how sustainable this is for the future. So basically, they're talking about individual from a personal perspective and subjective well-being. So reports of satisfaction, purpose, happiness, anxiety, that type of thing. That's sort of the core use of well-being in policy at the moment. They also produce other indicators of well-being for communities and so on, but the ones I've been talking about today are personal subjective measures of well-being. And just to be clear, these are the types of questions that are asked in the Office for National Statistics, and that are relevant for what we are talking today. There's the standard life satisfaction question on well-being, overall, how satisfied are you with your life nowadays, measured on a scale from 0 to 10, 0 being no life satisfaction at all, 10 being the maximum satisfaction. But there's also other questions in the Office for National Statistics. One of them is about what we call your demonic well-being, which is the extent to which people feel that the things they do in their life is worthwhile, are worthwhile, and the question is there on the screen, to what extent do you feel that the things you do are worthwhile. That's also measured in the Office for National Statistics, and the final one is a more instant measure of well-being, measuring a person's mood either on the spot or the day before in the case of the Office for National Statistics. How happy did you feel yesterday, and how anxious did you feel yesterday, a positive and a negative measure of this mood, emotion, feeling. So these four questions are asked in the Office for National Statistics, but the one that's most commonly used, including cost benefit analysis, is the first one. How satisfied are you with your life nowadays? The life satisfaction measure is the most popular. So in the UK we have a green book, which is actually green, that's the cover of the book, the green book, and it describes how major public sector investment projects, policies, programs, regulations are assessed. It's a fundamental book, it's hugely influential and it drives everything. It drives how government, agencies and everyone else evaluates their projects. It basically sets the rules of the game in the UK. So anything that goes in this book is very influential because people will then start applying it and looking at it and trying to learn how to do these things if it's part of the official guidance. So hugely influential, it sets the rules of how we do appraisal in the UK. And it's a relatively short book, the actual core of the book is 84 pages, but then it has a lot of the guidance, the detailed guidance on how to do things, are in the annexes, there's 53 pages of annexes, and in the supplementary material. And there's many separate publications with this supplementary material. And in 2022 the new edition, the book is updated every so often, but the most recent edition in 2022 includes supplementary guidance on using well-being in cost-benefit analysis. And there's 87 pages of supplementary guidance, so there's a lot there to sort of think through on how to use well-being in the Green Book. So it's in cost-benefit analysis. So it's a relatively recent sort of development, you know, the detailed guidance on how to do this. The whole focus of appraisal in the UK is on well-being. So there's a quote there on the slide from the Green Book that says the appraisal of social value, also called public value. It's based on the principles of welfare economics, and it's really about including all the costs and benefits that affect the well-being in society. So the ultimate aim is to measure well-being. So that's there as a basis of the Green Book. And by well-being they mean how people feel. So it's the subjective well-being. So that's at the core of cost-benefit analysis in the UK. And the well-being affects a range of sort of areas in policy development. It affects the initial research. If we look at well-being research, it might reveal what matters to people, which means then politicians want to create policies that help those particular areas that are found to affect well-being the most. It also helps at the strategic stage. It can inform the choices of priorities for governments based on this measures of well-being. And going to cost-benefit analysis, it can inform the range of implementation options that we have by looking at the well-being effects of those options. Or the populations that are affected or how much they are affected. And then it affects the actual cost-benefit analysis with the short list of options. It can inform the costs and benefits of the options, and I'll talk more about this in the talk. And it's of course useful as well for monitoring and evaluation of policies, because well-being is also an outcome. What is the well-being of these particular policies? So it also is helpful for evaluating the impacts of the policies exposed. But in terms of the actual conducting cost-benefit analysis, the Green Book says that, like most other appraisal manuals, that all relevant costs and benefits should be valued in monetary terms as much as possible. And well-being, measuring the impacts on well-being are central to this aim. And if we can, we should monetize everything. If we don't have enough monetized values that are robust enough, then we at least should quantify effects in some way, for example, just in well-being terms, or qualitatively describe what's going on. But the key aim is to try and monetize as much as much as possible, like with any other cost-benefit analysis. And then the Green Book actually lists all the methodologies that are approved and recommended by the UK government for doing this type of appraisal. And in terms of non-market methods, you have very commonly known revealed preference methods, stated preference methods, both willingness to pay and willingness to accept in addition to the usual market prices. But the new kid on the block and the thing that's sort of been inserted recently into the guidance is that we can also use direct well-being responses from surveys and from data and the value of that well-being estimated through subjective well-being valuation methods, which are different from revealed and stated preferences. So now although there's very little evidence and not a great body of knowledge about well-being valuation, subjective well-being valuation, it's been put on the same level as things that we know a lot about, like revealed and stated preferences that we have decades of evidence and experience on. So it's an interesting thing to do to put subjective well-being valuation methods and measures on the same level playing field as the others where we have a lot more evidence on. And of course now they will instigate new research because it's on the Green Book. It's a new frontier in environmental valuation using subjective well-being valuation, which I'll describe in a moment. It makes users of new types of data, namely well-being data, like the one collected by our Office for National Statistics, most countries now collect data on well-being. So we have a very large body of evidence across the world on well-being. And so it uses those types of data. But less is known about these methods and their limitations. We don't have decades of experience on this. We don't have data going back many, many years on this. So this is a case where we ask, is the guidance ahead of the research and the evidence? And it's not a bad thing to be ahead, but maybe it forces the direction in a particular way. But it's an interesting thing that we have this guidance that seems somehow to be ahead of all the research and the evidence on these methods and these metrics. It is recognized in the Green Book that it is a methodology that continues to evolve. And it may be particularly useful in certain policy areas where the other valuation methods are not so strong. For example, measuring community cohesion, volunteering, things like that where we don't really use standard valuation methods to measure that. But maybe well-being valuation methods are better to measure some of these effects. That's at least the hope. Right. So what do we need to actually do valuation using well-being measures rather than the other methods? Well, the first thing is we need to have very robust measures of the impact on well-being of particular policies. So if we have a policy to reduce unemployment, what is the effect on well-being? We need to have those metrics readily available to be able to use these methodologies in a simple way for policymakers to be able to use this. And we know that the confidence in well-being measures will tend to be highest if they come from random control trials, because that's the highest level of quality of these measurements. Or where you have natural experiments. For example, if someone won the lottery, that's completely random, we can check what their effect on their well-being is from this random event. So we could have these random conditions or random control trials, and those are the best measures to come up with these well-being measures. However, there's only a tiny, tiny number of studies that measure well-being in this way. The majority of studies are cross-sectional studies and surveys on well-being that are not random control trials and do not replicate randomization. And we could still have some confidence in those measurements if they come from large-scale cross-sectional studies that are repeated over time, the results being repeated over time, repeated across geographies with certain level of confidence. But we can't have much confidence in little studies with very low power of explanation about particular issues. So there's whole body of evidence that cannot really be used robustly for cost-benefit analysis. So we're still a long way of having a ready-recognised table that can tell us what is the lifetime well-being of a policy on unemployment or a policy to reduce depression or whatever the policy might be. We don't have the data yet, so there's a lot of work to be done in getting the well-being data associated with policies. The second thing that we need then is to value it. Okay, so we have the well-being measure of a policy. What's the value of it? In monetary terms, we need to monetise it. And the green book this latest version goes a step further. It's quite a radical thing to do and presents a measure of a value of well-being of well, first they describe a well-being measure called a well-being. So a well-being is a one-point change in life satisfaction for one year. So if your well-being changes by one point over one year, that's a well-being. So they define this well-being and then they say, looking at the literature, that the value of a well-being is 13,000 pounds and I'll explain how they get to that value 13,000 pounds. So basically they have this number that they're recommending and means that if you have a policy that changes reduces life satisfaction by 0.4 for one year, then the value of that policy will be 0.4 of 13,000 pounds, which is 5,200 pounds. So there'll be an easy way of policy makers using well-being measures and being able to monetise them in an easy way by having this value of a well-being. Now, how do we calculate this well-being? Is this a good situation? Is this reliable? Well, there's two approaches to calculate a well-being. And the first approach is, and there's a lot of stuff on the slide, but I'm just explaining very quickly what this is. It's based on a few studies and it's based on what's done in the health literature, where they have the concept of a quality, a quality adjusted life year. And the quality is the value of an additional life year lived with no problems in full health by a person. A quality is. And there are monetary values as a literature, a small literature that found out the value of or proposed the value for a quality. So one quality is worth 60,000 pounds as per the Green Book guidance. One quality is worth that. So what this literature did, it's tried to match qualities with well-being. And so they note that the average life satisfaction of someone in full health is 8 on a level of 0 to 10. So someone in full health has an average life satisfaction of 8. And someone with no health at all aligns with a life satisfaction of about 1. So they say that one quality is the difference between 1 and 8. So one quality is equivalent of 7 well-being, 7 life satisfaction measures. Therefore to get to a well-being from a quality you divide by 7. And so you get 10,000 pounds. So this is how it's done. It's a bit, you know, there's I'd like to see more evidence on this and more sort of theoretic background theoretical background for this. But it's an easy thing to apply and it's supposedly transparent. Well, it's explained in the book how this is done based on the qualities and these kinds of assumptions. The second way in which this can be measured is by estimating the willingness to pay for a life satisfaction change. And basically that we need to have a regression where we relate life satisfaction with income. And all we have to do is look at the coefficient of income. So when income changes, what happens to life satisfaction and that gives an idea of how much people value life satisfaction in monetary terms from this regression where you regress life satisfaction on income. And a recent literature shows that the coefficient of income is 1.96. And you know, if we use a marginal rate of substitution between income and life satisfaction, it gives us a willingness to pay for a well be of 16,000 pounds. So that's just using the regression between life satisfaction and income and using and looking at the coefficient. So a well be from this approach is 16,000 pounds from the previous approach was 10,000 pounds. And so the government chooses as their as their suggested value of this well be of this one point in life satisfaction for a year of 13,000 pounds. That's how they reached the estimate. But it's all based on a very small amount of literature sources not the body of evidence that we have for the other methods. So you know, this is something to consider and there's a lot more work that needs to be done on finding out how meaningful where well be is and how meaningful this 13,000 pounds is. Of course, if we don't think this is we can do cost-benefit analysis on this we can also do cost-effectiveness analysis using this well be approach where all we have to do is look at the costs of policies and measuring the outcome of the policy in terms of well being. So a policy that produces 20 well we would look at what's the best way of achieving that with the minimum cost. So the usual cost-effectiveness analysis can also be done with this well be approach. So what are the challenges of this we're still not closed. This is a radical new way of choosing public policies on the basis of impacts of well being alone. But we're not close to saying that we have an approved way of doing this. I mean, it's in the guidance but it's based on not a lot of evidence at the moment and there's no theoretical framework at the moment that explains all this. There's some but not a sort of approved one. And there's much less knowledge and evidence on first the well-being effects of policies we don't have a table that tells us what are the well-being effects of all these policies in all these areas. And we also don't have methods to do it that are well you know, well-trialled. I've just told you about the quality approach and the willingness to pay approach but they're new. There's not a body of literature on this or a small body of literature on this. So there's many issues in the measurement interpersonal comparison of well-being we're using life satisfaction but there are the forms of well-being. What if well-being effects because people adapt and then the well-being is the same how do we discount well-being they propose to use the same discount rate as for health effects which is 1.5%. How do we account for non-use values so there's so many questions around this and ultimately we do need for policymakers to be able to use this we need an empirical record a table that people can just go and say well these are the values that I want to use so they came up with the well-bees to provide that kind of ready-recognised table but the approach is based on a very small literature and a very large number of approaches and there are many other well-being measures out there in the literature there are different from this and it doesn't apply to non-marginal changes because it assumes that we apply this 13,000 pounds to any in a linear fashion and so if the changes are non-marginal then this doesn't apply. And how do we embed this in policy analysis really using this approach well it's in the Green Book but it's not widely used as yet and this is a very UK thing is it used elsewhere so is the guidance running ahead of the research and the evidence that's the question that I want to leave with you today to think about it's exciting but there's a lot of question marks around this and that's all I wanted to say thank you very much thank you thank you very much Susanna very very interesting and I'm starting to have a feeling that we will be very short in terms of times for Q&A because I'm sure that your topic will generate a lot of a lot of reaction from the audience we will hear next Professor Massimo Florio he's a professor of public economics at University of Milan and is the co-chair of the Monet Center of Excellence on the impact of European Union research and innovation policy he has expertise in social cost benefit analysis regional industrial policy and economics of science and innovation and of course anyone's had to do anything to do with getting EU digital funds from Europe will have come across the guide to cost benefit analysis of the European of the European Commission which Professor Florio coordinated in the various editions that it had his book on investing in science has recently published in 2019 by the MIT Press and he's led most recently research and advice for the European Parliament on issues to do with the pharmaceutical research and development and more specific with the development of COVID-19 vaccines Professor Florio is going to be talking to us today about CBA in the EU's institutional framework Massimo please you're muted Massimo okay can you hear me now yes okay so my topic is an answer to a question which is the question of how relevant is what we do governments really care about our work on cost benefit analysis because particularly for academics it is a let's say a professional risk that you are unable to really influence the policies so instead of trying to answer myself to the question I will tell you the initial results results of a survey in which we have answered experts about what is going on about the actual usage of cost benefit analysis in member states of the European Union beyond the role of the European Union institutions so this is the question the survey we have recently launched particularly with my colleague Rafael Articulos is inspired by the previous survey by the OECD and by previous work we have done at sealing certain areas such as transport social discount rate and so on we also have been looking at quite recent OECD regulatory outlook but we are also aware of the work of the OECD cost benefit analysis and the environment and Susanna knows well that particular contribution so the issue here is okay we try to go ahead with methodologies in cost benefit analysis but are we able to convince governments to use it and these are the topics that we have asked to a sample of potential respondents and experts and I will tell you something for each of these but with the proviso that this is an initial preliminary presentation because what we want to do in the future is to go ahead with the survey until the European conference of the society for benefit cost analysis. To set the frame thank you you have remembered that there are several cost benefit analysis guides supported by the European Commission regional policy and some of you and some of the participants have been involved in these efforts and in fact there is a long story behind all these that started in 1994 with its lean booklet now that I am a little bit less on the number of pages collapsed from 364 to 98 but in fact 98 in collaboration by the way with the jaspers and the European investment bank and the commission for a number of applications still mentions the 2014 guide as doramas mentioned the European investment bank is the other driving force behind the spread of cost benefit analysis in Europe at the Milano conference will be very happy to host a presentation by Doramas about the new edition of the guide very brand brand new and I will be honored to be the one who will introduce and comment about this Doramas this is probably a surprise for you, I didn't mention to you before I really urge everybody to download this document which in my opinion is there to stay it really contributes to the discipline so please download the new edition of the UK Green Book and this one these two things should be in your shelves next please okay no it's okay there are other bodies at the level of the European Commission or the European Space Agency that regularly perform cost benefit analysis but my focus now is on member states also because most of the public expenditure is managed not by the European institutions but by the member states so how far have academic been successful in convincing member states to use the tools that we have shipped over time so these are the initial results from 48 respondents 24 member states there are three still missing but by September I can promise that it will be on board and there is an interesting mixture of respondents between public administration private sector and academia let's start with the first question which is whether or not it is mandatory to use cost benefit analysis it is mandatory in nine countries but in other five it is mandatory for projects above a certain threshold so it seems there is with the exception of four countries there is the large majority of countries where CBA is firmly within the legal framework there is a legal base which in my experience is very important just in Europe in the US very recently there is a draft revision of the guidelines of the federal guidelines from for public investment and those guidelines played an important role in the US as well so in terms of the decision process it seems that CBA is used particularly for justification of project selection or decisions while less understood is the role that to me is very important for policy learning so this is a more traditional justification which is in a sense confirmed by the fact that CBA is regarded by our respondents as part of a package of tools that help governments to take their decisions about investment projects it is interesting also that most of the work is done at the pre-feasibility stage quite early and only in 12 member states there is some evidence of a exposed CBA which in a sense is a shame because in my opinion and in perspective of policy learning exposed CBA is as important as an example I am not advocating having a one-to-one ratio but I think it would be wise to do so to have more systematically exposed analysis who does this mostly consulting companies hired by the project sponsor this in my opinion is not necessarily a strong point I would be personally more happy to see that an evaluation unit somewhere in the central government or at the sector level line department is there as you can see this is just in four or five member states there is a question in the survey whether there are methodological guidelines and on these there is also evidence that in the majority of the member states there are those guidelines this is from a document in the chat you may find the link this is in the Wadimecum that you are by the European Commission and you may see that some countries have a number of not just of general guidelines but also of specific guidelines particularly in transport, environment and others Ireland is an example but also Poland or Germany usually we are interested we economists we are usually interested in the social discountries the variation across countries is now surprisingly high as you can see according our respondents with less than 2% in Germany and close to 7% in a Baltic country and certainly this is something that we have to study a little bit more why there is such big variations in terms of sectors you will not be surprised to see that still cost-benefit analysis refers to transport in different modes but personally I am particularly to see that now in 4 or 5 countries there is also projects in scientific research and technological development and innovation in the scope of cost-benefit analysis which is something I have been working on in the last years the overall assessment by the respondents is that in terms of increased efficiency in public spending with just one expert more pessimistic in one country but this is not too bad after all and also there is great variability in terms of when governments started to do systematically cost-benefit analysis within some countries since the 70s and others just joined I would conclude now with two invitations the first invitation is for all participants to this webinar to contribute themselves you would find in the chat now I hope this link that would lead you to a set of questions that will take I promise 5 to 10 minutes to answer and the idea is to deliver the results at the fourth European conference of the Society for Benefit Cost Analysis that as Eric announced in Milan organised by my department with the Society 11 and 12 September we have now a large number of submissions so it seems the program will be really rich and again in the website in the chat you can see the link in the description if you still would like to join this conference so thank you very much many thanks Massimo of course anyone who has already questions to pose to either Susanna or Massimo if you could please type them on the Q&A facility and then I would administer and edit it and deliver the questions to the speakers next we have Emil Kiné he is an emeritus professor at the Col de Pond Paris Tech an associate member of the Paris School of Economics and is a member of the French Academy of Technology his fields of expertise are transport economics, public economics and cost-benefit analysis and Emil is going to be talking to us today about challenges for CBA in a period of ecological transition Emil please Emil you are muted okay do you hear me yes thank you so the ideas I will present you are based on the idea that the transition period the ecological transition period where we are raises new challenges and implies new answers for CBA and especially on how to implement it and on its role in helping the decision making process what following I cannot I cannot go to the next one just click on the presentation then okay okay thank you very much so let us first remind the traditional framework of CBA in oversimplified terms it CBA considers a small marginal project at the margin of a macro and meso economic scenario and checks its net present value the scenario implying macro economic evolution such as GDP growth, price evolution and the state of regulation is mainly a matter of forecast based on extrapolation of the past and in the traditional framework of CBA too in which many of the current project are still are surprised some attention is paid to risk but usually not given any important role almost no account to uncertainty I indeed wish between risk which is probable and uncertainty which cannot be prioritized this framework no longer holds extrapolating the past is no more possible at the country level as well as the international level ambitious policy a being at coping with global externality in operation they imply dramatic changes in key sectors which are often the main subject of CBA which are energy transport, building and land use agriculture too and with some important characteristics the first is that so strange must be coordinated you cannot think of an evolution of transport without any consequences on the energy supply power supply for instance due to the rhythm of decarbonation and electrification of means of transport in all sectors and changes huge externalities are implied market solution does not work and public intervention is needed and also accessibility the success of these policies international and domestic cooperation on top of that risk and uncertainty which took not a very large part in previous missiles are now pervades decision however while the current CBA procedure does not work well decision ever need more and more light the general idea about is that you may use scenario but how so first I will see how to design scenario scenario would imply input including both external framework such as international agreement technological progress level of climate change and so on and also collective targets as many countries or group of countries or region as fixed objectives quantified objectives in term of decarbonation and also protection of biodiversity so scenario which are of course macroeconomic should give way to outputs which are for each sector of the economy programs indicating for instance in the case of transport the volume of traffic the share of public transport and the length of new public transport line built and so on and also not only program for each sector but also tools to reach those programs such as shadow price and taxes, regulation, research information and so on and of course the common idea on these points are to have a central scenario and also several other scenario taking into account the main street and uncertainties but the point is how to determine the outputs of each scenario the outputs are programs and tools ideally we should use economic optimization and here cost benefit analysis should play a new role there a role which right now is not often used in order to define the decision at the sector level but this new role of CBR has a lot of a problem the first problem is the problem of implementation because when designing a scenario you have a lot of possible variants to assess in terms of tools and programs if you use too few variants too few variants you have a risk strong risk of under optimization and if you have too many variants you have a lot of complexity very difficult to deal another problem is a problem of procedure is a determination of outputs implies a lot of political decision such as decision on taxes regulation and also infrastructure and so decision present a lot of problem of acceptability acceptability and political agreement we are we see currently right now many decisions taken in terms of taxes on the entire program for power supply are not conformed to the economic analysis for political reason for sake of political reason and acceptability so this the scenario should be set not directly by economist or by actors but by consultation between actors and also endorsed by the public authorities there is also a problem of tools to optimize the scenario the tool provided by economic theory are a bit feeble on the side of risk and uncertainty classical risk analysis through utility expectation is of moderate help as many of risk is on the form of ambiguity of uncertainty and ambiguity theory is not enough robust right now to my opinion we provide a robust solution able to convince political decision makers well once anyway once scenario would have been designed we have to assess each individual project and to ensure coherence between project and programs here we come back to the usual CBA and the general principle of well known to recon and pave a project using the framework set by the previous step in terms of taxes regulation information such a price and so on tools and theoretically the program the process procedure should allow to fulfill the program the lot of the set of project the passing the NPV should should give just the level of program which is optimized in fact there will be gap between programs and the same of individual project and problem arise how to fill this gap a classical arbitrage between price and quantity usually the main the main objective of public policy in terms of transition are expressed in terms of quantities and the economist translates them in terms of price but if there is not the perfect match you have to choose between them so the the procedure I suggest I very quickly analyzed seems very logic it may be it may seem also a bit ambitious and it's not sure that it will be possible to put it at the end but my view is that the state of mind is a good one is the only one possible to have a logical way to address a new CBA challenge and the main message I would leave to your reaction is that the street to climate change introduce new challenge for CBA first extrapolation of past tendency does not work and second the future is made of public decision and uncertainties so CBA should be adapted to this new situation along with the following line first CBA should be more often used for program assessment and adapted to this task which is not which pose may pose a lot of difficulties second the process of decision making and CBA implementation should pay more attention to the currency between programs and individual project second I've already mentioned a problem of taking into account uncertainty and top of that the whole procedure of setting programs and choosing individual project should involve more cooperation between the actor of the process economist public authority and private actor which up to now in each specific individual project acted relatively independently so thank you for your reaction of this idea and thank you for your intention many thanks Emil indeed we now at the European investment bank two three years ago we were turning to the climate bank which means that 50% of our lending has to go to climate action investments so directly involving either adapting or mitigating climate change and 100% of our investments since three years ago have to be parties aligned but the emphasis on investing in technologies to adapt to climate change has faced has forced us to face a new world of increased uncertainty some of that some of that has been managed by putting in a scenario to the price so the social cost of carbon but even so all the technological alternatives that we face and the need to invest in them still leave analysts like me doing conduct in CBA forcing us to exert a very high degree of judgment on that when we perform CBA so by all means a most relevant issue and that is bound to remain very relevant for years to come so to the audience again if you have questions please post them on the Q&A preferably identify the speaker to whom you would like to the question post. I think we've had three fascinating truly fascinating presentations I mean we've seen at the state of of CBA use across Europe pay in particular attention on the European Union by Massimo Florio Emil has has brought into the picture the fact that we are facing climate change and a brave new world as it were it brings the extent of uncertainty into CBA to a new level bringing all sorts of challenges to CBA practice and application and Susana Murato has made a very very interesting presentation on the issue of well-being and if it wasn't complex enough already to explain to decision makers the issues about welfare and so on what CBA is really telling us these I must say it is it poses a challenge but it's a challenge in the right direction I think because one of the problems that we have when we produce CBAs is then explaining it to the decision makers about what is it that we are telling them that the financial returns don't tell them and then we have to know it's not about jobs it's not just the value of the extra jobs that we create with the investment is we're looking at social welfare when we sell the social welfare what's that what do you mean and then it becomes challenging and by all means anything that is about improving, broadening and coming up with easier to understand measures of what that is whether welfare or well-being it's a huge help it's a huge help I must say to the actual practice so we move on to Q&A we have 25, 26 minutes left the first question, the first two questions actually are for Susana Murato one is how are these measures used to measure the impact of the costs of implementing and operating a policy that's one and the second one is to what extent is uncertainty in BSLY taking into account in estimating the value of well-being Susana please thank you Doramas and thank you Liza for the questions in terms of the impact of the costs of operating a policy and how that reflects into the well-being approach I mean if it's market values that you'd use to estimate those prices I don't think you'd have a need to use the well-being approach the well-being approach is mostly for non-market for non-market values it's harder to measure but there could be costs that are non-market costs of a policy in which case you'd think of using the well-being approach so the unit values for the well-being the £13,000 that they recommend to be used they're to be used for whether the change is a positive change or a negative change so benefits and costs they propose using the same value of course it's recognised that losses and gains in different ways how they think that could be adapted is that the change in actual well-being from a gain would be different from the change in actual well-being from a loss but then they'd use the same unit value to value whether it's a gain or whether it's a loss it's an assumption there could be better assumptions given that the different valuation of losses and gains that they propose there but this is only for non-market sort of costs so the market costs would be estimated in the standard way using market prices in relation to the second question also a very good question about uncertainty in value of statistical life here and how that translates into the value of a well-being that's based on that quality estimate one thing that they say in the guidance is that if the values are used to be updated regularly as governments change the value for statistical life here that they use in their appraisal everything then changes including the well-being values if it's based on that quality and value of statistical life here value so they say this value needs to be revised on a regular basis they also advise the user of sensitivity analysis when we have values that are very uncertain or can change but beyond that they don't offer any guidance on ranges of values that we could use for example they just say £13,000 is the value that we should use the value based on the quality is is a bit lower than that but that's all they say they don't actually specify a range but they do say we should use sensitivity analysis and so I think that's the way they go around that and change the value regularly to reflect possible changes in the value of statistical life here is values used in governments many thanks Susana I move on to a question actually by Gabriela Borges it's a very interesting question I'm not sure there's two ways of interpreting it it says can you please comment about EU experience with CBA in the financial sector I guess this is directed to Massimo and I I have two ways of interpreting that question one is use of CBA by the financial sector and the other one is application of CBA to the financial sector I don't know whether Massimo would you want to answer both questions both interpretations or you would rather have Gabriela specify be more specific what she actually means well if it is the impact on the financial sector I think Doramasu are more qualified than me because after all the European investment bank is the biggest development bank in the world so in your case there was an impact and I suspect that also for some others development bank it has played a role I don't know Doramasu if you have information about this yeah well I mean talking about the use of CBA by the financial sector we we see an increased overlap I mean we see for example the banks with whom we co-finance because the AB normally can invest up to 50% of the project cost but we see that the commercial or commercial investment banks are increasingly moving into areas of valuing most of them or many of them already have a view on the cost of carbon not just the social cost of carbon but also what carbon pricing will imply to the promoters that they advise for example and they have to take a view on that you cannot invest in a in an airport you cannot invest in an airport without taking a firm view of how do you think how much the passenger will have to pay in 20-30 years time for carbon emissions and we see that commercial banks are moving into that another area where we see increased use is is valuing ecosystem and biodiversity infrastructure, linear infrastructure and so on new facilities that do have an impact on ecosystems and biodiversity as well as various other sectors and even if at the moment there is no need to conduct a CBA one would expect that CBA will point you in the direction of where future policy will take you over the life of very long-lived projects so ignoring it or ignoring what CBAs will have to say about it is something we see that commercial banks have commercial banks have moved towards without having to be told to do so so you see they are the private sector acting rationally as it were so that's one element in terms of the actual usage of CBA itself by public banks there is a lot of divergence in use but there is a recognition to the extent that you are going beyond cash flows into public policy objectives however that is defined in various across the country the way you address that is through CBA there is no real other tool to address that so the direction of travel is towards more CBA and it was very interesting to see in Massimo's slide how when CBA started to be when guidance was started to be used in the in the EU the trend has been to come with ever thicker more detailed guides to the extent where we are now we leave it to the commission to the member states to implement it to this side on the other interpretation I would just say that to the extent that regulatory impact analysis uses some kind of cost benefit analysis there is an obvious scope for cost benefit analysis in financial regulation because it should be recognized that in some cases even modest changes in financial regulations may have a lot of socio-economic impact so I am aware of some work in this area but it is certainly underdeveloped relative to what it should it should be and I would expect that if we move towards a bank union in Europe certain rules some kind of cost benefit analysis as part of regulatory impact analysis should play a role yes I agree Emil of the three speakers you came across as being the one a bit more skeptic about markets could you please elaborate on that yes yes I am not sceptic on markets because everyday we rely on markets I am in fact sceptic on public authorities because I think that in the present situation of transition they have a major role to take because most of the changes which have to happen are not driven by market forces but by collective collective concern and can only be arise through collective economic calculations with cost benefit analysis and in my view they should ensure more currency between the policies they imply they impulse on each sector because they have enormous power to impulse policy in each sector in each important sector which are transport, energy where they are already very strong actors and also agriculture and land use and so on and I am worried by the fact that they don't play the role of a coherent season and leader of a clear direction which they should in my view they should in my view take so I am not against skeptical about the market but I skeptical about the role of market in this transition I am sure that Elon Musk has a lot of good collective instant influence but it's not sure that it's sufficient to change the pace of economics and it would be better that public government take more firm role and leadership staying with you Emil a bit longer and obviously the focus of your presentation was on uncertainty so we have this world brought to us with the increased awareness of policy relevance of climate change which has brought uncertainty straight into the center of the picture now traditionally in economics one useful way of dealing with you know uncertainty or between uncertainty and risk is anything that you can quantify probabilities and uncertainty is where probabilities are much weaker or nonexistent but still when we give the results of our analysis to decision makers they want to have probabilities not even you warn to them I can tell you that it's 30% that this happens but there's no evidence back in this this is just almost a hundred could you please elaborate on how you would see at this stage I mean over the next two or three years uncertainty playing a greater role in CBA it's a huge huge problem which should deserve a lot of research and studies which are now undertook my view is that we are very often in between the situation of polar situation of risk where we know exactly the probability and you can do classical expectation evaluation and total ambiguity where you know nothing in fact you have a lot of information for instance on the future of climate change but depending on models and models are very widespread even in their distribution so I think that some some methods such as Maximine can help can give information but what I what I fear is that this information is not sufficient to to give the politician decision maker robust face in the result and give him tell him the decision I think that the decision is that case should be for so the matters which are matters of collective choice for collective concerns that those matters should be debated and given to the forefront and debated among citizens and decided through a collective choice and in that case it may happen that the CBA does not tell exactly what I do what I must be done but just to tell if you think that the probability of such event is less than 10,000 or 10,000 very small then the decision is correct and reversely you have to take the other decision and give the information to the public to the citizen to the democratic power without really telling here is the solution which should be done and I think that in the present this inserted this should give a pace to more debate it's already the case in the private for insert insurance and reinsurance you have a lot of calculation and you have also after that decision which is taken collectively by the member of the board and so on on expert judgment and the same is the case of of collective decision such as such as so the implied transition the final decision should be informed by CBA but not dictated by CBA understood yeah I think actually this issue of as we're socializing CBA and decision making which is you know springs from what you're saying links up very nicely directly with one point in Massimo's survey which is who performs CBAs Massimo would you like to elaborate a bit on that? Yes there is clearly a market for cost benefit analysis and this market tends to be occasionally biased because those who pay for a cost benefit analysis report as part of an investment appraisal project tend to be the promoters while those who do the analysis tend to be consultants there is nothing wrong about this except if the those who read the reports in the public administration or in the agencies do not have the skills to really critically read those reports and discover where there is optimist bias or other traps that could be there so nothing against having this market of consultants working for the public administration on this but I would urge more hiring in the public administration of economists with the right kind of training to properly properly read those reports Doremus can I also take the opportunity to answer one question in the chat to why there is no Norway there is no Norway unfortunately there is no more UK that should be there we focus in the survey on member states of the European Union but I think it would be worthwhile to enlarge as a second step the analysis to other constituencies the advantage of focusing on the EU member states is that those states has to follow certain rules in public standards in state aid and so on this is different from others but from an intellectual and practical point of view I agree the country has a UK Norway or Switzerland for example should be there okay thanks what question for Susana the you mentioned in your presentation that the theoretical background to justify measures of well-being is still if I understood you correctly still the the I happen to have with a very practical aim of explaining to decision makers some of whom have no training in economics of explaining to them what a CBA tells you and explaining to them what economists mean by welfare I went back not just to a traditional microeconomist but I went to Marc Blaug's economic theory in retrospect and he has a number of chapters on the historical development of welfare economics and how the various concepts and linking up willingness to pay with welfare and so on it actually opened my eyes quite a bit into seeing the degree of granularity of how complex that issue is so the question to you is did you see such a development theoretical development into well-being as a precondition to measures of well-being being more widely applied to CBA or are you happy with the way the Green Book has gone which is, you know, this looks useful, this looks good let's use it and then perhaps let's worry about the granularity of theory as we go along as we work. Yeah, thank you for that it's an interesting matter in fact the theory doesn't matter for applications in practice we have a whole body of consultants doing social value measurement in the UK and they don't follow any of the academic papers they just do it in their own way in a simple way it doesn't necessarily follow the Green Book which I think is why it's important to have the Green Book measures in place because at least they relate well-being measures to CBA and to the general welfare economics even though the well-being part is not still as well defined as it could be but without that guidance people just do it anyway because there's a requirement for example in the UK through the social value act to actually do social value measurement and so people go ahead and do it following whatever rules there might be so it is important to have this because at least it's some rules rather than no rules it is a shame that there's not enough evidence and theoretical sort of developments of this and it's because it's not a priority the priorities are, you know changes in meal setters and a lot of money put in certain areas not necessarily in this development and so it's lagging behind I know I have many colleagues at LSE that are experts in this area and keep applying for grants to do this area and the grants are not awarded so it is a little bit that it's not seen as a priority having the actual sort of theory there so I think my view is that actually let's just go and do it because otherwise nothing will maybe this will lead to some more funds to do the actual research behind it rather than the other way round although I'm a little bit uncomfortable about these things being there side by side with stated preferences and reveal preferences which have a lot more body of evidence I would just put it in a different layer of not at the same sort of level put it at maybe a level below because of the lack of evidence that's how I would change it but I think it's important to have it there to prevent people doing things that are completely incompatible with the green book and to sort of maybe create the impetus for more research in this area very good I mean it's actually a very interesting very interesting point because we in the AIB also use multigradie analysis for example and one of the things which we sometimes we have no choice because the projects or the programs in which we apply them happen to be so broad that it's very hard to apply CBA to all the elements of the project but we do struggle linking up the MCA to the CBA and making sure that we value with them on the same level play field so it is actually an issue of a lot of practical I mean we practitioners it's a difficult which is very relevant to us very well I think we are very close to the end now we have only one minute left and I'd like to use it to thank our speakers for what has been very illuminating and a very very interesting set of presentations I hope that the audience has found it as interesting as I have and I'd like to thank the School of Economics Henry and Stephanie for their invitation and assistance throughout this webinar and of course the Society for Benefit Cost analysis and I'd like to for the invitation and for the opportunity to have this session and of course I will also finish with the reference to also that Massimo said at the beginning that there is this conference coming up in September of the European edition of SVCA and that we we look forward to meeting up again there so thank you very much to everyone and looking forward to the next exchange bye bye