 Hello and welcome back. Projections of inflation and the risks surrounding them are a key part of how monetary policy is conducted. Yet when times are unpredictable and uncertainty mounts, forecasts can struggle to provide enough information to policy makers. Forecasting becomes all the more difficult when economists are faced with measuring the impact of extraordinary events such as the pandemic and Russia's unjustified invasion of Ukraine. Both those events, as we know well, have led to price pressures shooting up, and despite estimates that the shocks would prove short-lived, they've certainly proven anything but. This panel will ask experts from central banks and multilateral organisations how the processes and techniques used by economists can best adapt to a volatile inflation environment. It will be chaired by Philip Lane, Member of the Executive Board of the European Central Bank. Mr Lane, over to you. Thank you, Claire. In the last session, I think there was some discussion of the uncertainty about understanding what's already happened. But when we're trying to understand what might happen in the future, the uncertainty takes a different form. As Claire said, forecasting is a very central component of monetary policy. We know there are lags in the transmission of monetary policy, and so it's inescapable. And then the question is, when you're forecasting, do you actually have a forecasting process, or do you use rules of thumb or intuition, gut feeling, and so on? And obviously there's a balancing act. It's good of a machine, it's good of a regular forecasting process, but ideally you want that machine to be sufficiently agile that indeed, as Claire said, when there's extraordinary shocks and so on, that it can adapt. That, in turn, leads to the balance between using the models versus using judgement, and in fact some blend of modelling and judgement in creating forecasts. And then maybe a very important issue, because I think it's important for this panel ideally to, if you like, forecast the future forecasting, not just have a retrospective about what just happened. And the fundamental issue is what just happened, basically a collection of black swans, very big, but they don't necessarily define the future in the same way. Will we go back to having a more regular business cycle, or indeed do we have to think harder about structural change? So I think we've assembled, I think, a fantastic panel. What's partly interesting from my point of view, it mixes multilateral organisations where you have the conundrum of forecasting the aggregate, as well as forecasting individual countries. And we, in the Euro system, have that issue, but we also have people involved from, I was going to say, a unified United Kingdom, but of course having the Bank of England as the Bank of the United Kingdom is very important, and of course with the Fed. So it's a great panel. I'm under severe instructions to finish on time, given the rest of the agenda. So I will be a tough timekeeper, I hope. So let's start with Alfred. Alfred, over to you. So I will discuss some of the common challenges that we are facing in assessing the developments and to protect the economy, and how we have been dealing with that in the IMF. I will structure my remarks in three themes. First, the challenges we were facing and how we approached the IMF. Then I will go into the issue of how we adapted our forecasting methods during these uncertain times, especially with regard to how we dealt with it during the pandemic and now during the energy crisis. And finally, I will conclude with what lessons we can take forward. The three lessons. First lesson is be balanced. And that means when we are doing our forecast, we need to be balanced with regard to how to incorporate common factors, which are across countries, and also taking into account the idiosyncratic country-specific issues and the plumbing of countries and how things will affect the country in particular. The second issue is, under the second lesson, is be nimble. Be ready to continuously develop and add new tools. And that's something we have been doing during the pandemic, as well as during the energy crisis. And the final lessons for us that we learned is be modest. Focus on avoiding misses that would cause major policy mistakes and live with marginal variation around the model forecast. So those are the three lessons we have. So macroeconomic forecasting is difficult at the best of times. As you all know, we are dealing with economies which are complex social systems. We take stock occasionally of our forecast performance and we compare our performance to also other forecasters. And what we find is that we are fairly good at short horizons and that means one year. And as soon as you go out, even one and a half or two years, the forecast is deteriorating fairly quickly. That applies to the IMF as well as to other forecasters. And the reason is that economies are suffering constant shocks and I should also say we see policy changes that we did not expect. Another insight we have with regard to forecast is that the higher the volatility of GDP in a country, the larger the forecast error. And on a side note, when we are trusting for higher volatility, the forecast error is the same for low income countries or for advanced economies. We had some big shocks over the last two years which made forecasting in particularly challenging. And many of the macroeconomic relationships broke down during the pandemic. Think about Okun's law, the link between unemployment and output that weakened because of job retention schemes. It also weakened because there was a preference to work fewer hours during the week. Think about our famous story on excess savings of the past year. Essentially that represents a breakdown of a historically very tight relationship between household disposable income and consumption as you can see in the left chart. And another challenge we faced during the energy crisis was the massive volatility in energy markets since Russia's invasion of Ukraine. The right chart shows the breakdown of this relationship between Russian oil and Russian gas prices in 2022. So what does this all imply for forecasting? One question is the trade off between focusing on country specific factors versus common forces. And what do we mean by that? When we are doing our projections at the IMF in the world economic outlook, this is a bottom up exercise done by 190 country teams. They are starting to work on their country forecast looking and assuming a number of key global variables such as commodity prices and benchmark interest rates. In the forecast round we are undertaking then we perform many checks to ensure cross country consistency in how these assumptions are incorporated. And that is an iterative process that's not just a one time exercise. But important to note that country teams have substantial discretion including in terms of deciding how much the economy is affected by a common factor, embedding country specific information in their forecast and also importantly the tools and the models they are using. Giving country experts room to incorporate iterative factors factor is crucial because what we know is that domestic factors tend to be more important than foreign ones for the path of economic activity on average. But it's also true when we do exposed evaluations we find that common components are not sufficiently taken into account as they could be and should be. Especially during times of large global or regional shocks the common factor can be dominant enough to justify a more top down approach to forecasting. As this chart shows on the left side you see the GFC where we were off and our forecast was too positive. We were off in our initial forecast to the pandemic. But in between the years the forecast errors were nicely distributed around zero. To give a concrete example during the early months of the pandemic it became clear that the expected progression of infections mobility restrictions and the sensitivity of output to mobility would dominate the immediate outlook. Translating these quality factors into a quantity forecast faced three challenges. First when we did our March 20 spring view outlook we knew and saw that infections were clustered around some countries. But we had no idea how these infections would actually percolate through the world economy and we didn't know how highly that virus and contagious is what and how quickly it would spread. Second it was unclear how severe the mobility restrictions would be in different countries. And third there was little historical data to extrapolate how economic activity would respond to such restrictions. We decided to take a centralized approach engaged with epidemiologists. We consulted on how the virus would be spreading. We then made an assumption on depending on the economic structure that most contact intensive share of activity in a country would matter more for a particular country. And we translated that into lost days of GDP. We added domestic disruptions, country teams factored in impact of international demand and supply spillovers. And finally based on available policy space we factored in offsetting policy support. Overall this did quite well. With the same year forecast we made it in spring 2020 for the euro area being off only by about one percentage points. Not a large error given how massive the shock was. The elephant in the room with repeated underestimation of inflation over the past one and a half years points to the need of being nimble. And to see how off we were from this charts. And of course we know that the pandemic supply shocks were unpredictable but even when the shocks materialized it was challenging to quantify the effects on inflation. Forecast errors were large. We highlight two specific shortcomings of our models and how we try to improve them. The first issue is about the right amount of granularity. Most of our forecasting was previously done using international oil prices as proxy for overall energy prices. This worked well in the past. It did not work well when gas prices shot up because Russia cut off gas to Europe. Now allowing gas prices to enter inflation projections separately thus was a problem for projecting energy inflation and ultimately the pass through from energy to core inflation. We now project inflation more disaggregated as you can see in this chart. The second key shortcoming and much harder to correct has to do with non-linearities. Firms might not react strongly to a 20 percentage point increase in prices of one of the minor inputs. But that changes when that input is increasing by 500 points. And the models miss the marks because these movements were well out of range and these models were not trained based on these parameters. So forecasting is a humbling process. What lessons can we apply going forward? First, an enhanced role for top-down guidance will probably stay with us at the IMF. That helps react in real time to important developments which in a possibly more shock prone world could prove important. But we need to be balanced. It is in fact, it is in credit factors, country specific information matters. Second, be nimble. We need to continuously monitor and enhance our tools. Part of the way forward is also to exploit under-use data sources and incorporating new data including big data in a flexible way. And finally, we should be modest. Macroeconomic data remains difficult to collect, notably GDP. And GDP as you all know is prone to large revisions. De-emphasize point estimates and focus on forecast ranges especially when forecasting informs policy making. Avoid forecast errors that would yield cross-policy mistakes. Scenario analysis has a clear role to play allowing both to implement policies to avoid a downside scenario and to contingency planning. And that's what we also did with regard to the Russian gas shut off when we came out that this would be a deep incision in growth. And that triggered policy reactions and actually took care of the potential deep recession in Europe. Thank you. Thank you, Alfred. And the next speaker will be from the OECD. I hand over to Claire. Hi, everyone. Thanks very much for inviting me to speak on this today. I'm going to talk to you about how the OECD approached forecasting over the last few eventful years and what lessons there are from that. Before we get to the substance, just a short intro on what we actually do in this space. So we've published four sets of projections each year, two economic outlooks which do projections across a range of variables for every member country of the OECD and then selected non-member countries. And we do two interim outlooks between those which provide basically updates on inflation and GDP for the G20 countries that we consider. So we've got 38 member countries, mostly but not exclusively. Those are high income countries and added to that we do projections for another 10 countries including China, India, Brazil, Russia, Indonesia and some others. Like we've heard from the IMF, it's an iterative process. We have country desks that produce individual country projections and then we have a central top-down set of guidelines and a common narrative that ties that together. The individual country projections together into a global picture with consistent trade numbers. A key part of our process is we have a set of committee meetings where we discuss the projections with member countries. So every central bank and finance ministry comes along and we take comments from them into account. We don't agree them with countries and in some instances they're quite far from agreed. I've been using the word projections rather than forecasts. What we do is we take a set of assumptions and we produce a modal projection based on those assumptions. As an international organisation we have the luxury I would say of being able to take quite a flexible approach to how we treat assumptions including assumptions around policy. I think that's quite important. So we can adapt our approach to circumstances and we do that so it focuses on the most relevant issue of the day. I'll show you a couple of examples of that. We're telling basically a story about what could happen under that set of assumptions in an internally consistent way both within countries and across countries. What we're doing is really aiming to make recommendations on appropriate economic policies. For example we might choose not to assume a fiscal response in response to a shock. We would then show some projections about the growth prospects that might lead governments to change their fiscal policy and to boost discretion spending for example. And so you'd see a growth out term quite different from the one that we've projected. Obviously for policy making institutions they're arguably much more constrained in how they treat policy. If you're both the user and the producer in that sense of a forecast it's slightly different task and that of course complicates the task in terms of producing the forecast and communicating it and its role in policy decision making. If we look at projection errors obviously what matters is why they have happened. So there's useful information in them that you can learn from. You can see here the errors of OECD projections of the last 50 years. The picture is very similar to other macro forecasting institutions and as Alfred said you know as you'd expect errors are greatest around major crises. We never have a tail event in our central scenarios. A pandemic, a war, an earthquake. You never have that before it's happened in your scenarios. And again you can see here that the decay rate on the accuracy is very very rapid. Looking ahead even by a further sort of six or twelve months you can see it makes quite a difference. In the case of the most two recent large shocks that I'll talk about. I'll show you a bit more detail what we did. In the case of Covid and the Ukraine war two things in particular were underappreciated in the projections. One was just the resilience of economic agents and how economies adapted and the other was the scale and the impact of fiscal policy. So how to deal with exceptional events and highlight uncertainties that policy makers face. So in exceptional circumstances what the OECD did is describe its projections as bimodal. What I've shown you here on the left hand side is what we did in June 2020. If you think back that was obviously a really challenging time to do projections. At the time many economies were in lockdown in response to the pandemic and the debate at the time was all around what would happen when they reopened and whether or not they would need to be closed down again. And so the OECD decided to produce a bimodal forecast at that point with two growth scenarios. As you can see here one that had the single lockdown and one that had another one later in the year. Basically because this was dominating every other issue at the time. Another thing that we did you can see on the right hand side was around using data to track what was going on. Lots of people did this but of course lockdowns and restrictions meant that the usual relationships were very difficult. We're not holding it was very difficult to interpret this and of course official data comes with quite a long lag. So we produced a tracker based on high frequency data, mobility data, container ships, Google trends, internet searches, the like. And we used machine learning tools to develop that into a GDP tracker and you can see on the right hand side its performance there. Moving on to the Russian invasion of Ukraine. So we did an interim set of projections the week after the invasion. So the first week of March obviously uncertainty at that point was huge. And actually we decided not to publish a detailed set of projection numbers but rather to highlight the risks and the exposures through different channels. So we looked at energy, food, refugees and the like. And what the OECD did was produce macroeconomic model estimates of the potential effects and you can see that on the left hand side. The example you see is a global simulation of GDP impacts of the war. We also did inflation and the potential to offset these impacts through fiscal policy responses such as those cushioning the effect of... energy price spikes. I'll pause briefly here on the treatment of fiscal policy. It's an issue that's come up a bit in the discussion yesterday and this morning and it's worth pausing on what we can learn from the experience of this. In terms of forecasting there's quite important questions about how fiscal policy is included. Typically this is done after it's been announced but in recent crises there have been points when it's been very clear that a fiscal response would be forthcoming in some form but not the detail of it. So it's worth thinking about how to think about that and anticipate it. It becomes really important for both forecasting and of course for monetary policy decision making. They're very tricky. There's then a question about how we model the impact of fiscal policy. In the last few years what we've seen is the policies become much more active and quite a bit more creative in the discussion earlier. And it's not just about scale but also about policy design. Different types can have quite different impacts on growth and inflationary effects. Some obviously are directly on prices as was discussed earlier where you can calculate a very arithmetic impact on inflation. More traditionally we think about fiscal policy in terms of the impact on demand including through job retention or transfers to households and of course it can also affect supply. That's a longer term phenomenon but not exclusively and some fiscal policy can have a short term impact on supply particularly some labour market policy. Just briefly on the right hand side I've shown another example of some innovative tracking that we did around risks. As part of the OECD projections in 2021 we did some simulations of European gas reserves and that's an example basically of how in shops become quite complex. Economics can adapt and learn quite a lot from other disciplines and apply some of that. Here using energy expertise obviously there was quite a lot of work around epidemiological expertise in the pandemic and how you build that in. Obviously weather and climate modelling will be the other area that will expand in this space. But despite learning lots obviously we have to be humble. You can see here we like everyone else has consistently underestimated how persistent inflation would be as this shop has fed through. On the left hand side is the OECD version of the chart that Gita showed Monday evening. It's the same story. Others were investigating further aspects of the inflation persistence and what we've learned including around profit shares and how those were early on in some sectors and the space there then for those profits to absorb some weight rises going forward. We've also seen in many countries it takes longer to bring inflation down when all countries are acting simultaneously tightening monetary policy. The exchange rate channel is limited. There's obviously a lot of uncertainty that we still don't understand the role of inflation expectations, how they're formed particularly after a very long period of low inflation. Questions about the labour market and whether unemployment is still the best indicator of slack post pandemic when we've seen quite substantial structural changes to labour markets. And as I said the size and the timing of fiscal policy. I'll show you what's on the right hand side here just because following what Alfred said we obviously do detail forecast across a range of countries. We had a look at whether or not there are any patterns to the errors there looking at it by individual countries, size characteristics, those sorts of things, income. Very hard to see any pattern at all. What you can see here is if you cut it by country blocks kind of what you'd expect which is where the shocks hit more largely. The errors are larger. I'll just conclude then with three points really. So one is when you're facing novel shocks or particular uncertainty scenarios can be really helpful in teasing out the impacts of particular assumptions and in some cases they can be more informative than running full forecasts. Second we need to think quite carefully about fiscal policy and how we handle it particularly as fiscal authorities have become more activist in the response to recent shocks and have applied fiscal policies in different ways and that's likely to continue. And finally the importance of improving and updating our tools and the data that we use including borrowing from other disciplines where this can help us with particular shocks. Thank you. Thank you Claire and now we switch to Hugh Pyl from the Bank of England. So good morning everyone. Let me also say thank you very much for inviting me to this excellent conference and it's a pleasure to be part of such a distinguished panel. So I want to make three points. First of all I think I want to acknowledge the difficulties that have been faced in forecasting inflation in recent times and to do that I'm going to show you an MPC forecast published in the Bank of England's Monetary Policy Report from the spring of 2021. I mean given our role and given the lags in transmission this is a forecast that is probably relevant for policy considerations at the turn of this year. So as Silvana said yesterday the Bank of England has a long history of emphasising the uncertainty around forecasts and downplaying central projections and reflecting the views coming out of the forecast process in a fan chart. And that fan chart is quite wide so the outlook for CPI inflation for 95% confidence is plus or minus six percentage points and yet as you can see in the chart we have managed to overshoot that quite significantly and in quite a persistent way. Now it's quite well trodden ground to discuss what the implications and causes of that are so I will come back to that in a second. But I think it's also important or interesting to think about what the discussion around this forecast was and the discussion around this forecast centred on what would happen to that accumulation of excess saving or money overhang that Daniel described yesterday. And the concern was, the risk was that there may be a consumption boom that is coming from the spending of that accumulated saving which would be inflationary. But what I think is sort of important to keep in mind there is again as Silvana said yesterday is that we have not seen that consumption boom in the UK. So on this measure of consumption not only has UK household consumption not re-attained its pre-pand, the level that would be implied by continuation of the trend pre-pandemic it hasn't yet achieved the level that we saw prior to the pandemic. And of course the reason for that is one that has been at the centre of lots of the discussions we've had over the last few days. Namely that we saw a very big and significant shock to energy prices in particular to European natural gas prices on which the UK energy system is particularly dependent following from the invasion of Ukraine. And as I think this chart shows that shock was both pretty unprecedented in terms of magnitude but also introduced a very heightened level of volatility into the system which of course makes forecasting in general and forecasting inflation in particular particularly challenging. So for the UK reflecting it is a net importer of energy like the Euro area. This was not only a big cost push shock to inflation but it also implied a big deterioration in the terms of trade. That is the basic reason why we saw this squeeze on national income reflected in lower levels of consumption. And as President Lagarde said yesterday I think it does threaten even as those direct and indirect cost push effects on inflation dissipate to propagate into second round effects as various actors in the economy attempt to catch up in terms of raising their real income after the consequences of the shock to energy prices and the deterioration in the terms of trade. So of course from a forecasting point of view the issue is it was clear to me on joining the bank that the shock was not anticipated but there is a question about whether it was unanticipatable. And I took some solace from but not much comfort I have to say from what I heard yesterday from Ida in the discussion around energy prices that energy prices are both difficult to forecast and likely to remain at least as if not more difficult to forecast in the future. We have tried as you can see in this picture assumptions based on both futures prices and random walk and neither has worked very well over the last year. So if we couldn't really forecast that shock and in some sense it wouldn't have been a shock if we could have forecast it I think the challenge is more do we have as Giancarlo highlighted in the previous session do we have a way of thinking about how that shock is going to propagate which is meaningful. And I think that sort of gets to my second point. So the first thing I want to say in that context is as you can see here the forecast made by the NPC are conditional so we are assuming some things and making forecasts on that basis and that is an important source of why the forecasts may be poor predictors of the outlook even if they are nonetheless useful bases for discussion in the policy committee. So some of the things we condition on are the market implied path for our own policy rate and I know Philip wants to talk about that in the discussion so I'll hold my fire there. But we also produce forecast conditional on announced fiscal policy and on the assumed path which is very through time on energy prices. So of course what I want to bring out is not just the sensitivity of the forecast to those assumptions but also how important is to understand the interaction among those assumptions. And I think the forecast we made last summer is quite instructive in that respect especially in the light of the fiscal discussion in the previous discussion this session and Christine's comments. So you might have thought that last August assuming a random walk for energy prices as it's shown in the yellow line here sort of locked in energy prices at a very high level and implausible level and that might have given you some cause for concern. You might also have thought that at a time when we were going through political leadership changes in the UK and there was no announced fiscal response to the rise in energy prices. That it was implausible to assume that that would be the fiscal response at a time when as we've seen most European countries other European countries were introducing substantial fiscal interventions. But what I think was particularly unlikely is that you would have the combination of energy prices being at this very high level forever and yet they're never being a fiscal response. So the point I'm trying to bring out is it's the combination or interaction of the assumptions that matters. It's not just one or the other. And I think that reflects the fact that as inflation moves substantially away from its target following such a big shock the sort of everything else equal assumption that allows us to break down the contributions to the drivers of inflation in a linear way tends to become unworkable. And I think the sort of big example of that which came up this morning and was discussed yesterday is that the likelihood of second round effects entering the inflation process following an external commodity price shock is likely to be much stronger when there is a tight labour market. So if you like the impact of the shocks is not just additive one on the other but it has an important multiplicative component which means linear models are not very successful in addressing that. So that brings us to the sort of topic of yesterday's discussion around how we deal with nonlinearities. This also came up this morning in talking about convexity in the Phillips curve and in particular the key question of whether these nonlinearities are asymmetric in character. Whether they can explain why inflation goes up fast but then also lead to the possibility that inflation gets stuck at that higher level rather than coming back down towards target and inflation demonstrates the kind of persistence that we fear. So what I'll conclude and this sort of brings me to my third point is just to sort of discuss how the bank staff is trying to address that problem. Trying to act in this humble yet nimble way that Alfred talked about facing the fact that there may be behavioural changes in how different parts of the economy set prices and respond to higher inflation. At a time when our benchmark models on our benchmark framework is based on the experience of the last quarter of a century of inflation targeting a period when inflation expectations have remained well anchored and we've seen little empirical experience of the type of propagation and inflation persistence that we're concerned about. So one of the sort of things that the bank staff have been doing so I'll make a partial list. So one of the things that's been suggested is that we draw on an empirical experience from the 1970s and 80s which is an earlier period which does show this higher level of more persistent and elevated inflation. And indeed the story about the terms of trade and the higher narrow that may involve is very standard to the layered nickel jackman type of view of that period. But of course lots of things have changed since the 1970s and 80s. The structure of the UK economy is very different and the monetary policy regime is changed. And inflation expectations respond to that as we see in the very different behaviour of long rates today than we had in the 1970s and 80s. So this is sort of something you can learn but it's not the full story. So I'm of an age now where my younger colleagues tell me well we have learned something in the 30 years since you were in graduate school when you were talking about those models. And so they emphasise the point that we should be looking at some of the more modern labour market literature coming out of the matching models. As was said earlier emphasising the ratio of vacancies to unemployment or job to job flows rather than the unemployment gap as measures of labour market tightness. I think that again is useful and we have models in that direction which are helpful. But as I often tell my younger colleagues yes we may have learnt something over the last 30 years but maybe we've learnt something about the last 30 years. And in fact the experience now is one that is more akin. That's the mirror image to their critique of using the older style layered Nicol Jackman perspective. So then they respond oh yes what we can do then is use very modern techniques which focus on the very latest data using non-linear machine learning techniques, neural networks to try and explain the Philips curve. That's all very fancy but I think it runs the risk of overfitting essentially the one observation we have in a very non-linear way and then maybe applying that non-linear model to a different domain where the conclusions could be quite questionable. So it's interesting but I'm not sure it convinces me. But what I think we do have more scope is to try and use as Alfred also said some of the big data. Silvana mentioned yesterday that we do have models that look at heterogeneity, the tank model she mentioned. I think that's a helpful way of trying to have some more richness in inflation dynamics. And I know fully it wants to talk more about big data in the discussion so I will hold off on that. But then I think finally what we also do is we do ask businesses and households themselves. Silvana summarized a lot of our work on surveys of inflation expectations but I just want to sort of pick up on what I thought was the actual paper of Alberto and Francesco looking at state dependent pricing. So just like the Bonnet de France we also ask companies about their pricing patterns. As you can see in this chart as in France the frequency of price changes has increased as we've entered this more volatile environment. We also ask firms not quite in this language whether they are state dependent or time dependent prices. And the fact that more of them say they're state dependent and the fact as shown in this chart that you see that those state dependent prices are reporting that they're raising prices more quickly than the time dependent prices. That's the kind of combination that would lead you to have the convexity in the Philips curve that I think we're inserving. The good news is and I realize I haven't given a lot of good news about forecasting in my discussion here but the good news is a little bit in line with Isabelle's comments yesterday. When we ask those state dependent prices what is their expectations for inflation next year they predict lower inflation next year than the ones that are still time dependent pricing. So it does suggest that maybe there's more symmetry in the system and perhaps that can lead us to a little bit more of a sanguine view of where things are. So let me stop there. Thank you Hugh and the next speaker is Chiara Scottie from the Dallas Fert. Hello everyone, I'm going to be the odd one out here not just because I were read but because I'm going to change a little bit the topic. In that yes I will be talking about forecasting because after all you know this panel is about lessons from recent experiences in macroeconomic forecasting. But I will be focusing more on uncertainty what it means to forecast in an uncertain economy and I will bring in the financial side to understand the macro side. And of course because I work at the Federal Reserve Bank I have to start with the usual disclaimer that these are just my views and do not represent the views of the Federal Reserve System or the Federal Reserve Bank of Dallas. I think you will all agree with me that the past 15 years have truly been a collection of rare events. If we think about you know the GFC reminded us about the importance of a stable financial system to a well functioning economy that is an economy with stable inflation and maximum employment. The recent banking stress made us ponder on the issue again and then you know there was a pandemic which was also a huge shock, surrounded by a lot of uncertainty to the point that any of the point forecast were really like not useful. And then more recently there has been really a lot of discussion about potential risks and I think that between yesterday and today we've talked about a lot of those including the probability of a soft landing versus a deeper recession. And so in this presentation I'm going to take you know again just trying to be the odd one out a different route which is I'm not going to tell you about the models that we use at the Fed. But I'm going to tell you I'm going to use some of my research to think about some of the issues that I think are important for policy makers. And with the idea that I believe my role and the role of you know the researchers at the Fed is to offer models and alternative views to our policy makers to make their decisions. And so in particular I draw on lessons from a recent paper that I wrote with Dario and Molly, a couple of colleagues at the Fed to understand the links between macro and financial shocks and to study the role of uncertainty and tail risk. And I do have one slide of inflation because I understand that you guys are very interested about inflation but this is a framework that can also be used for inflation and it has been used for inflation. And I have a couple of slides about messages for policy makers so in case I will wake you up at the end. So the paper uses a model with just two sides, two variables GDP for the macro side and corporate spreads for the financial side. It's estimated on US data but it can easily be applied to other countries including the European Union and the European, the Euro area. And I think that you know even if I'm using data and estimates for the US the message should come across regardless of the country. Let me just start with a couple of basic slides so that we are all on the same page. Think about the blue line as the predictive distribution. So you're making a forecast and instead of just looking at the point forecast you have a distribution that is telling you how certain you are about your forecast. Then look at the red line at the red distribution that is a distribution with more uncertainty. So it's telling you that there is higher probability that your forecast might actually end up on the tails, on the extreme parts of the distribution. Similarly another thing that I think a lot of you have heard over the recent years is the stock of tail risk. And in particular what people in Germany have been calling shortfall and long rise. Tail risk is really giving information about the tails of the distribution and so the shortfall is the left tail of the distribution telling you bad events. And the long rise is the right tail of the distribution telling you about good events in the case of GDP. And the higher the tail risk the more extreme events you can have. Why am I talking about both uncertainty and tail risk because they don't always move symmetrically. And so it's not just the distribution getting wider but the two tails can move in different ways so that you might have very little change in the probability of good outcomes but a big change in the probability of bad outcomes. And so what are the three lessons that I want to talk about today? One is uncertainty and tail risk, f-c-r-r variation. The second one is that financial shocks not just macro shocks have big downside risk to the macro outlook. And the third one is about the fact that the effect of shocks are stronger in period of high volatility. So first one, you know how I told you if you look at the top two lines the left one is macro uncertainty, the right one is financial uncertainty. You know when I showed you those bell, those densities basically assume that at every point in time we compute the forecast one year ahead. And we take the measure of uncertainty that describes our density and we plot it through time. This is what the top is showing us. The bottom is doing a similar exercise but plotting through time our measure of shortfall and long rise. So what do we take out, take away from this? If we look at the top part we see that uncertainty spikes during recessions so you know it changes over time. It's not always constant and so you know we should pay attention when it changes. And then the bottom part shows that GDP shortfall so the negative outcomes actually move more than the positive outcomes. And so again you know they both have cyclical variation. Lesson number two, financial shocks have a big downside risk, can have a big downside risk to the economy. And so if you look at the black line that is the forecast distribution as of the fourth quarter of 2008. And then within our model we think about you know shocking the system with either a macro shock or a financial shock. And so the blue and the red line give you the predictive distribution one year out under these two different scenarios. And you can see that the financial shock is actually almost as important like it can have important consequences for the macro outlook. Third lesson, the effects of shocks can be stronger when volatility is high. And here what I'm plotting is the impact of a one standard deviation macro shock on the left or a one standard deviation financial shock on the right. Under on GDP growth, under two potential situations, one where volatility is low, the blue area and one where volatility is high, the red area. And so you can see that there can actually be significant differences in the way that GDP growth responds to a shock depending on whether volatility is low in the economy or volatility is high. So I told you that I would tell you something about inflation. My paper is about GDP and corporate credit spreads, but there are a number of papers out there, a number of frameworks out there that look at unemployment at risk, inflation at risk. And I'm just using, this doesn't want to be an academic presentation, I'm just using my paper because it was easy for me to pull those charts. But the idea is really just to show you that when we are forecasting inflation in this case, it's really different, the predictive distribution, the forecast can be very different in different moments in time. And so here in particular, I can't see from here the colors and I don't have it on the screen, but there is one line which is in 2022, March 2022, right at the beginning of the hiking cycle in the US. And then the other one is as of the end of May. And they actually look very different with, I believe it's the yellow maroon one which is more shifted to the right, highlighting the forecast for inflation that would forecast it to be higher and with higher upside risks. So lessons for policy makers. I told you that I would wake up everyone here. So this is the time. Just a couple of things that I think my model made me think about that I think should be important. The first one is really that policy makers and forecasters have to be mindful of the impact of financial shocks on the economy. And I think that we all agree about this and we were, as I said, reminded about this with the banking stress more recently. I think that so far there is probably been a disconnect between credit tightening and corporate risk premium in that normally the increase in lending standards is associated with higher risk premium and the deterioration in macro outcomes. But so far the first link has been really muted and so this also calls into question the second part of the link. But in any case I think that we should all be reminded when we are doing our macroeconomic forecast that there is a financial sector out there that can have important consequences on the macro forecast. And then the second one comes from my lesson number three where I was talking about the fact that the volatility environment can really amplify the fact that shocks can have on GDP on the economy. And so because of this I believe that it is really important to think of ways to limit financial market volatility and in this sense macro prudential tools ex ante and liquidity tools exposed should really be thought as the right answer to try to keep the volatility down because this volatility can have important consequences on the macro outlook. And I conclude here. Thank you. So that's quite intense over those four presentations but you also heard many common observations or common challenges facing forecasting. So I wanted to give the panel the opportunity to kind of respond maybe to two three sets of issues some of which you have taken up but maybe it's helpful to ask three standard questions and ask each of you to respond. One is essentially exactly in this question about the modeling of risk. For me there's two elements of that. One is how to model risk should it be to kind of try and capture the full distribution or should it be scenario analysis. And then the other side of it is how to communicate. So Hugh that was very striking with the fan chart and of course sometimes the fan chart said well it's so wide it's so wide what are we really saying. But so how to communicate the intensity of uncertainty is I think quite important. The second issue which I think came up quite a bit was we should be learning institutions. How do central banks and other organizations incorporate innovations such as the ready availability of big data. Now we also have the example in the pandemic the mobility and disease are super helpful. It's super helpful to talk to public health officials and so on. But more generally the noise to signal ratio when you go to high frequency big data is maybe worth it. So thinking about and in turn connecting to that is the role of replacing or not replacing augmenting our staff with AI and machine learning. And then the third element is I think we have the same issue. Ours are projections. We project based on assumptions but sometimes that gets lost. And indeed there is a clear issue about projecting conditional on the fiscal response and Hugh also highlighted that. But let me also give them the diversity of the panel and indeed there's many forecasters in the room also from the market side is how to incorporate monetary policy. So in our system we take the market yield curve. That doesn't necessarily correspond to what we think is going to be the policy path. Other central banks indeed have a joint reporting of that the forecast is conditional on the policy path that they expect. And also that there's pros and cons to each of those but I don't know whether now among the panel members but then when we opened up to the floor we can come to some of these issues. So let me just go in order and I'll come back to Alfred. On the first one we are doing pretty much the same as everybody else in terms of fitting distributions around our model forecast. We do that in a video by shocking a G20 model. We do that in the GFSR with a value at risk exercise. But what we also now what we did before also we had tail risk scenarios and over the last few years we added another tool. And that is alternative and plausible alternative scenario because we didn't quite know for instance how the Russia gas shock would work out. So we added a scenario which was close to where we thought the baseline was but it was not the tail risk. And I think that is hugely informative in terms of the policy making part because we are interested in using it for policy making. But it has been a communication nightmare because the press is focusing on the baseline the alternative plausible scenario. They're not very different and in the end it's difficult to communicate what we base our policy on. On your second point we have been using machine learning big data for a long time now. We use that for GDP now casting. We lose satellite imagery. We are using Google search volumes. We have a standard trade tracker which is available to everybody that was hugely helpful in the pandemic. That was also of interest in tracing Russian oil vessels. So we are doing a lot in that space. We use that information in order to inform us it's an input but it's not something what we necessarily base our forecast on. Country teams have that tool. They use that but it's not a common thing. But we are moving forward on that one in terms of creating a big data roadmap. We want to create a big data center where we are getting together with other IFIs policy makers in order to exchange more information on how to use big data for our purposes. On your last point so maybe that is of particular interest on how we are taking the monetary policy response into account. As you did Philip over the last years we did take the market yield curve in terms of describing the policy rate. We stopped doing that in April because we concluded that this market yield curve is not a plausible policy rate. This is not what we expect the ECB to do. So instead we used a model with a Taylor rule DSGE model in order to protect what we expect the ECB to do. I think that was a much better way of coming out with our forecast taking into account the monetary policy side. Why is that important? It's important because our big country forecasts are informing the rest of the universe, the 190 member countries. So if you are getting the big countries wrong you are getting 190 countries wrong and that's why we moved there. We have not moved yet on the futures curve for energy prices. That is again one of those things which people have been asking because that's what we have been taking into account and that has been a frustrating exercise as well. On the monetary side we did. I guess on the spectrum between standardised approaches and more flexible I would answer your questions in the more flexible space. I think on the risk distribution question it just depends very much in a sense what sort of shock or situation you are dealing with. The shocks we have seen recently have just been very different to what we have experienced before emirating outside the economic system if you like. In that world you are much better off using scenarios than you are doing a full fan chart of the distribution of risks because you are just in a world where a small number of very significant things will be affecting your results. If you think about the very long period in which inflation was low and we weren't experiencing those shocks it makes much more sense to do a fan chart then and think about the whole probability distribution. But when you are dominating issue is one or two particular factors I would make the case that you are better off spending your time and your intellectual effort on thinking through a couple of scenarios that can help you inform those decisions. On the big data question I distinguish between big data and machine learning which I think people are using quite effectively and increasingly from generative AI. I think those are slightly different things. On both though I would say that the noise to signal ratio is likely to rapidly change in the coming period. And so it is worth us thinking about and being open to how you can use that in particularly the generative AI where it is very early stages that macro forecasters are thinking about it. But that is likely to change. We don't use that yet. We do use machine learning. On the final question I am loath to comment about the monetary policy decision making to central banks. It is a lot harder. For a central bank it is virtually impossible to get away from the fact that the dominating thing the entire world wants to take from the forecasters is about the policy decision. It is very hard to have any sort of sophisticated discussion in which all of the media, all of commentators are just trying to extract some signals about the policy decision. So it is completely different in the sense that if you are a central bank what you assume about monetary policy and what you say about that and you have to be incredibly careful about that. It is a much easier circumstance to be honest if you are outside that for us for the IMF for others where no one is trying to interpret it in that sense and so we can be much more flexible. The important thing for central banks obviously is just to be incredibly clear in the communications what is being assumed and what isn't. Over to you. So I want to echo and emphasise I think what Claire just said. So first I am not going to say anything about anything to do with the curve or the outlook for interest rates right now. What I would say is as you know Philip we have the same process so we conditions as I said on the market path of interest rates. I think our situation is slightly different from the ECB situation in that the NPC so the policy makers own and embrace the forecast in a way that the staff is producing well. The forecast of the ECB and that makes if anything intensifies I think the sort of set of issues that Claire was referring to. Now that is not to say that we might not want to look at ways of thinking about how to reflect on our own reaction function and communicate that. But I think you probably have to place the forecast and the conditioning basis for the forecast in a broader assessment of how you are communicating about monetary policy. And it becomes crucial you have consistency across all those communication channels rather than just sort of change one piece without reflecting that more widely. On your point about reflecting uncertainties more generally again I agree a lot with what Claire has said. The history of the Bank of England has been to use the historical distribution of shocks to make some view of what the reasonable bands of uncertainty are and then make some judgmentalist adjustments to that. If we are hit by exceptional shocks and maybe we've seen a period of shocks that are exceptional once in a century type shocks. I'm not sure that that historical distribution is necessary and indeed I think that's a little bit what Gary was saying as well. So I think there is a place in that context for more scenario analysis and as I was trying to hint at is that once you move to being a long way from the steady state I think the scenarios that we have published. For example showing both random walk and full futures curve for the evolution of energy prices it becomes harder to just make that one change to a conditioning assumption because that's going to have a likely impact on other conditioning assumptions and that leads you to having to entertain I think how those are affected and ultimately I think that does make the question is monetary policy really plausibly captured by even as a sort of neutral case by the current market pricing. On big data I mean I agree with what's been said about that a lot of this is going along there's a lot of promising that I would have a little bit of a moment of caution from a central bank point of view. So if you think about what we're doing now we're trying to learn using big data a lot from kind of cross sectional information that's where a lot of the richness comes but the core issues for the monetary policy functions in central bank is really a time series dimension. We're talking about inflation we're talking about inflation at the horizon where the legs of monetary policy decisions unwind and we're talking in particular at moment about persistence inflation which is inherently a kind of slow moving time series concept. So if we are going to find value in using these kind of cross sections rich cross sections of data I think we need to have models and the Hank and Tank and other types of models that Silvana hinted towards yesterday maybe are examples of that. We need to have models that provide a connection between cross sectional variation and what implications it has for time series variation. The other thing which is perhaps more a word of caution is there is a danger I think that the richness of some of these big data sets encourage you to kind of if you lose your keys to look where under the lamppost. Right so for example it came up yesterday with John Moobow's comment about the UK mortgage market we have very rich data on the UK mortgage market loan by loan data. And so we can do very impressive calculations and modeling to me at least of what the cash flow impact of changes in mortgage rates will be and whether people are likely to refinance and are they likely to extend or repay early and those type of things. But maybe the bulk of households are not so constrained by those cash flow effects and they're already smoothing out and we may be missing some of that big picture. Maybe some of the story of transmission through the housing market in a UK where the number of households with mortgages has declined quite significantly is now through the private rental market. And because we don't have these rich big data sets there we're not looking necessarily in the right place for where the transmission is. So it's a little bit of a caution that you know monetary policy makers I think need to recognize monetary policy is a macroeconomic game. It's a medium term oriented game and it is about big macro trends and we shouldn't be too ambitious in trying to understand every wringling economy. I think I already spoke a lot about uncertainty but the couple of things that I would like to add are that we have a number of alternative scenarios that we look at and that we propose to our policy makers. And the idea is really that sometimes when there is a lot of uncertainty you have one scenario that you think is the most probable scenario but you have other scenarios that you think well they might be just as probable as this one so it's good to see what these alternative scenarios would look like because the idea is really that the scenarios are a narrative to understand how a shock would play out in the economy. And so you know under this lens I think it is really important to have these alternative scenarios and understand how uncertainty can play out in different parts of the economy. Regarding the second question about AI, ML and all of the connected things I agree with everything that has been said so far. I have one or maybe two additional things that I would like to bring up. One is that as an institution it's always important to keep in mind the risks that come with all of this and so I just want to put that on the table because clearly all of these new things are great but they also come with risks. And then the second one is more an observation as a researcher which is related to the fact that yes there is the noise that you were talking about, the noise to signal but there are also issues about the causality when you use this data and these new techniques issues with standard errors, conference bonds and everything else that is related to these new techniques. Having said that I'm the first one using textual analysis machine learning to trace out how the FOMC communications spreads through newspaper articles and Twitter so of course you do what you can and I think that there is a lot that researchers can do to use that. Third question was about the forecast and whether to use the market forecast versus the internal forecast. I think I'm coming at the question in a couple of different ways because if I'm thinking about the goal of using this forecast and the goal is really to inform policy makers, then I think that it's actually good to show everything, like the internal forecast, how different monetary policy path would influence the macroeconomic forecast as well as using what are the implications using the market path. If we think instead about releasing it to the outside so then it becomes also an issue of communication with all the issues that were already raised and I think that the Fed is in a slightly different position because the ACP is none of the above and so I'm not going to get into the issues of the ACP but I just want to recognize that I think there are pros and cons on both sides. Very good. So what I want to do is collect a number of questions. Realistically we're going to have one round so I'll take a handful of questions and then offer back to the panel. So I'm biased looking that way at the moment and I see Helene, so Helene first. Thank you very much. Great panel. So I have two points, one on machine learning and one on climate. So on the machine learning side, so a lot has been said but there are different ways of using machine learning techniques. So one has to do with big data and the problems of that and the potential also upside in terms of real-time data etc have been emphasized. But another way of using machine learning is about optimal model aggregation and allowing for data generation process which are totally unknown, updating optimally in real-time the weight that one puts on each model. So allowing essentially for very quick changes in structure in the economy while maintaining some models in order to do forecast. So this is different type of machine learning techniques which could I think maybe be used a little bit more given that we are hit by very big shocks and where the structure of the economy may be changing quickly and we should be learning about that at a learning rate which should be commensurate to those shocks. So I'm just putting that out there that's a different type of use of machine learning which may be a little bit under explored right now. The second point on climate, so I'm wondering how shouldn't we be thinking now a little bit more about how to incorporate the physical risk of climate, the horizons on climate change have been shrinking dramatically. So how do our medium-term forecast or even short-term forecast incorporate now this increased physical risks? Should we be thinking about that for monetary policy also potentially for all the analysis on fiscal sustainability? I'm a little bit worried that by underestimating this type of risk we are following policy paths which may not be as sustainable as we think and that we are therefore under investigating in mitigation and adaptation. Thank you Helene. So I see, okay, let me see, scan the room so I'll try to be comprehensive here. So please. Andreas Bilmaier, I'm Brevin Howard. I wanted to come back to the point that Alfred made about the downsides of alternative scenarios. I remember in June 2020 the OECD produced a forecast projection that had a second round of infections in COVID. I think as a consumer of official forecasts that was incredibly helpful at the time because nobody had really talked about modeling that and given the lead time going into the publication this was incredibly timely and very, very quick. So I think the point about worrying about the message being diluted is a fair one but I think there's also a messaging of thinking about a number of potential scenarios that in particular in this sort of circumstances is really quite useful for international organizations but also policy makers alike. So I understand that you run the risk of saying too much too often but occasionally I think there's a very good case for doing these alternative scenarios. Okay, so if you come over, stay in the middle here to Fernanda here. Thank you. So taking maybe the cue from Madame Lagarde and trying to look forward a little bit. You all know that forecasting is an important part of our central bank decisions. Would you say that our usual workhorse models, the one we have been using for a long time but incorporating the pandemic years, they are safe to be trusted again. As policy is being tightened across the board and unusual shocks are fading, we might be entering this more normal phase of the economy or should central banks be moving towards larger and more complex models such as the one from Bakai and Farhi. They are macro network models, they have a lot of detail and they might portray better the unusual developments that happened but they possibly might have a worse forecasting outcome under normal circumstances so I would love to hear from you. Thank you. Can you just pass the mic to Richard just in front of you here? Thank you. First a quick comment on uncertainty. I was a bit struck by Chiara's first chart that seemed to show no increase in uncertainty after the 2008-2009 financial crisis. I used to teach my students about uncertainty using the famous fan charts from the inflation report of the Bank of England. Before I flash up a fan chart from say 2007 and a fan chart from say 2011. Oh, they look pretty much the same. Wait a minute. Look at the scale on the vertical axis. Of course it had widened very considerably. A question for both Alfred and Claire. The forecasting in multinational context is subject to a, to some extent, a negotiation, discussions, shall we say, with individual country representatives. And one might think that this could introduce an upward bias in the forecast. You didn't show any charts that indicated the average, not absolute error, but the average error of forecasts. And I'm wondering whether you would find that, if you look at the forecast errors, that they have some degree of bias upwards, possibly because of the efforts of individual country representatives to say, oh, things aren't as bad as you think they are. So I think, Richard, you're referring to the GDP forecast, probably not the inflation forecast. Yes. Oh, I am indeed. I'm referring to the GDP forecast. Just pass the mic right behind you there. Okay, please. Thank you. Thank you and congratulations for this panel. I didn't hear explicitly talking about two issues. First, the role of money as a leading indicator. BIS has published recently a piece saying that had you used the money gap as an indicator, forecast errors would have been much lower. And second, expert judgment. Maybe it was implicit in this scenario analysis and so on and so forth. But could you, is no particular advocacy, what are the lessons that you take on both? Thank you. Thank you. And in my geographical sweep in this direction, Lars Fansen, say the front row for the mic. Okay. There is an inherent consistency from a policy point of view of making inflation forecasts conditional on market interest rates. The market forecast of the policy rate, especially when the MPC owns the inflation forecast, because the MPC may know that it will actually follow a different path for the policy rate than the market interest rate. This means that the inflation forecast conditional on the market interest rate is not the best inflation forecast because the market forecast is not the best forecast of the policy rate. The solution is completely obvious, namely that the MPC or the executive board agrees on a policy rate path and the inflation forecast and other forecasts are conditional on that policy rate path. And this is what already Norges Bank and the RBNC and the Riggs Bank and to some extent the Fed is already doing and it works perfectly well. So I think that is still something to be done for MPCs and executive boards to make a joint forecast for the policy rate and publish it too. Thank you, Lars. And then I'll just take John Mulvair and then Michaela at the very end. Yes, I wanted to ask the panel what their view was on the use of rational expectations or model consistent expectations, which has been a very popular idea in economics for a long time. But how do models that incorporate model consistent expectations handle the big structural breaks that we've been talking about in the last two days? Okay, very good. And then finally, if you bring the mic to Michaela in the corner at the far end. So thank you for very interesting presentations. I wanted to ask two questions. First of all, when I look at the consensus, what really strikes me is that the uncertainty shows up, if we're looking at the standard deviation, it shows up in different points of our time. So first it shows up in GDP, then it shows up in inflation, and then it shows up in uncertainty on rates. And I wonder if that's something we should think about more as a kind of early warning on uncertainty and think about how that can play out. And the second thing that really strikes me is that what we've been through is a tremendous sectorial shock. And I wonder if we shouldn't be, especially as we're looking ahead to things like climate change, biodiversity, if we shouldn't in our forecasting be placing much more emphasis on sectorial models rather than just focusing on the macro side. Thank you. Okay, thank you very much. Now many of those contributions were a mix of a question and a comment, but I will invite each panel member to see if they want to answer or respond in any way. So I'll go in reverse order this time. So Chiara. We are already in negative time, so I'll try to be brief and maybe try to find a common thread through the questions. So I think that a lot of those questions were related to really the models that have been used, how they far out during the pandemics and after that. Are they still good models? Should we look at sectoral models? I think that, I mean, we are all having this conversation today exactly because we recognize that there is room for improvement and we are all trying to figure out what are the best ways to do forecast going forward. I do think that it is important to continue to look at a different set of models and also going back to the machine learning. I think that it can definitely be useful. There is just a lot that we need to explore and I'm sure that everyone at the different central banks will continue to do that going forward. Very good questions, which I can't do justice to. I think there is a sort of interesting balance to be kept between trying to look for new techniques and new methods and by implication giving up on some old techniques and old methods in order to address the problems that Helen was talking about, the need to be nimble in the face of big structural changes. I think that also a bit addresses John's question. From a monetary policy point of view, I think there are also benefits in trying to seek out what you might call eternal verities because if they really are eternal, they are reliable and they can be quite a good basis for policy over the medium term. In the end, as I said earlier, monetary policy is a medium term business. I think finding that balance is a difficult one. It does go a little bit to the question about money. I smiled when you said you were not directing that question at any individual, given our history together. I think there is a case for seeing money as providing some view into nominal trends in the economy, which I think central banks should be concerned about. That is not necessarily a popular view with all members of the NPC. I look at my boss when I say that. I think it is something I think we should keep in mind. Just finally, and a bit related to that, I do think it is important while taking Lars's comment. I basically agree from a logical point of view with what he said, but the role of central banks and central bank forecasts is not necessarily to produce the best forecasts. The role is to support the best monetary policy decision, which brings inflation back to target. I think that does lead you to a whole set of issues about how you organise your discussion internally, which was part of what we discussed. You allow that role for judgment, which I think is necessary. That role to have many different models and many different analytical frameworks to help you have a robust and resilient view. Ultimately, how you convert that into a way of communicating with the public and financial markets. The monetary policy and, indeed, economic policy making more generally can have an effect on behaviour, which supports what you are trying to achieve. Internalising that benefit is one that is pretty key to having a framework which doesn't put accurate forecasts in some abstract sense on a pinnacle above what the real importance of the underlying process is. Thank you. Over to Claire. Great set of questions. I'll just make two comments. I guess one is picking up a couple of the questions around climate, which I think we all agree is something we all will need to think about more and factor in more. Not just the physical risk that Elaine talked about, but there's also huge uncertainties around what that will do to energy prices in terms of how the transition will flow through and the shocks there. Also, the rate of technology adoption and the cost of technology, huge uncertainties. The challenge in monetary policy space is how to, over the time horizon, the monetary policy forecast intends to be shorter time horizons than some other forecasters do. Thinking about how to do that, definitely sectorial models will help. They're definitely going to need to be part of that. The other thing, just to pick up briefly on Richard's point about, is this different if you're in an international organisation versus an authority? The data on that is we're no better or worse if you look at the errors over time, no better or worse. That doesn't mean that members aren't having an impact. As I said in my presentation, I've sat on both sides of this. I can tell you for sure countries don't agree with the forecast. Sometimes they really don't agree. It could be the case that international organisations are affected by member views, but if that's true, something else is affecting authorities because we're no worse or better at forecasters. Stepping back, in a sense, everyone doing a forecast, there's a risk there of bias, optimism bias in those forecasts. At the end of the day, they are still all done by humans, so there's a kind of human nature element to all of it, but there's no consistent error there. Just to add on robust forecasts for policymaking, that is in the end the point. We want to provide a basis for good policymaking and avoid cross errors, and therefore point forecasts are useful. A plausible alternative scenarios are required. You need to look at tail risks, and you need to be open to use many, many tools, especially in the time of structural change, including when it comes to climate change. That means you need to have expert judgement on how you put all of this together. To Richard's point, we are not negotiating our VIA forecasts with country authorities, and as you saw, the forecast errors are scattered around zero. But it might be a question is when we are doing fund programmes because they are negotiated, and therefore of course the forecast relies on the implementation of the policies which were agreed. That could be a question in the past. And the final point I want to make and should have made before, we failed or were too slow to understand the persistence of inflation on the way up. We should be careful and not overcompensate. Whenever you make errors, we are prone to overcompensate. So, on the way down, we should be open to that persistence may not exist and it may not be symmetric. Very good. So I'm not going to try and summarise the panel. I've learned a lot. I hope it's been a helpful discussion, and of course I spent a lot of time giving the ECB view, but time has run out.