 doesn't work. Okay, let's do it here. Okay, so good morning. My name is Laura Ferranti and I'm working at ECMWS and I'm working more on the sort of the operational side of the activity of ECMWS, so looking in the forecast department, so looking more really at the evaluation of the forecast. So I would like to give you today a sort of point of view of how we evaluate extend-range forecasts and try and try to assess their skill with particular attention to the ability of this forecast to simulate, represent the tropical and intertropical, tropical and estratropical interaction. So I would like to introduce really the concept of conditional skill in the extend-range forecast, the long-range forecast. We have, we know that some sources of predictability from which we justify to do extend-range forecasts and so when these sources are active, the skill is higher, when these sources are inactive, this skill is less announced. And here is really how my sort of responsibility in our team is to work and try to diagnose and improve the forecast on representing and exploit at a more effective way such a source of predictability. So ECMWS issues forecast at medium, subsistional and seasonal ranges. And if you look at the subsistional ranges, we are looking, we're trying to do prediction of events that don't represent the day-to-day variability but represent variability on the order of, let's say, week, week or five days. So events that tend to persist for about five days. And these such events are cold and warm spell or blocking events. And the source of predictability that is associated to this kind of forecast are based on the effect of the MGO, which we have heard many times during this week. And also other sources of predictability on the stand scale come from the land surface condition, the cryosphere, and the stratospheric southern warming. For the seasonal forecast, we are trying to predict and so, the variability on and so, and therefore we are trying to predict anomalies that tend to persist for longer time, typically months or seasons. And again, this kind of long range forecast relies on the source of predictability such as ENSO, global climate change, decade of variability, land surface cryosphere, and interaction with the stratosphere. So because here we are talking about tropical interaction, we focus on the MGO for the subsistional time and the ENSO on the seasonal time, but we should not forget that we are also looking at other sources. So the idea here is really the conditional scale. So we know, for example, if you look at the seasonal forecast, that if we stratify the scale of taking ENSO here or taking the whole sample, we have a different value of scale. So this is a paper from Melissa and Simo Payeva in 2008, but actually did exactly that. So they look at the scale for a three-month average over US, which are temperature forecast, and you have the scale in terms of the function of the lead month, the time, and then you see that they have actually put all the samples together, and you have the scale that is defined by the diamond somewhere in the middle of the three curves, and then if you actually state it, just say to the ENSO happy source, you have a much higher scale. So this is the concept of the sort of conditional scale, looking at the opportunity given by the ENSO. So you can do the same for the subsistional time scale, and this is actually a bit of an historical thing, because at the beginning the subsistional was something that was not actually thought that was possible, because the MGO, which is the most important mode of intracellular variability in the tropics, was actually very well defined and detected, only in the 1970s. And so at the beginning, when there was a numerical weather prediction, we were seriously focusing to improve the forecast on the extra topic, and there was this idea that the tropical didn't really matter very much. But then with the knowledge of the existence of the MGO and the importance of the interaction between tropics and extra tropics, it became clear that the idea that although the mid-latid variability is really driven by the internal dynamics, there is an important part of the tropics role that can modulate such variability. So we can have predictability in a time scale, or in a subsistional time scale. So this again is another old paper from Alejandro Natale in 2000, where we are trying to look at the same thing. So let's say, what about if we look at the skill stratified by looking at an event where there is an MGO on an event where there is not an MGO? And actually, this is a skill measure in terms of anomaly correlation over the extra tropics, so there's another MGO. And this is funny that they got exactly the opposite of what we are saying here. So they got a higher skill for the event where there was no MGO, and they got a lower skill for the event where there was an MGO. And you wonder why? And the reason why is actually that the focus at that time, they were not very good. So it was very difficult actually to represent MGO events. And so we had to put quite a bit of effort to represent the tropical variability right to get the predictability associated with the MGO. So at that time, this was the sort of the statement that was coming out from this paper. So less skill for forecast with MGO events is actually due to the inability of the model at that time to sustain the MGO. And therefore, there was no rugby way source that was creating this interconnection into the extra tropics. So there was several papers that at that point state the importance of using the other in the tropics, the importance of representing the MGO correctly. And in this way, to be able to exploit really the response in the extra tropics. So importance of reproducing the teleconnection for the focus field on the subsistional time scale. So after that, lots of work has been done to try to reproduce, to try to make models more accurate in reproducing the MGO. And so even producing the variability in the tropics in the right, more accurate as possible. And at this MWF, we measure very careful the kind of progress we do in representing the MGO event. And this is a paper from Frederick Vita in 2012, which by using a re-forecast has actually created a sort of the evolution of the skill of the MGO. So how a system progress every year we do, we apply, we implement roughly two cycles per year in which we include, include the modification and progress in representing the physical representations, we announce resolution, horizontal, and spatial, and so on. And so this is basically the evolution of the skill of the MGO as a function of the from 2002 where we started to do the subsistional process. So this red line is basically the anomaly code is the days, the skill is as a function is the time in the years, the evolution in time, and on the Y axis is the number of days, the days. So it's basically how, when, it's when the skill, which day the skill goes under 0.5, this is the anomaly correlation. And so you can see that we have actually gained quite a bit of number of days from 2002 to 2013. Associated with that, there is a sort of correspondence in looking at a similar evolution of the skill, but this time looking at the parameter in the northern hemisphere, so this is the standard. So you have an evolution. So that's where we believe is the foundation of working on the subsistional time scale. So I guess if you have already seen these pictures, I'm going quite quickly on that. So one of the, the work we are, one of the points we are trying to do then is not only assessing how much the skill, how the skill is modulated by the different source of predictability. So we are looking at conditional skills, but also trying to understand how the teleconnection in the model are reproduced. This is goes back also to all the work that Franco does. And this is an example from again in which using the SOS data set, look at the teleconnections associated with the MGO. So these are composites of geopotential height, 11-15 days left with respect to MGO events that are invasive, so there is an active phase over the Indian Ocean. And you can see the response are quite, there is a quite different phase of the response, which as a reference should be the one on top, which is the response from the era in the ring. So it's now a source of reference verification response. And it's again the MGO structure that we are looking at after. What are these numbers? These numbers are believed to be the correlation for, it's a correlation to an MGO standard pattern, just to give a sort of report if I measure what flow is. So as I said, we don't look only at, so when we do this skill, a conditional skill, we don't look only at the probabilist or topical interaction. We try to really look at all the impacts, all the sorts of stability. And in this case, this is a work from C.C.I.D.D.A.L.D. in which it actually stratifies the skill as a function of whether you have or not the stratocerics of the world. So in this case here we are looking at the condition that's still associated with the stratosphere. And you can see that for the event, this is a function different week, week one, two, and three. And this is still for two weeks of tension over two different regions, Russia and Canada. And you see that the red bar represents still for events with the stratocerics of the world and the blue without. So you see that there is an effect. So this is one of the work typically we are doing. But today I would like to talk a bit more of trying to ask, at the moment we are focused on our effort in trying to ask the questions of this tool. Can we predict weeks ahead, the changes in large-scale flow? And in particular we are interested here at predicting in advance cold spell over Europe. The idea is that cold and also warm spell in summer, they actually, they have a strong impact in society. This is an example for February 2012 where the water of Daniels were completely frozen. And so the idea is really, can we use, can we make a good use of subsistence of forecast for the society? So we have been looking at this for quite some times. And for example, we have been looking at the ability of the subsistence forecast in predicting the summer 2003. And I've been looking more generally at the ability of the forecast in predicting heat waves. And we had this impression that really the important thing is to get the transition to an anti-cyclonic flow regime right. So because both east and cold waves, they're really associated to an anti-cyclonic circulation that persists for some time. And so we are really focused really on the ability of predicting this special flow circulation that facilitates this high-impact weather. So for this we are looking at, we are trying to look at weather regimes and stuff like this. And these are the weather regimes. I think we have been looking at weather regimes quite a bit. So I'm not going to describe them. We have these four weather regimes which have been founded by many people. Franco has given a very thorough overview about this concept and the theory on it. So I'm not going to make too much comment about it. But I would just say the reasons why we're looking at this. So we're looking at these four regimes because they are associated with the high-impact events, particularly the blocking over Greenland, which is also called the negative state of the LAO, and the seminarian blocking. We have two final regimes which are certainly associated with persistent anomalies of temperature, which can be defined as a high-temperature event. We are looking at them because they are typically quite long life cycles. They tend to persist longer than a week. So they are really a typical object that can be focused in this seasonal timescale. And also because they are associated through erotic waves, they are associated with Indian in the tropics. So we have all the ingredients to be the right target for the substance of forecast. So here I show you sort of the skill of these four regimes starting in terms of anomaly correlation from the function of day from day 3 to day 30. The skill has been filtered with a running mean. And I've actually picked up, this is a based, it's a quite robust measure of the skill, because it's based on reforcelment to S, and I've picked up a number of models available on S to S. And you can see that there is some potential there. We have quite a distant skill that goes well beyond 10 days for these four regimes. There is quite a bit of range of variability among the models, but the last one is the European Central. So I think it's encouraging this kind of result. Yeah. This is anomaly. I mean, I show this, but I could show you also we have done a lot of also skill, probabilistic skills of the forecast. It gives very similar results. Yeah. The question here is that going back to the fact that, okay, the fact that you, we have a decent skill on predicting these regimes is important, but we are really interesting on understanding whether we can say something on the transition from one regime to another on the onset of this heat or cold spell. And this is a much more difficult kind of questions. And trying to address this, we have sort of created a very, very simple framework. So we are now going into two dimensional world defined by two, the first two leading UF in geopotential height. So if you calculate the first two UF in the geopotential height, you can find that the first one typically is NNEO pattern. And the second one is the kind of Scandinavian blocking, if you like. Now, this is very simple, very symmetric. And these two UF is playing together just 30% of the winter variable. So it's a very, very simplified and very limited kind of framework. But we wanted really to understand the transitions and, you know, if you are in two dimensional space, this is possible. So here is an example of our space. So here we have the UF1, and here we have the UF2. And basically, if you are familiar with the MGO index from Hendon and Wheeler, this is exactly the analogy of that index in the isotropics. So to show you if it works, we have actually projected the daily fields of two winters, two extreme winters in the North Atlantic sector. The winter 13, 14, which was a winter that was very stormy and with mild temperature. And it was actually projected most of the time into an annual positive. So we experienced that winter very strong, westerly going across Atlantic. And you can see this is a, you can actually follow the evolutions of the anomaly of this, of this winter, the daily evolution from December, January, February, this color coded looking at this diagram. And on the other side, you see the winter 2009, 2010, where instead it was a rather contrasting winter, because it was a winter in which most of the time we had a kind of a grill and blocking. So just an opposite condition. So very, very cold condition of Northern Europe. And you can see there that there was quite going between blocking and the annual negative. So it seems like that the list for these two particularly contrasting winter, the description in this very simple framework is effective. These are the two mid-attempt, two mid-attempture anomaly for the, for all the winter seasonal anomaly for 2013 and 14 and 2009 to show you the relation with the temperature. So we started to use this device for the forecaster. So now the forecaster are actually taking this kind of information here and they make quite a big use. They have grown very fond of these things. And so now here what we are looking at, example of operation of forecast. This is a medium range forecast. So these are two different forecasts. This is initiated the 3rd of May. The other one is initiated the 15th, 25th of April. And you can see if you start with this one, this is the analysis that comes before the forecast. This is the initial condition of the forecast. So it's time zero. And this is after one day, after two days, and so on. And so you can see that the cloud of the members, we are looking at 51 members here, become bigger and bigger when you go further in the forecast. And here you see the transition is going. And this is another forecast for the other case. So looking at this is very easy visually where we are going. And the forecaster seems to have quite passionate about it. It's available for member states at the moment as a testing product. But next year we are trying to make it available. Sorry? Member states. Yeah, is the MWF as a member? No, no, no, no, American. Sorry. So this actually, you can actually do, we can also do it. This is even more sort of on a research side. We can also apply this for forecaster that reach up to 46 days. In this case, this is an example of a forecast initiates on the 27th of April, which start here somewhere here. And then now we are looking, because this is an extended range forecast, we are looking every five days. So this is day zero. And the blue is day five. The light blue is day 10. And you can see how the clouds open up. Now I have to apologize. The graphic is not ideal here. Should have been smooth out to making different colors and so on. But you can appreciate that even at this range by day 15 to 20, you can actually see that the forecast is doing a very big evolution into this phase space. Actually, this is the verifying analysis. So I plotted a posteriority for this sake. And in fact, this forecast actually gave an anomaly of cold, because here we are going into negative temperature here. So there is a strong relation with the forecasting temperature, which was verified. So of course, this is a successful story is one particular forecast. It doesn't mean that we get it right all the time. But we went to see why this forecast was good. And the reason why this forecast was good is because we had an MGO. And this is quite impressive. So we had an MGO traveling around. And we believe so these are two MGO forecasts for a similar period that give you really the hint of how these two are connected. And we always, when we're trying to train a forecast to use our forecast in the substantial time scale, we always say, even if you make your outlook of Europe look at the MGO, it's very important. Because if you look at the MGO, you can have confidence to your forecast. So going back now, this was one case. Now we want to look at the evaluate the skill more in a general way. So we look again at the reef workers now, and we look at the anomaly correlation for UF1 and UF2 separately. And we can see that these are all the same different models for S2S taken from S2S. And you have the UF1 is the pattern that represents the NEO variability, the symmetric part of the NEO variability. And the UF2 is the pattern that represents the blocking. So you can see that the blocking, the predictability of the blocking, the skill is a bit of a shorter time, drop up to about 14 days. The NEO is a bit longer. But then again, stealing the ideas from the MGO guys that they have come out with a bivariate correlation. You can also do the bivariate correlation for the UF1 and UF2. And this gives us really the skill of really how good we are in performing the evolution in these two dimensional phase space. And again, so it gives us some hope that we can do, we can get some prediction of cold spell or warm spell beyond the medium range. So we can also go back now to the nice talk that Angel has given some days ago about how we can look at frequency transition. And this is exactly one of your diagram. I thought, oh, he has told them my word. So this is one of the diagram that you are doing as well. We have applied the same diagram, but in the two dimensional phase space. So here what you are looking at is trying to understand how the evolution in this very simple framework works. How is the typical, the preferred transitions. So what we have been looking at is the frequency of the transition going into blocking. So for example, if you fix it into this part of the diagram. So here I have counted the, I'm looking at about 36 years, the daily data of error in looking into transition into blocking. And so this is the column with a sort of a solid line, represent the analysis, the verification and tell you what is the percentage of frequency in which if you end up in a blocking, you are eight days, between eight and six days, you were already in the blocking. So give you the frequency of transition for persistence. So this is the persistence contribution. And this is the contribution coming from any opacity, for any negative and for getting sort of no blocking or Atlantic reach. So you can see that when the colors more or less correspond. And this is done also for different focus ranges. So if this diagram I would have shown, because we have done a different focus range, you can appreciate whether in the focus there is a bias. So if the focus is actually has a problem to reproduce this transition, you would have a very sort of inconsistent colors here in this part. But we don't find much of this. Actually we find that it's quite, I'm looking at the different ranges from 11, 16, 21 up to 30 days. And you can see that the focus actually is quite able to reproduce quite accurately the transitions divided into the different colors. I think what is extremely interesting is this part of the diagram where you're looking at transition into the annual. So this is going in this diagram going on this direction. I put to that very thick arrow, because this is a very strong statistics. So it tells you that basically most of the days that go, most of the situation in which you go into a negative is they are coming from blocking. So it's really a typical, the typical evolution into a Greenland block is a block, a Scandinavian block. And then you can study as much as you like. But I think this is really, again, is an important fact, the fact that you have to check whether we have strong biases in the model. So then you can go back and you apply again this idea of the conditional skill into the MGO, because we have seen that there are cases with the MGO. And we know also that the frequency of the NEO is connected with the MGO activity. And this is basically the Bavarian correlation between UF1 and UF2 for cases with MGO and without MGO. This is only our own forecast, because otherwise it would be too many curves to understand. And you can see that there is definitely an improvement when it's better. We have a higher skill in making, following this evolution in this two-dimensional phase space for cases where we have an MGO that is active. Then you can also, I'm also finished, when we're finished. You can also split and going to look at the symmetry between NEO positive and NEO negative. And this is what we have been doing here. Now we are looking at the skill in terms of a probabilistic measure. These are the Bavarian skills course, which measure the mean square error in probability space. And it's a skills course, which means that if you go up to 1, you have a transfer model. If you go to 0, the climatological forecast is better than your forecast. So you want to be above 0. And again, these are the skill with and without MGO. The skill with MGO is the red line, without is the black line. And then the sort of almost parallel curve over there, parallel to the x axis is actually the resolution. The reliability skills course, which gives you a measure of how the probability of the forecast match with the frequency of occurrence of the event. And so you can see that the difference between these two curves is not very large. And in fact, it is not even significant. So what the statement we can say here is that for the NEO positive prediction, the skill sensitivity to MGO is small and not significant. But when you look actually, you do the same things for the negative NEO, you actually find a quite significant skill. And this is quite impressive actually. And it has a skill in the Brazky's course and also in the reliability component. So the forecast is significantly more reliable. So at the beginning I was scratching my head and I thought, why we have such a symmetry? But this actually seems to be fit very well with the recent work that Helene has done, which shows that there is an asymmetry actually in the interconnection between NEO positive and NEO negative. And the interconnection with the NEO negative are much stronger. It seems to fit with these results. And so I think here is the summary. So forecast scale as subsistional and seasonal range is conditioned by the main modes of tropical variability and so on MGO. So when we assess the model, we need to make sure that the variability of these important modes is very well represented. And also we need to make sure that forecast subsistional and seasonal scale are quite accurate in reproduced teleconnections. Then just say words that the SOS archive is extremely valuable tool to assess predictability. I'll show you a bit the potential of these two-dimensional diagram which is very simple, very limited, but it has quite, it facilitates understanding transitions among some important flow patterns. I'll show you that we have this asymmetry. We have found this asymmetry in the skills course sensitivity with MGO. So where we found the most highest sensitivity for the NEO negative and not significant sensitivity for NEO positive.