 You probably already saw on the program. But tonight we have the reception in the area here. So I think we've got it down for 6. There'll be some drinks and snacks and food from 6 onwards tonight. So we can all get to know each other a little bit more, a little bit of a icebreaker. Why I want to do now for the next hour is, I'm going to, almost take a step back, this is going to be a very basic introduction to forecast systems and their set up. So, in a way it's almost a kind of, I do say a pre-introduction to some of the things that Frederick was introducing this morning. Mae'r bwysig o'r adroddau yn rhan o'r rhan o'r cyffredinol. Mae'r bwysig o'r adroddau yn gwneud o'r gweithio'r dda. Yn mynd i'n cael ei fod y ffordd o'r ysgrifennu i'w ddweud i'r hyn o'r masygar yn ddweud i'r cyffredinol o'r ddod i'r pwysig o'r pwysig. Felly mae'n holl o'r ddweud i ddweud o'r ddataeth. Mae hynny'n gwneud o'r cyffredinol o'r syddai'r systemau. siwydd y gallwn gweld o hyncyffwys, ond y llai, mae'n cael ei ffix, mae'n cael ei ffix, mae'n cael bod yn cael bod yn gweithio'r cael dŵr. Felly dyna'r parwysydd gwaith yw'n cael ei ddaeth i chi, ac mae'n sgwr i'n fawr i'r ddaeth i'r ddaeth i'r ddaeth i chi. Mae'r rhaid i chi wedi'i'r rymlu yn ysgol yn ysgol iawn, i'n gofym i'n cael y rhaid i'r rhaid i'r ddaeth i'r rhaid. I'm a native speaker, as you probably noticed, and I sometimes speak too quickly. I will try and keep slow and clear. If I get too fast because I get excited sometimes, as you saw this morning, put your hand up and slow me down. I tell my diploma students to do the same because they're used to me now. Sometimes, don't be afraid. Tom King's, calm, deep breath, slow down. I already talked about some of our programmes, but I've left the slide in so it will be on there for reference as well for shortcuts about some of our programmes and some of the things that we do. Frederick already showed a plot like this this morning, but I'm going to start there again to emphasise where we are. This is one of these. If you Google, you find 100 different million versions of this kind of charts. They use them in all sorts of fields. You find them for ecology. It's amazing. Evolutionary plant studies have them. They run from paleo, climate and so on. This is one that's applicable for our field for metrology. We have a spatial scale across the X and a time scale across the Y. Essentially, what the chart is showing is different kinds of... The colours are not showing up very well here, but different kinds of basically phenomena on the time scale in which they occur. Some of the things I was working on in my own early days were the dynamics of thunderstorms and so on. The point we have to remember, and Frederick already mentioned that in passing quickly this morning, is that all of these with their time scales are associated with a time scale of predictability as well. For example, for evolution of thunderstorms, we're really talking about short time scales and now casting. When I'm on my bike like this morning and I want to cycle home, what do I do? I look at... It doesn't rain so often in Triestas where I come from in England, but I look at the radar and I use my brain to make a now cast forecast to extrapolate the rain to see when exactly I should leave my office to make sure I don't get wet. That's a forecast. You can also purchase software that does it for you, takes the images, moves them forward in time and works out when they're going to pass over where you are. That's now casting. Of course, if we start to look at the forecasting systems like ESMWF for the next few days ahead, we're not going to be able to predict where the thunderstorms will occur, even if we have the statistics correct. If you look at the ESMWF forecast for three days ahead, you can tell if there's going to be a chance of thunderstorms, but we don't know if they're going to arrive here or up the road in Montfacony. Because then all you have is the smaller scale processes essentially are, shall we say, stochastic and you can only predict some of these statistics in the envelope. If you go up to longer times on the days, you're basically now the larger scales such as fronts. Three days ahead, as Frederick showed very nicely, we can predict the passage of fronts but the smaller scale phenomenon, we're only really able to predict the statistics. What is this workshop about? This is these next two tranches, so basically sub-season out to this interface here. Frederick was talking about some of these phenomenon this morning where basically on the month to multiple months timescale, then you can predict larger scale, for example, evolution of planetary waves like in the tropics of Madden-Julian oscillation, so large scale waves, but some of the things such as cyclones, you're not going to be able to predict their precise position and evolution and genesis on those longer monthly timescales. So that's again why he was picking up these phenomena. So during this week, we are going to have lectures, for example, from Frederick Cicharski introducing the basics of planetary waves. We were going to have lectures on the MGO and moving out towards ENSO. Because, as Frederick said, we're really concerned with the sub-seasonal out to the borderline season, the first season. And then, of course, we moved beyond that to the decadal initialised prediction, which was the subject of last week's workshop. So if you're interested in that, you're in the wrong week. You arrived seven days too late. So, as I said, that was just to emphasise what the target of the workshop is. So the things I'm going to very give you a basic overview of are the things that we need to consider. So ensembles, why we need ensembles, initialisation and hindcasts. And there's going to be more detailed lectures during the week, so this is only an introduction. So Frederick, for example, is going to talk about forecast initialisation in more detail. So that's just an overview again of the aim of this course. So to introduce a sub-seasonal phenomenon that's in week one, such as the MGO, give an overview of the NWP systems that we're going to be using to emphasise the use of the new S2S database, how to look at the web interface. I think we'll show this to you briefly in a demonstration this afternoon. And then tomorrow in the lab we'll be looking at this. And then with Powler especially in the labs, we're going to be showing you how to retrieve with the Python scripts. So this enables you to do it from the command line. It gives you a lot more power, but it is a little bit more complicated. Most of you are familiar with using a Linux interface, maybe I think, no. For those of you who haven't had that much experience, don't worry, we will go over slowly and we'll have the chance to actually perhaps have some small breakout groups for those that need a little bit more instruction on that. Powler is going to introduce the database, as I said, and we'll have a little bit of introduction to the reanalysis, an error interim in the afternoon. And then week two is the application. So now I'm going to talk about in particular uncertainty and how that relates to the way these systems are set up. So as I said, I'm starting from something really basic. So NWP systems, they're basically made up, underlying these systems are a set of fundamental equations. So we have the equations of motion, equations of state, thermodynamic mass balance and water balance equations. So it's important to realise that these underlying equations are basically derived from first principles and they're well known. And we solve those equations on a grid. So when we talk about model resolution, what we're essentially doing is dividing the atmosphere up into a series of boxes. So we can imagine this room, we want to model the atmosphere in this room. We divide it up into a grid, a grille. And in each box we assume that the atmosphere is essentially a continuum. And so we have just one number that describes the properties of the atmosphere in that box, such as the temperature, the water vapour, the velocity and so on. So the key thing to realise is that in each box these properties are considered to be basically uniform. We have neglect the sub-grid scale variability in most models. That's not true for all models. One of the works I was doing in the ICAM model was to also introduce prognostic predictive equations for higher order moments, such as the variability of the water vapour within the boxes. And these equations are integrated numerically forwards in time. Step by step over a time step. So, for example, the ECMWF system now, the highest resolution, is probably using around a 10-minute time step, is that right? So it's 10-minute time step. So basically you step forward 10 minutes in time, integrating the thermodynamic and dynamic equations. So how large are these boxes? Well, it depends on your problem. Of course, if you have a global model, then you have to have a lot more boxes to cover your world compared to a small-scale regional model. It depends on your problem. So if you're only forecasting for the next three days or you're forecasting for the next 10 years, again, that takes computer part-time on your need to change your grid box size. When we start to talk about sub-olds, if you want to have many forecasts, again, it's going to affect the affordability. So these were some of the choices that Frederick was alluding to earlier that we're going to talk in a little bit more detail about. So just to give you some examples from the two extremes, this on the top is showing you the progress of the climate model resolution at the various IPCC reports, starting from the first report right through to this is the fourth report. I haven't got the fifth report graphic here, but in fact, the resolution between AR4 and AR5 wasn't that much different. There was a small improvement. It tended to be that AR5 increased the ensemble size. So you can see, for example, in Europe, we're starting to resolve the difference between France and Britain. That's Frederick's from France. We can make lots of jokes about the English Channel and so on there. Okay. And if we then go to the NWP end, so this is kind of the recent upgrade about, I think, four years ago at ECMWF, moved from T799 to 1279 over here. And this is showing how basically the state-of-the-art centre like ECMWF resolves the topography. So most of the S2S systems are kind of sitting in between these ranges, as we said. So Frederick had a table of the resolutions. Of course his S2S is here. The finest S2S is bordering on this kind of resolution on the left. Okay. So what's the issue concerning this finite grid scale? Well, of course, as we said, in each of these boxes, everything's considered uniform. So if we have a box size that's on the order of 50 or 100 kilometres wide, you're not resolving any of the fluctuations that occur on smaller scales. Okay. But of course we know that the weather that we experience from day to day, thunderstorms and so on, is essentially on scales which are much smaller. So a typical deep thunderstorm updraft span is, I mean, what's the typical size of a deep convective updraft? Mm? Yes, exactly. So on the order of magnitude of a kilometre or so. So you've got these fast updrafts on the order of magnitude of a kilometre. If you really want to resolve the process as well, you have to, for example, the mixing into the updrafts, you need to have resolutions even much finer than that. Otherwise, all of your mixing between the updrafts and the environment is going to be handled anyway by subgrid scale schemes for diffusion and diffusion mixing. Okay. So if you can't afford that, you have to represent these kinds of scales of motions in convection, cloud micro-physics processes, and the interaction with radiation by essentially what's called parameterisations. So these are small simplified models that take the information that you know on the large scale, so it might be the vertical stability profile, and uses that to describe the statistics of the small scale that you can't represent. Okay. So a lot of my work at ESMWF was focused on developing those kind of parameterisation schemes. How can you represent the statistics of what's going on on the small scale by what you know on the larger scales? And it's very important because what happens on those smaller scales feeds back onto the larger scales. Okay. So just to summarise the kind of things that you have to parameterise, deep convection, shallow convection, cloud micro-physics, and interaction. And you're never going to get away from this. No matter what resolution, there are always going to be processes such as micro-physics that are going to occur on smaller scales. So even using one of my PhD students at the moment, we're looking at a convection organisation in the tropics with a model using two kilometre resolutions, but the way that the convection organises is extremely sensitive on the assumptions that you make concerning the sub-grid-scale mixing, so the parameterisations. Even though that's considered to be a cloud-resolving model and it's the same for the ocean. Again, you've got sub-grid-scale processes that are not represented. Okay. So why am I worried about parameterisations while they're not always derivable from theory? They may contain ad hoc assumptions, particularly to close the equation set, and these parameters might be difficult to measure from observations or to derive from theory. Okay. So the result, which I'm highlighting in red, is model uncertainty. Okay. So a lot of the uncertainty that we attribute to models is not on their fundamental equation set. The equations of motion and basically the momentum equations. Okay. The way that you solve them are our differences. The way you solve the advection equation is accuracy and its stability. There will be differences between models, but a far greater source of differences between models is often associated with the representation of the physics. So just taking the climate models, for example, cloud parameterisation was a big source of differences. This just emphasises that these are all different models that contributed to the latest IPCC report, for example. And the top line here is showing basically the net and the short and the long-wave cloud sensitivity. And then these are the clear sky feedback. So you can see there's a lot of agreement in the clear sky processes. If you look at the cloud, they're all over the place. And this is the result, not just of the cloud micro-physics, but also the convection, the turbulence processes, shallow, deep and so on. The radiation assumptions, the assumptions about cloud particle properties. Okay. So we have basically an uncertain model. So why or how is that going to impact our forecast? So let's imagine in this schematic that we have some state x of t and we're trying to predict the future at some time in the future at x t plus t. So this might be a two-week forecast. We're one of the S2S models. So if the observed state evolved in this way, we might conduct a forecast and the forecast might diverge over time from the observed state. So what are the reasons why we're going to have this divergence? It's because we have an imperfect model. Our representation of the physics is not perfect. But we're also going to have our divergence, of course, because there's uncertainty in the initial conditions. So how do we account for that uncertainty? I mean, we can try, over the long term, to improve our models and try and bring that down. You said the word. You run the ensembles. So this is why we run many, many forecasts. It's just basically a sledgehammer approach to try, it's basically the only way to really know of trying to assess our uncertainty. So rather than simply running a single forecast over the two-week or four-week or two-month period, we run a whole set of forecasts. So we might perturb our initial conditions and we might perturb our model during the actual forecast. I'm going to talk about how we do that in a second. The result of that is rather than just one forecast, we'll end up with 50 or 100, depends how big our ensemble size is, a cluster of solutions. So if we have a very tight cluster of solutions, we might think that we're quite confident about our prediction or if we have a very large range in our solutions. We might think that we're not very confident about our prediction. But how do we know what that is? Well, let's show a couple of examples first before I ask that question. So this is to try and account for both initial condition uncertainty and model uncertainty. So here's an example just from a recent Nature paper where we start with lots of perturbations and schematically and these are all different precipitation forecasts for the UK and you can overlay them and put a probability map together of where you think it's most likely to rain in a certain time in advance. This is another example back from over a decade ago, but again, it's showing basically three-day forecasts for a severe storm in the United States. I quite like this storm because I was actually there at the time and I got in one day earlier, luckily, because all the airports were closed down and the whole place was snowed off and it was very poorly predicted. I was very surprised at the chance at the time you found that basically you can look at different members and some of them had an extremely low pressure system and some of them were basically failing completely to have any kind of prediction at all of the event. So it was very, very sensitive to small perturbations in the initial condition. This is another example on a longer time scale. So this is looking at seasonal predictions on that borderline sub-seasonal to seasonal in Ethiopia and you can see again that you can have huge uncertainty between the different members. So this is just nine members out of a larger ensemble showing predictions for rainfall over Ethiopia three months in advance. So how many of you have seen plots like this? This is a spaghetti plot of Nino. So this is the Nino 3 forecast. This was done a few months ago. This was back in August. So each of these red lines is again one model forecast to the future of the anomaly of the SSTs and the Eastern Pacific. So you can see at the time that most of the models were forecasting an amplification with a peak towards the end of the year and then scaling off. So this is a tremendous variety between these different model integrations. So some of them have the ENSO tailing off straight away. Some of them have a very strong amplification. So this is an example more close to home. So this is today's forecast of Trieste's weather this week. I thought if everybody was completely asleep by this point maybe this would wake some of you up at least because it's quite relevant for your week ahead. We have precipitation. In the middle we have wind speed. On the bottom we have the two meter temperature. So the boxes and whiskers are showing again the uncertainty, their statistics, about the ensemble for the prediction ahead. So you can see that as the lead time this is the forecast was made yesterday on Sunday. As we go to the right time is increasing so it's looking further and further ahead into the future. So you see what you would expect that the box whisker size that shows you the uncertainty so the larger boxes is going from the 25% to the 75% tile. And then the thinner boxes are going from the 10% to the 90%. They're widening as the uncertainty increases the further ahead you go. So you can see that we have a borah predicted for tomorrow. It's going to get pretty windy. It calms off and then it gets even stronger so Thursday, Friday. So you're going to have an experience of a borah. Those of you that haven't experienced it before this is going to be about medium strength. I'm scaling up here when I say medium strength because the wind speed here says 8 but we've got box sizes which even though this is a world leading centre on the order of 16km box sizes I know from experience that when you downscale this there's a very fine topography over to Ester and I've got to scale this up by a factor of 2-3 to get the true wind speed here. So even then you have to do some post-processing which is something else we're going to talk about during the week post-calibration which I do in my head. The same is with the temperature. Topography is very complicated. You only have to go in the winter just up to Al Pacino two or three kilometres away and it can be 7 degrees warmer at night because again of the topography holding the cold air over the lip and here we're down by the sea. So again it's this issue of resolution and not being able to represent those small scale fluctuations so I know that this thing tends to be by so the temperature at the bottom here due to that effect to the cold side. Even though this is adjusted to station height for the location I'm using a lapse rate adjustment. Question? So the deterministic let me answer that a little bit later because I come back to this slide later when I've introduced the different systems at the different ranges because if I start to answer the difference between these now it might confuse some people. I have a slide in a minute where I show the different range systems and then I come back to this so let me come back to that question in a moment. So how do we actually introduce these perturbations? So initial conditions we could do a variety of things. We could just apply random perturbations we can target them we can even have ensemble data assimilation so what's the difference between those? Well if we look at just short range for example the median range the idea there is one looks at a window normally a range of around two days it's still two days now the targeting for the singular vectors so 48 hours and that 48 hour window one assumes that the evolution of the dynamics is approximately linear and then one uses a targeted technique based on singular vectors to try and work out where your perturbations will lead to the most sensitive growth and divergence between the forecasts so if you're building an ensemble and you only have 50 shots you don't want all of those 50 forecasts to look exactly the same maybe they might be very similar because the atmospheric state is very predictable but it might just be that you haven't found the locations where it's very sensitive for error growth because at the end of the day butterflies have a very bad reputation we all seem to think that they're destroying half the world through their flapping of their wings but that doesn't happen the butterfly really has to be in the wrong place at the wrong time to cause carnage through tropical cyclones so the atmosphere will be sensitive in particular locations depending on the stability and in others that butterfly can flap as much as it likes those perturbations will just dissipate away so you can try and target that's what we do at ESMWF that's what ESMWF does it targets those to try and work out to have the atmosphere to have the fastest growth now when you go to longer time scales we don't really know how to do that yet so when you look at seasonal forecasts we don't really know yet and it's one of the areas that's under active research how to do the equivalent for a one or a three month forecast so for S2S systems or so on or a seasonal forecast system tends to be the perturbations that are just putting randomly well that's fine anyway we still have the growth do you really want to target it's a much more open research question of how you perturb your ensemble to have the right spread over longer time scales and ESMWF now have introduced the system as well where they have the assemble data assimilation what does that mean well if we're trying to I'm gonna talk about data assimilation again in a very introductory talk after lunch but if you want to assimilate information okay and you have some sources of information to be quite sparse you have an observational error attached to different kinds of observations whether they're station data or satellite so your data assimilation answer it's not just one correct answer you can have a whole set of shall we say a simulated pictures of the atmosphere that are consistent with the observations that you have so you might want to try and incorporate that by actually having an ensemble of assimilations okay so ESMWF now have an ensemble technique which I think was that operational last year no two years ago now time flies so these are all ways of actually having different perturbations so these are quite different this is really to account for your lack of precise information about the observations whereas this is to try and in fact that you want to really sample the entire space perturbations to model physics well we can change the parameter settings if you have for example a convection scheme you might want to just change one parameter in that scheme so it might be the entrainment rate so you might run five forecasts that have all slightly different parameters for the entrainment rate so climateprediction.net that's the other extreme for climate prediction that's how they created their ensemble they had each climate forecast model was basically using a different set of parameters for the different parameterization schemes so the actual model was the same it's just the actual parameters all you might want to use stochastic physics so at its very simplest level stochastic physics merely takes the output from the parameterizations, the convection scheme the radiation scheme and multiplies it by a random number on a patchwork just simply multiplies it by a random number between a half and one and a half your radiation says I should be cooling one k per day you multiply it by a random number it becomes one and a half k per day over a patch for a certain period of time again there's a lot of research into ways of actually increasing the complexity and should we say justification you do this or you can combine both of these and actually introduce multi-model systems so you have multiple centres of multiple models and you combine them now I've put this in red because sometimes when people talk about multi-models especially for the longer time scales especially for climate but also for the seasonal prediction often people interpret differences between multi-model systems as being the uncertainty due to the model physics that happens in climate a lot you compare 10 different CMIT5 models and you say this is the uncertainty they're different this is the uncertainty due to the model physics it's also the initialization because all of these systems have different systems and different ways of setting up their initial conditions different data assimilation systems so you have to remember that when you look at systems it's accounting for both initial condition uncertainty but also model physics not just the model physics so it's important to emphasise so on the top I showed you an example of the basically the ENSO plumes just using the UCNWF system on the bottom here this is a much thicker plume because this now contains four systems it's from the EuroSIP basically multi-model system ok so here I've basically listed and we're going to talk in a little bit more detail about some of the databases out there which have basically all been put together over the last, should you say one to five years so in a way I would say that the NWP and sub-seasonal and seasonal prediction has lagged a little bit behind the climate change problem in a way but the databases for the CMIT process have been around for quite a long time now and with CMIT 5 there was a big effort with CMOR to bring them into a standard format to make them much easier to use now we're catching up in the atmosphere so for example for the short to medium range there's TIGI that's been around for a number of years now based at UCNWF that Frederick mentioned already ok so in the final four casts there's EuroSIP which is a combination of four modelling systems ESMWF, Meta France, Met Office and NSEP recently joined them at the moment it's only available online as graphical products but I believe under Copernicus it will become open source with digital products as well so the reason why I've included this is though it's not digitally open at the moment it will be in the near future so watch this space there's a funded Copernicus program which is the climate services side is actually based at ESMWF in Reading there's also the North American multi-model ensemble, NMME how many of you have heard of this or used this just out of interest so there's quite a few six or seven of you so again this is focused on the seasonal forecasting timescale and I have a slide in the moment showing some of the centres that contribute to that and then there's CHFP how many of you have heard of CHFP those are your IRI involved to less, okay that's interesting so that's something that's come out of one of the working groups WG SIP that's focusing on the basically sub-seasonal to inter-annual predictability and this is a database which actually tries to accumulate the hindcast suite which I'm going to talk about in a moment in one location so one can document how seasonal forecasting systems have improved over time it's supposed to be a long-term archive and then here in the middle as I said we have the sub-seasonal so the S2S database at ESMWF which is the one we're really going to focus on that's just come out this year so NMME I think went operational in 2014 CHFP has been around for a couple of years I said EUROSIP is not online yet so this is all really in the last few years I think TIGI was perhaps a bit long how long is TIGI going around for now four or five or yeah about five five years, okay oh 2005, okay a bit longer then so that's been around for research and then there is actually plans to actually supplement the NMME with additional higher resolution forecasts on the sub-seasonal timescale as well which I believe is penciled in for 2016 as well so there will be a second database that complements the S2S database at ESMWF which is focused more on the American systems that's coming up in 2016 okay now I hope I haven't forgotten anything from that list that I should have included feel free to interject if I have okay so it's quite exciting at the moment because we really in the last couple of years we've got all of these sources of information where we can look at how these systems are working, the predictability just weren't there just a few years ago essentially so this is an example from the NMME from Ben Kirkman's paper that came out just last year and BAMS showing the ENSO plumes and the red line is how the model ensemble as a whole is doing compared to the black line which is the observations and these are the modelling systems that contribute to that just to mark the IRI we've got some people from IRI of course that are helping us out here their system have got a star because it's no longer operational and that's one of the things about these databases is do you have it that's purely operational groups or do you open it to a wider community of research models as well so the slight difference in strategy between for example the S2S database and perhaps the NMME and the others is that this has a mix of operational and research models whereas the S2S is purely operational centres as you saw which does mean that you've got perhaps a wider ensemble here but you have systems that might not be sustained it would change over time so for example these two systems now are no longer operational ok so if we go back to my simple schematic of this forecast we have this cluster of solutions so we could look at a root mean square error difference between all of these different forecasts and we refer to that usually as the ensemble spread so it's not what you put on your toast at breakfast time it is basically a root mean squared error difference ok between all of these forecasts now how big should your spread be remember your spread is telling you your confidence so if you have a very small spread your forecasts are very similar so maybe you're very confident tomorrow I think it's going to be 10 degrees plus or minus a half maybe the next day your forecast will be after is 10 degrees plus or minus 5 so you're not very confident but if you set up your ensemble how big should this spread be rhetorical question maybe but what should it be roughly equal to should be less does everybody agree with him I would say you don't want it less than the error what do you think I mean I would say I kind of agree with you but I think it should be on the same order of magnitude as the error so I mean essentially again you want roughly the spread of the ensemble is roughly the same as your your RMS error so if this is my observation you want that observation to kind of be in that cluster ok so if your observed state always lies outside your cluster that means that you think your modelling system is much better than it is so you say ok I think in 3 weeks time it's going to be 10 plus or minus 2 or 15 so you don't want your observed state to be out here you don't want the spread to be less than the RMS because that means you have an overconfident system ok likewise if your observed state is always near the ensemble mean and you've got this huge spread that means that you're perturbing your initial condition perturbations and your model physics perturbations are agitating the system too much ok ok so then you have an underconfident system you're giving people error bars that are too large you're saying well more or less I think it's 10 plus or minus 5 degrees tomorrow but in fact your system is always within 0.5 or so ok so you don't want to be doing that either because you're basically throwing information away ok so you want your spread to be roughly of the same order of magnitude and that's your RMS error now that sounds simple but it's not as simple as it seems why not say again well if you have an NWP system the nice thing about S2S time scales and so on is that you can look at your handcast which I'm going to go into a moment over the past and you can evaluate your model so why is this not as simple as it seems because you could say ok I'm looking at tomorrow's weather and say my spread is too large always well I've just tuned down my perturbations making my perturbations a bit smaller and then not perturb the model and I reduce the spread until I get it just right everyone's happy yes but why is it not as simple as that ok so very good so you can't do this just for one particular day you have to average it over the statistics of a lot of forecast which may be regime dependent so you might find in certain regimes you're always over confident and others you're under confident also your spread behaviour will be a function of lead time location and variable ok so you might be looking at for example now that I'm going to show an example in a moment where the Met Office I've been trying to predict ENSO for the season ahead at lead times of 3-4 months or even longer when the system is under confident but they may be able to adjust that but they might find then that their predictions for other variables are over confident so it's not that you can just tune the system for one particular kind of forecast it depends on what variable you're focused on what your location is so one of the I should have put a slide in to demonstrate it and now these days it's improved but when I was at ESOM WF 10 years ago one of the key problems was the spread in the mid-latitudes looked nice in the medium range out to the month so they had the forecast ensemble nicely spread out but in the tropics the system was over confident and it was very difficult to make changes to the system that made the tropics forecasts diverge enough to account for the error without having the mid-latitudes spread out too quickly so you had this disparity between behaviour in the tropics and the mid-latitudes so it's not that simple it sounds like a very simple thing to fix and tune but it really depends on your lead time on your area that you're focusing on and so on so this was that example I wanted to show from the Met Office where essentially they're looking at ensemble skill at predicting the NAO and this is this black line here and this is just of the number of members in the ensemble but don't worry about that for the moment the thing I wanted to show this was shown in a talk last week by Doug Smith was when they actually looked at the model predicting itself so that means if they pretend that one of the 20 members is actually the truth and they look at how well the other members predict that member they find that the prediction is much less skillful so their interpretation he said was well they were thinking maybe that's a sign that the system is basically um underconfident and again I was surprised that they hadn't gone away and just looked at this kind of diagnostic just simply compared the RMS error against the ensemble spread which again I think is why it's important to emphasise this they hadn't done that but I think this is just another way of showing that their ensemble for that particular metric is actually underconfident there's too much variation between the ensemble members so we're going to talk a little bit about how one uses the forecasts and the variability so a very simple example we imagine we start from our perturbations initial conditions and we look at basically our different forecasts for a certain future time we might be looking at a certain event so I've just got Ray no Ryan to keep it simple for S2S next week we're going to be looking at for example prediction of drought occurrence or flood and extreme flood so the time range is basically should we say generic here how would we actually validate this so if I say tomorrow I think so 70% of my forecasts say it's going to rain and 30% say it's not going to rain if it rains is that a good forecast if I say tomorrow 70% chance of rain how many of you bring your umbrella quite a few of you that's because the cost of bringing the umbrella is basically zero it's a little bit of weight now if you had to go and buy an umbrella that starts to change it a little bit you've got to buy an umbrella for 50 euros how many of you would buy an umbrella for 50 euros if I gave you a 70% chance of rain forecast no not many because your loss in that case is quite small you get a little bit wet so most of you would rather keep the 50 euros and spend it on something else so this is why so how would we actually validate that if it doesn't rain tomorrow how many of you would be crossed with me I say it's 70% chance of rain and it doesn't rain am I right or am I wrong how many of you think I'm right nobody on one thank you my friend in the front row you're wrong nobody you're all very shy well I just gave you a probability so I mean I can't be wrong as long as I don't say zero I can hedge my bets I could just say every day 50% chance of rain and I'm always going to be right but it's not very useful is it so how do you test that system well basically you have to look at many of my forecasts so you say okay let's look at Tonkin's forecast forecast's over the last year let's look at all of the times he said 70% chance of rain how often should it have rained 70% of the time okay if it rained all of the time I've been under confident it comes back to that cluster of points okay means I've been under confident so when we talk about how to validate this we're not going to do just thinking about ensemble means this week we have to start to also think probabilistically and that's important when you get along beyond the deterministic range okay so I know it's a little bit of a silly example in a way because it's very simplistic but it's important to start thinking about that when we look at these products okay on the longer ranges okay so I've got a few more slides I'm in danger of overrunning here to start to introduce some of these systems so I just wanted to put a little bit of framework into place so I'm going to show the ECMWF system and the Met Office system not all of the systems because it would be tedious and I'm not so familiar anyway with the other systems if you're interested we can find out this week about all the nitty gritty and so on so at the top end we have this deterministic run which is the highest resolution system and we have a single forecast okay now ECMWF for quite a long while has been supplementing that they started with an ensemble run that was running out for 15 days and that was many different forecasts to try and account for this model and initial condition uncertainty by perturbing the initial conditions and perturbing the model physics then basically almost two decades ago they started to supplement that with a system which was basically looking at seasonal forecasting time scales okay so we have the highest resolution the ensemble was lower resolution because you're running 50 or 51 of these things so it's a lot of computing power so they have to run at lower resolution the seasonal forecast was running at even lower resolution because this thing is running out for many months in advance okay some of a subset of those were extended is this still four times a year now yeah so it's out to 13 months okay so they run each month for seven months but four times a year they run out for 13 months ahead but you can see we've got a little bit of a gap here okay that's what that bridge in the gap is all about that Frederick was talking about this morning so it started off Frederick was basically if I understand correctly you pretty much put the whole thing together yourself back in 2003 so Frederick when he had a spare moment here and a spare moment there I remember at the time it was like on top of everything else he was doing started to put this system together which was basically sitting at an intermediate resolution between these two and at that time the research mode basically run once per week for 32 days if I get any of this wrong you step in and tell me yeah so it was running 32 days and it was run the system once per week so it had this like burst ensemble okay once that went operational there was a decision to basically try and bring these together a little bit in a similar way that mimics what's done with the seasonal system okay so the ensemble now is essentially extended out to 48 days not 32 days okay and it's actually operated now twice per week okay so you essentially have this and it's split resolutions so you have a higher resolution at first and it switches to a lower resolution for the extended run okay one of the key differences and I'm going to talk about this and in a couple of slides time is the hind casts I'm going to talk about why we need hind casts in a moment first so just a flag there's a big difference between these systems in the fact that here we have dynamic hind casts here we have fixed hind casts so I'm going to come back to that in a moment I just wanted to flag it now and both of these systems at the moment have 51 members okay so this is what now I wanted to come back to this forecast so this blue line is showing this very high resolution for example system run here these bars are showing the ensemble for the EPS so you can see that this lead time only shown here just basically goes out to 10 days but these barred forecasts are carrying on now twice per week right out across the blackboard to day 48 which is somewhere over there so that's how that fits in the red line is the control which essentially what does that mean it's essentially the basic forecast system without any perturbations okay now I actually personally think that if I became director of ESMWF I wouldn't do this anymore because I don't like personally the way that you have a system where you have perturbations and then you have one forecast system that doesn't have perturbations because the modelling system is not the same and that makes it very very difficult it's quite a constraint on your stochastic physics because what it means is anything you do in a stochastic sense has to have a zero mean perturbation you can't have if you try and do anything that doesn't have a zero mean perturbation it means then you start getting into all sorts of trouble because your unperturb system has a different set of physics to your perturb system and it's very difficult to keep the control inside your pdf so I said the talk was a bit basic that's sitting brackets for the slightly more into stochastic physics and perturbations don't worry if that's confusing just ignore me switch off switch back on again now okay so you can combine these two so next week if I show you the health system what we've done there for example of our health modelling system is we've actually taken and we need to upgrade this now to the 48 day we take the 32 day because we were looking when the older system once per week we take the 32 day system for month one and then we combine that supplement it for months two to four with the seasonal forecasting system calibrate and then use that to run the applications model so if we find that there's more interest for that I will show that at the end of next week okay so why would we do this one of the reasons of course is the resolution we've talked about the higher resolution and if we look what I'm showing here is just a simple correlation map of the temperatures I'm interested in Africa so I'm showing Africa on the left the right panel is showing the increase in skill of using the sub seasonal system the extended EPS for 32 days compared to the seasonal system and this is exactly like for like so what I'm doing is if the if the because of course the one thing that's difficult comparing these systems is they start at different times the seasonal system starts on the first day of each month okay and the as we said the sub seasonal system starts twice per week so I think it's Monday and Thursday yes if I remember rightly so every Monday and Thursday so I'm comparing exactly like with like if I have a forecast that starts on Tuesday the 10th of March runs for 32 days I take those exact 32 days predicted from the seasonal forecast that starts on the 1st of March so it's exactly the same days for a whole year's set of forecast and hindcast okay so where does this skill advantage come from let's see that's right you will have some variability in some areas I mean just from chance you wouldn't expect to have zero everywhere so there will be some sampling issue there and you might find that there are some aspects of the system that actually gives you detriments or so on but on the whole if you average it you have an advantage in skill yeah so where would the advantage or difference because there is actually a point no it's an important question you made because there are some things that may not necessarily be an improvement let's see how many of you have been listening so far to my basic lecture so where is this going to come from I've told you one already resolution what else do we expect to have a skill advantage when I'm looking at this is a year's worth of forecast plus all the hindcasts of the S2S system compared to the seasonal system the way you're initialising is different actually and that's one of the reasons where it could be better but sometimes you could get worse because the seasonal system actually they have a longer time before the issue of the forecast whereas the twice per week forecast you don't have the time to do for example all of the things of the ocean so Frederick would be talking about the ocean's initialised but you have to have a system that's a little bit faster so you blow winds across is that right to get the ocean perturbed state which could be worse could be better you have to look at that systematically so because of your fast lead time there could be things that are actually detrimental with the monthly system because you've got to get the forecast out quicker it's no good waiting 15 days like you do with the seasonal forecast because it's twice per week it would be too late very good any other ideas a different timescale but these are exactly the same dates I think I want to know what you're getting at so you're talking about the different start times or the different timescales because I'm looking at the same 32 days so the seasonal is the first 32 days of the seasonal so I'm not looking at different should we say I'm not looking at the whole four months of the seasonal but you are the answer is the right answer but for a different reasons I was kind of sedating to earlier it's the lead time advantage it sounds like a really stupid thing but your seasonal forecast is once per month your S2S is updated twice per week so if on the first of March you make a forecast well they both start at the same time but say I'm running operationally for my case malaria in Uganda I can continue to update it so when it gets to the 28th of March it's the most recent S2S and it's almost a whole month newer than the most recent seasonal forecasting system so it sounds a complete banality but it's something that's very important filling that gap is not really so much about the resolution the biggest advantage you've got is that you have a system that's much more frequently updated but goes out to 48 days so you have a lead time advantage so you have a lead time advantage that's your biggest key step in fact you have a framework advantage you've got the ability to have more computing power spent so higher resolution also there are other differences in initialisation and then there's a middle one which I'm going to come to now you have basically your physics now what do I mean by that and it comes back to these fixed hind cuts so I'm going to come back to this point in just a second so I have a few more slides and then I'm going to wrap up so there's just hind cuts just in case I've confused somebody why do we need hind cuts well imagine this is evolution of some variable forward in time and this is the real world so this could just be temperature and this is my forecast so maybe I have a forecast here normally it's 10 degrees I forecast 8 so that means we're 2 degrees colder than usual but of course these models have biases so this might be my model world ok so my forecast is warmer than the model world so we have to basically translate this anomaly onto the real world this is just a very simple case of bias correction of course I've actually drawn both lines identical I mean the real model world we had the real world, the model world and then the real model world is that all your variance and everything else is going to be different as well which means that bias correction as well it's not as easy as it seems we could have a whole workshop on bias correction techniques so you need to have hind casts for the past to be able to calibrate your model ok so how's that done let's introduce first of all another modelling system this is quite a complicated diagram it's from a paper that just came out this year in QJ from the Met Office system so they have a slightly different way of doing things they actually have each day four forecasts two which extend right out for seasonal timescales and two which only extend for sub-seasonal timescales and they do this every day so in a week they've got seven sets of these seven times four 28 ok and then for a month three weeks worth of these forecasts so and they combine that together to get their basically their forecast set so their S2S product uses seven days of these four forecasts to give you an ensemble of 28 days ok so the difference in that how is it different well you can see again it comes back also to this lead time ok if ECMWF has a burst of 50 forecasts on a Monday then your information for Tuesday is going to be nice you've got a lot of ensemble members ok for that Tuesday with the the Met Office forecast four of those forecasts will have been made six days earlier ok so there will be much older information but of course if you then look a couple of days later for the ECMWF if you look on the Wednesday then you won't have any information newer than two days whereas the Met Office will always have some of these which are just one day old so their system has a very continuous should we say lead in time whereas the ECMWF is a little bit like a sawtooth you know each twice per week you have a burst of information which then decays in terms of how new it is so again it's different strategies of how to set this up and they also have a dynamic hindcast suite so just to make sure I put this in just before because after Frederick talked about on the fly and fixed I just wanted to make sure that everybody understood what that meant so I just added this very quickly it's why it's a little bit of a rubbish picture that's all I could manage over coffee break this morning because I wanted to make sure that was clear so what do we mean by on the fly what it means is ok this is my forecast I'll assemble 50 members which I'm doing now today so the date today is 22nd 23rd now you can see why I arrived 20 seconds before the start so today is 23rd of November and we make a forecast so the dynamical on the fly hindcast is ok let's also make an ensemble for the same day for the last 20 years excuse me it used to be 18 and now it's extended to 20 years am I right so it's now recently changed last 20 years so you say ok let's also start for the 23rd of November 2014 23rd of November 2013 12 all the way back for the last 20 years so you don't just run the forecast now you run that every single time also for the last 20 years for exactly the same date ok so that's a kind of dynamical system by contrast you might have a fixed system that says ok we have our new system, new model let's run a whole load of hindcasts and then we'll forget about it that's my database of hindcasts so no matter where my forecast starts you might run hindcasts or the last 18 years maybe starting every single day I don't know but you run this hindcast suite just in time why would you use one or the other well it depends on the number of factors and the key one is how often are you updating your system ok now for example the seasonal forecast system is not updated as often as the others on the order of every four to five years a little bit like climate modelling systems each modelling centre tends to have a new system for each IPCC report but then they use that system for the next four or five years if your modelling system stays the same then you might go for something like this because for the next four or five years your model physics are not changing so you can run all these and then freeze them what's the advantage of doing that well because you just do it once you can perhaps have your hindcast suite with more members ok so you have a large ensemble to sample your uncertainty however if you have an NWP system that's updating quite frequently so on average it's been a little bit less often recently but ECMWF will update their system two or three times a year why, well there might be new satellites that come online that you want to incorporate into your data assimilation system there might be new model physics updates if your forecast quality and so on and you want to bring them online as quickly as possible you don't want to wait five years before you start using a satellite probably the satellite won't be there anymore in five years time they have a five year lifetime so if your system's updating all the time you don't have that much advantage to do this so you really want to basically each time you have a forecast redo those hindcasts to make sure you're always using the same modelling system there's no point having a hindcast with a different model because it'll have different bias characteristics the bad thing of course though is it makes it more expensive to run which means it hits your ensemble size if you didn't have to do this each time instead of 50 members you might be able to afford 100 members here or even more so the system used to have just five members per year which has been increased now but it used to be just five members per year so as Frederick showed you different centres have different methods a lot of the centres that perhaps update the model less frequency are using this fixed methodology rather than the dynamic it also means that the database construction was a little bit more complicated due to this disparity between the centres it made it more difficult to have a generic interface it also means when you want to look at the NAO predictability how do you do it we could take the operational forecast the system at ESMWF has been running from 2008 so if you wanted to look for a few years you could look at the inter-annual changes in the predictions using the actual operational forecast ensemble but the problem is that that model is changing all the time there are updates to the system so what one tends to do is for any particular start date is actually use the hindcast so this is year minus one, year minus two, three, four down to 20 years or beyond to look at the inter-annual variability so I just wanted to emphasise that cos that can also sometimes cause some confusion that you actually have two time axes cos you can look at the operational forecasts and how they change in time but then the model version is changing or you can look at the hindcast sweep but then you have a smaller ensemble size so the malaria work I was showing was just based on five members so also by using the hindcast sweep again you're restricted to the length of the database range and as I said this particular work we only had five members in the hindcast sweep whereas the operational system of course has 51 members as time goes on you're adding more and more operational cycles you're going forward in time so as I said it's been around since 2008 so it does make it a little bit more tricky to set up the database you'll notice for example for the NNMME that the hindcasts are all fixed and they're all roughly covering the same period which does make it easier to inter-compare these systems that's not necessary to the case for things such as S2S where you have dynamic hindcasts are moving forward in time so your oldest years are actually dropping off if you look at the system in 2015 you won't have the same set of years as the system had for 2012 so I'm going to stop there so I've overrun actually by 10 minutes apologies and as I said in the afternoon Frederick is going to show just a quick demonstration of some of the aspects of the S2S and I will be showing a similar thing in parallel for the reanalysis and so on ok, I'll stop there, thank you