 I'm very delighted to introduce Anish. Anish is an assistant professor at the Department of Atmospheric Oceanic Sciences at the University of Colorado. He also was at San Diego and in Oxford, so he has worked both in Europe and here. He has many research interests around climate prediction, data assimilation, coupled processes and privatization development. He has worked in the modeling of the Madden-Julian oscillation. I think one of the work that stands out is his studies on climate change impacts on the MDO and how the physical and improvements of the physical permiturization schemes and stochastic permiturizations can help with the representation of MDO in the tropics. Anish, I'm looking very forward to your talk and please do take the full 30 minutes. Thank you. Thank you. I'll be talking about exploring physical and a little bit towards the end of machine learning methods as well for stochastic modeling and ensemble prediction of weather and climate and here I say weather and climate which is essentially a continuum where S2S is the bridge between weather and climate. I would like to acknowledge and thank all my colleagues and collaborators who have contributed to much of the work I'll be presenting. A lot of the work on stochastic modeling I started when I was in Tim Palmer and Anjie's group at Oxford in collaboration with Fredrik, Peter Bechtal and Magdalena at ECMWF. I'll be presenting some results from Will Chapman who's a student in this school as well. It's been a real pleasure to work with Will for the past half a decade and then your colleagues at NCAR, David John, Judith whom you'll hear from later in the school and colleagues in Europe and in San Diego like Judith mentioned. So yeah I presented this slide in our introductory talk on Monday. I'll not go over all the applications and the timescales and information again but one thing I would like to emphasize in this plot which I did not on Monday was the two orange lines which are not highlighted in this figure in any way but it is really key for using our information for applications in the society. So being able to give what's known as forecast uncertainty or how uncertain or certain we are about our forecast on different timescales is really key for using that for decision making in societal applications and a lot of what I'll talk about today is how do we get this right? How do we get our uncertainty right in our forecast and what does that imply both in terms of improving our models but also improving the information we provide at different timescales and as you see in this figure the uncertainty grows in time and what we would like is our prediction systems also to reflect the same on growing the uncertainty and growing the uncertainty accurately. So just a brief outline of what I'll talk I'll be giving a lot of introduction on how uncertainty grows in our system what do we mean by ensemble prediction why do we need ensemble prediction and stochastic perturbations in ensemble prediction systems then I'll touch upon some brief results on the impact of stochastic parameterization on mean precipitation a lot of work done by RTA over the last decade and also sub seasonal focus of the MGO which is ongoing work and then towards the end I'll touch on machine learning for post-processing and stochastic parameterization. So yeah we mentioned about the Ed Lawrence's concept of sensitivity to initial conditions and the whole concept of chaos both in Joe's talk on the first day as well as in our fun debate on Tuesday. So Ed Lawrence came up with this really simple model a three variable model paper was published in 1963 called deterministic non-periodic flows. This essentially it was not the first reference to chaos there have been there has been work done on chaos before but essentially I think later light was a milestone in terms of understanding chaos in weather sciences and weather prediction. So the model is really simple it's a three variable model with non-linear interactions and three parameters it's a model for what's known as Rayleigh Bernard convection or convection of a flow fluid flow between two plates which are at different temperatures and Ed Lawrence's paper and like following work that really had an impact on ensemble prediction is this sensitivity to initial conditions if we have small perturbations in our initial conditions how do they grow and in a chaotic system they would grow non-linearly and that's essentially the concept of why we need to project uncertainty accurately because we need to capture that error growth well. So I'll demonstrate this with a simple schematic here in a minute I'll play an animation what we did was to run the Lawrence model for a long period of time and all these great dots are solutions to those three equations over a long period of time and this is just for the X and Z variable you can think of this as the climate of the system it's a climate attractor or it's the space where all solutions of the model would lie then what it did was to pick like one value of one initial condition and then perturb that 50 times so these red dots here we have 50 different dots of red color and I did this in three different locations of the attractor so one over here with a value of Z between 40 and 45 and X between minus 10 and 15 another here I initialize an ensemble here and another one here and then I'll integrate the model forward in time and we'll see what happens in terms of the ensemble how do these 50 members evolve in time all of them were initialized very close to each other with just a small Gaussian noise added and as we integrate forward in time this is about 12 model time units you see different behavior in these 50 ensemble members as we integrate them through these 12 you can think of it as 12 or 15 days of forecast in the model right if we initialize them from this part of the attractor they all stay close to each other for this time period and the ensemble spread is low if we think about this location the ensemble spread becomes large and then as we get close to this region of bifurcation between these two regions of the attractor we start seeing divergence in the ensemble much before it like closer to the initial condition and here I call them as predictable semi predictable and unpredictable because the uncertainty is really large in the in the third case and in the first case the uncertainty is low so if our true system was what I picked as the initial condition here then all the ensembles will be able to predict the true system with low uncertainty here this kind of an example can also be seen in the real world here what I've shown below is three different examples of ensemble prediction of hurricane tracks or tropical cyclone tracks one on the left here is for hurricane Haiyan which impacted Philippines and the west Pacific region western tropical Pacific the colors here indicate the probability of occurrence of the tropical cyclone track the blue and purple colors show high probability 80 80 percent of above and what we see here in the ECMWF ensemble system is all the ensemble members projected a similar track the black dark line here is what truly happened so that they all lie within the ensemble spread on the on the middle panel is the hurricane Katrina which brought a lot of destruction and death to the Gulf Coast in us here we see initially there was large probability of where it would go large certainty but then the spread became larger and the third one was hurricane Nadine where it could either go out into the Atlantic or towards Spain and have a big societal impact so these are kind of examples where we want to predict the uncertain uncertainty accurately in our forecast so what do we call a reliable ensemble forecast or what qualifies as a reliable ensemble forecast a reliable ensemble forecast should provide realistic estimate of uncertainty in our forecasts so there's just a cartoon of what we mean by a reliable ensemble here what's plotted in the blue dots you can think of it as ensemble estimates at some lead time so let's say this is a a 10 day forecast of different ensemble members for two meter temperature in Boulder so they we get different values of temperature from the different ensemble members the white dot here represents the ensemble mean of all these blue dots and the red dot here we can think of it as observations of the truth so what we want in an ensemble is that this spread in the ensemble should equal the error in the ensemble mean because the ensemble mean is the best estimate of of the forecast so if this error in our ensemble mean forecast matches the spread then we would say that the ensemble is reliable which means the uncertainty we get from the spread of the ensemble is similar to the error we have but in other words the truth or the observation cannot be distinguished from the other ensemble members so we can have unreliable forecasts and there are two types of unreliable forecasts as you might have guessed from this cartoon one of them is the under dispersive ensemble or overconfident ensemble where all the ensemble members are all close to each other so the spread is small but then the error is too large so the from this kind of a forecast we might think that the forecast has low uncertainty but in reality the error from the true observations is large so this uncertainty is not useful and it's under dispersive similarly on the right we have the over dispersive case where the uncertainty can be too large and the error is small but still given the uncertainty is large we cannot really make decisions based on this kind of an ensemble so in terms of uncertainty in our forecast system there are two kinds or two main sources of uncertainty one of them is the initial condition uncertainty that we just discussed related to the Lawrence's paper and idea of this sensitivity to initial perturbations a second kind of uncertainty is model error so this comes in from error in our subgrid scale parameterizations so in our earth system models which now is being used even for weather prediction we have a fully coupled earth system with the ocean land surface processes atmospheric processes cryosphere and all of these different processes act together to create our weather and climate over subseason to seasonal to longer time scales right we need to represent all these processes we have some kind of representation of all these processes but but almost all of them are inaccurate we make approximations and parameterizations we make approximations including in our dynamical equation so this uncertainty or approximations need to be specified in our earth system model and there are two main contributions to this uncertainty I'll quote Donald Rumsfeld here who is the US Secretary of Defense back in the 2000s he got a lot of flak I think he even won a foot in mouth prize for this he quoted that there are known knowns there are things we know that we know there are known unknowns that is to say there are things that we know we now know that we don't know but there are also unknown unknowns there are things we do not know we don't know know and he quoted this in a political situation which the press and the large-scale media did not take too kindly but then in our scientific respect we can come back to this quote and it's relevant here as well we we know that there are known unknowns and there are also unknown unknowns and I'll give a couple of examples here so in let's think about convection as one problem so if we have convection and clouds that would make some kind of a temperature perturbation through the atmospheric column so let's think of that as this blue line which is the tendency of temperature at one given time step from convection alone and then if we think of our discretized model it's impossible to represent all the complexity of convection even the broad-scale circulation of convection our current generation weather and climate models don't resolve this so we make approximations and what are known as parameterizations of convection in our numerical model and this parameterized convection can give us some estimate of this convective heating or the temperature tendency in the model from convection but this will very likely and almost always be erroneous that it would not match what the real world would get from a similar convection so there is this error and we know that there is this error so this is the known unknowns also called as epistemic errors or epistemic uncertainty and then there are these unknown unknowns which is there are processes that are not resolved in the model and here I call it as unknown unknowns because the model does not know about these processes so an example is the upscale transport of kinetic energy two larger scales I just showed this cartoon here which is an example of 2d turbulence they have a lot of eddies with noise but then they also self organize into jet-like features which is an example of upscale transport of energy into these coherent structures so both of these kind of uncertainties the known unknowns and the unknown unknowns need to be represented in our forecast system I'll be just describing examples of how this is done in the ECMWF forecasting system the two methods of emulating uncertainty in the ECMWF system the first kind which is the uncertainty for initial conditions that is currently represented using these singular vectors that Joe described on Monday in the on the first day's lecture and more recently they've also added ensemble data assimilation perturbations to represent this initial condition uncertainty and the model uncertainty is represented through two different types of stochastic parameterization this has been developed for more than two decades in ECMWF Roberto Buica's paper from 1999 was one of the first describing such an implementation of stochastic parameterization Tim Palmer has a paper in 2009 which is a really good review of stochastic parameterization and then Antia has a paper more recently on implementation of this in the seasonal forecasting system and Judith as well as a very good review in BAMS on why stochastic parameterization important for weather and climate models so the idea is we put in these stochastic perturbations into our forecast model we have some kind of perturbed initial conditions this goes into the forecast model but now with the stochastic forcing the forecast model is also different for each of these ensemble realizations previously if we did not have any stochastic perturbations every single ensemble member would get this exact same model physics integration but now with these stochastic perturbations the model is also different so then that adds to the diversity in the solutions that we get it's not only coming in from these initial perturbations but also from the stochastic perturbations in the model so in ECMWF and this might be technical so if you don't follow it fully that's okay and we can talk about this later but I'll go through this quickly so in the ECMWF model modeling system or forecasting system currently the operational version uses two different types of stochastic perturbation the dominant one that is used is this stochastically perturbed parameterization tendencies which the idea essentially is you take these tendencies just as I gave the example of convection you take the temperature or u and v winds or humidity tendency at every single time step so at one time step you add up all the physics tendency in the model then you perturb it with a stochastic term which is represented here so x prime will be your stochastically perturbed tendency which is equal to one plus the stochastic noise times this x tendency which comes from the deterministic model integration so this perturbed tendency is what's used in in the integration of the model at the next time step and here the SPPT it has three different spatial and temporal decorrelation length scales so you can think of it as a autoregressive stochastic term and each of them have different standard deviations representing uncertainty in the mesoscale synoptic scale and large-scale variability so this is you can think of the scheme as representing uncertainty in known unknowns the physics parameterizations that are implemented in the model and then another type is the stochastic kinetic energy backscatter scheme which Judith also has worked a lot on this one tries to represent the process that is not being resolved in the model so it's upscale transfer of energy from subgrid scales to large scales it's representing how momentum is transferred upscale and it uses stochastic random 3d patterns to represent how much of the energy is partitioned into upscale versus dissipating out of the system and then more recently ECMWF has been testing what's known as stochastically perturbed parameters where the different parameters that are part of the parameterization schemes are perturbed prior to the physics integration so if you if you think of convection for instance you have different parameterizations to represent micro physics or convective closures so these parameters or coefficients are perturbed or are drawn from PDFs such as these such as this schematic presented here and there's I think 20 or there's a lot of different parameters that are cataloged and they have been cataloging and studying how much uncertainty are there in these parameters and how do we add these or draw these parameters from PDF different types of PDFs so this again is a different approach this one you add uncertainty before you do the physics integration and you expect the models physical integrations to propagate this uncertainty in parameters into the output variables of the system what I'll be presenting is largely impacts of SPPT the first stochastic scheme I describe so these stochastic perturbations not only impact ensemble forecasts and they not only have an effect on the ensemble spread or uncertainty in our forecast but recent work led by RTA and others at Oxford have also shown that they have an impact on the mean state of the model or the mean state biases in the model they help improve low frequency variability also even though the perturbations are at higher frequencies they have an impact on the long-term climate of the system so this is an example of the mean state bias and precipitation in the ECMWF system the top left is observed precipitation from this product called GPCP the bottom two plots are showing precipitation bias so two experiments were conducted a seasonal forecasting experiments over 30 year period one of them there was no stochastic perturbations in the model and the model was run for seasonal time scales and then you collect 30 years and you compute the mean precipitation in the model and you take a difference from the observations and what this will show us is where is precipitation biased in the model so what we see here in the blue colors are increased precipitation in the model compared to the observations so the model has too much precipitation in the ITCZ region in the Indian Ocean region the SPCZ region of the Southern Pacific and similarly in the Atlantic and then another experiment was conducted where stochastic physics was active in the system and here too we see this increased bias that the model has more precipitation but even with i you can see that the bias is reduced compared to the stochastics of case and that's shown in the top right panel here which is the difference between the stochastics of and stochastics on case and here what we see is that the ITCZ region and the South Pacific region and maritime continent there is increased precipitation when the stochastics of is switched off which means that adding stochastic perturbations helps reduce this mean precipitation bias in the model and then what's the impact of stochastic physics on the Madden-Julian oscillation which is now looking at intracesional variability in the model this again is looking at a similar experiment where we did sensitivity studies of of the different scales in the SPPT scheme and how it impacts MJO prediction so don't pay attention to all the different colors they're the blue green and purple or purple colors are the different SPPT experiments what I would like you to pay attention to is the red the difference between the red color where there's no stochastic perturbations and the other colors where there is stochastic perturbations and what we see is at least in the first two to three weeks there's a big impact in terms of stochastic perturbations help increase the scale of the model in terms of MJO prediction and this is predicting the MJO PC1 or RMM1 and RMM2 the two components of the MJO phase space and then some results from climate model experiments we have done so led by the group in Italy Paolo Davini, Susanna Corti and Yost one Hardenberg in collaboration with the group in Oxford have run climate model simulations with the IFS ECMWF system for yeah climate time scales 10 year long integrations at different resolutions so we compared climate model simulations without stochastic physics which is plotted on the left and with stochastic physics which is plotted on the right the middle panel is the observed fields what's plotted is the correlation of precipitation in an Indian Ocean box so it's 70 to 90 east we take an average precipitation in that box we compute a time series and we do a lead lag correlation lag is minus 30 day lag on the y-axis two plus 30 days on the y-axis so this propagating signal in correlation is corresponding to how precipitation for the MJO starts in the west Indian Ocean and then propagates through the maritime continent across all the way to the dateline the line contours are wind correlation showing convergence into this region what we see here is the control run does not have much of this signal of propagation but when we add stochastic physics we see at least in some of the ensemble members we get an improved representation of this propagation of the MGO and Hannah Christensen has done work in collaboration with Judith looking at similar diagnostics in the NCAR CESM model we see some improvement in the system and this is ongoing work where we are further analyzing why we see this improvement and how statistically significant this signal is now briefly in the next three or four minutes I'll touch upon quickly some machine learning ideas on improving both stochastic parameterization and ensemble predictions so the first work I'll present is work led by Will who's a grad student at Scripps who's also a student of this summer school we looked at how do we use convolutional neural networks a type of machine learning method to improve post-processing of atmospheric river forecasts the idea essentially is similar to image processing so convolutional neural networks can be thought of different layers of parameters and we can give it an input for instance here we give it either cat figures or dog figures but then we train the network parameters to identify cat figures specifically so we train it with a training data set of cats and then if we give this two different type of inputs it will be able to tell us if the input was a cat or not and the way this is done is we use like a cost function and we compute gradients to minimize this cost function that's what is shown in this cartoon below it's essentially optimizing parameters in this convolutional neural network and the optimal set which is the minima here in the cost function will give us the best set of parameters to do this prediction in an image sense so can we use modern these kind of modern deep learning neural networks to operate on image data to improve upon our dynamical forecasting systems and can this also inform on broader scale patterns rather than just looking at station data and improving or post processing statistics of individual grid points and once this neural network is trained it's fast and cheap to implement for post processing so we did do work this is published now where we showed that implementing such a convolutional neural network can help improve prediction of integrated vapor transport which is relevant relevant for atmospheric river forecast so the blue colors here indicate improvement in skill of this model to predict atmospheric rivers and then one idea to extend so that was all work done in a deterministic sense of just one deterministic forecast of the AR how do we post process it ideas on improving ensemble forecast there are more modern techniques in machine learning called Bayesian neural networks that draw samples from probability distributions and the lost functions are also defined based on probabilistic skill scores such as CRPS CRPS so we are exploring using these methods to improve ensemble forecast as well and will us work using CNNs as well to improve ensemble forecast of vapor transport in the Pacific so I just wanted to highlight this one data set that is now available online this work was led by Sue Ellen Hopp who will be talking later in this summer school about use of machine learning for both post processing and other aspects as well so it's a recent workshop in Oxford and this came out of it where we have posted data sets that can be used as benchmarks and training data sets for machine learning methods based on large-scale indices such as MGO PNA then one final remark in the next minute or so I'll quickly go through yeah it's somewhat recently in the last couple of years David John Gagney led this work yeah Hannah Christensen Adam Monahan and I were collaborators on this where we used a Lorenz 96 model a two variable system where the y variable the high frequency variable is coupled to this lower frequency x variable and it's coupled through this term and that can be thought of as a sub grid scale forcing to this x variable so then what we want to do is to parameterize this we don't integrate the y variable but we parameterize this term and how do we do this with machine learning so we used a machine learning method called GANs generative adversarial networks which essentially has two types of networks a generator which generates samples or synthetic samples that are similar to the training data set and then we have a discriminator or a critic network which essentially checks if this synthetic sample is part of the true sample or not and the more we do this check and give feedback to this generator network then it can correct itself and improve the output that it gives so we did this for the x and u variable in the Lorenz model and the output would be the sub grid scale tendency at the next time step which is what we want and we were able to do this in a stochastic term because the GANs we can add noise within the neural networks to generate stochastic perturbation so we tested a few different configurations of this I'll not go into details here just highlights we tested white noise and red noise inputs into these GANs and we ran these for like climate time scales and weather time scales and our benchmark was this polynomial parameterization which is similar to deterministic parameterizations in weather and climate models and we show from all these different configurations of these stochastic GAN machine learning methods there are a few configurations that can improve upon the deterministic deterministic parameterization of sub grid scale tendencies so I will stop here and take questions maybe for the next five five or ten minutes sorry to go over time no no you're not over time at all thank you so much maybe one minute that's okay thank you so much that was a very comprehensive introduction to uncertainty representation and then also linking it to processes