 In this module 5 we have been developing methods for assimilating data into deterministic dynamic models. The models are deterministic and we assume the models are perfect. The observations are noisy we are given a finite set of observations. We would like to be able to assimilate this observation into the model to be able to determine the initial condition. In this context we developed the 4D war method. It is also called first order adjoined method. We developed the details of 4D war adjoined methods in the last lecture. We demonstrated the power of these methods using linear models, linear observation, nonlinear models, nonlinear observation. In today's talk we are going to be talking about another related approach to assimilating data, to assimilating noisy data into deterministic model which we assume to be perfect. This method is called the forward sensitivity method. It was developed by us around 2010 and it can be shown that this method is in some sense develop the 4D war and we would like to be able to present the details of the forward sensitivity method to correct forecast errors using data. Data simulation can be viewed as forecast error correction using forward sensitivities. That is the view we are going to be taking in this class. The correction to the control, please remember the solution tiered dynamic model depends on the initial condition, the boundary condition and parameters. So initial condition, boundary condition and parameters because changing them changes the solution. It is customary to call them the control. So we are assuming that the model is perfect. So if you start the forecast model with the wrong control namely wrong initial condition or wrong boundary conditions or wrong parameters that would the error in the control will lead to errors in the forecast. We would like to be able to correct the errors in the control by using observations and we would like to be able to formulate this as again a linear awaited linearly square problem using forward sensitivity. In this method we do not need any adjoint but we need matrix recurrences that compute the forward sensitivities of the solution with respect to initial condition, boundary conditions and parameters. So adjoint is not needed that means the backward integration is not needed. Everything is forward but we need to be able to solve a system of system that describes the evolution of the forward sensitivities. So we are going to start with a non-linear model. Assume the model is perfect. A linear case will be a special case of this. Let X be a state, let X be a state, alpha be a parameter. One difference between this lecture and the previous lecture is that the previous lecture we assume the model is perfect. The parameters define the model because the model is perfect we assume that there is no error in parameters. The only forecast error in the previous talk was essentially due to errors in the initial condition. One way we can think of this as a slight generalization even though the model is perfect we have not set the values of the parameters when trying to generate forecast to the right value therefore we assume that there could be errors in the initial condition, boundary condition and our parameters. M is the model map which is the vector valued function. M depends on two vectors one is the state vector X another is the parameter vector alpha. Alpha is a vector of size p, p is an integer much like n is an integer. So M of alpha is given by M1 of X of alpha, M2 of X of alpha, Mn of X of alpha. The deterministic dynamical equation is given in the form of a discrete time dynamics Xk plus 1 is equal to M of Xk and alpha. X naught is the initial condition alpha is the parameter. I am going to assume the initial value the control c to be X naught and alpha. We try to generate the solution using 1 it can be readily seen that the solution at time k is a function of the initial condition as well as the function of the parameter. Therefore, to indicate this dependence of solution under control we call the solution Xk as Xk of c when there is no confusion we will simply say Xk. Observations are given observations are M vectors sorry I am sorry there is a typo this must be a M vector observations belong to Z observations are M vectors they are denoted by Z and belong to Rm. H is the standard forward operator the observation at time k is given by H of Xk star plus vk vk is the true state of the system c star is be the unknown optimal control that will lead to the true state of the system vk is the white Gaussian noise here I would like to differentiate between two types of state one is the true state which is not known to us the mother nature does not reveal all her secrets. So, Xk star is the true state of nature Xk is the state predicted by the model the state predicted by the model will be equal to the true state only when the model control are set to X naught star and alpha star which corresponds to the true state of mother nature but we do not know what X naught star and alpha star is but mother so, but we only know indirectly through observation what the mother nature has selected what is the value of the initial condition that corresponds to the true state which is X naught star and alpha star. So, we have we have picked X naught and alpha mother nature has picked X naught star alpha star if X naught is not equal to X naught star if alpha is not equal to alpha star the solution generated out of this will have errors that corresponds to forecast errors once we know the forecast errors our job is to be able to use these forecast errors to be able to control to be able to alter the control X naught and alpha such that X naught moves closer to X star alpha X naught star and alpha moves closer to alpha that is the basic idea. So, we are it is useful to think about a classification of forecast errors which we have alluded to in module 1 itself but it is worth repeating some of the principles of classifications of forecast errors for our purpose. The model is perfect we are assuming that the model is perfect we are assuming the forward operator H is very faithful that means I have the physics has been the physics that that is part of H is very good that means the relation between the model state and the observation is pretty accurate. In that case the forecast error EKFC is dependent on the actual forecast generated by the model so H of XKFC is the model counterpart of the observation ZK is the actual observation but ZK is equal to H of XKC star minus VK. So, if I substitute for ZK here this I can rewrite it at BFC minus VK where BFC is equal to the difference between the model predicted observation and the true observation if there is no noise. So, this becomes B. So, BC is called the deterministic part of the forecast error which depends only on C the control VK is the random part of it we cannot control the random part random part is part of the deal we have to contend with this. So, the only way we can hope to be able to correct the forecast error is to be able to correct the deterministic part of the forecast error. So, the best way we the best we could hope for is to select a control C such that it annihilates BC it reduces BC to 0. If you can reduce this deterministic part to 0 then whatever is left is simply random errors which is beyond our control which is totally uncontrollable. So, this is what is called forecast error correction and this is the aspect of the forecast error that we are concerned with the model is perfect the forward operator is perfect the forecast error if any arises because of the errors in control. So, I would like to be able to move the chosen control towards the unknown using the forecast errors in a feedback loop to be able to control to be able to correct the deterministic part and annihilate it. So, with that as a basis and now I am going to state the problem formally let C be the chosen control let C be the chosen control where X naught and C are arbitrary well arbitrary to the extent that general engineers and scientists know they may not know the exact value but they do know the region where the exact value lies. So, X naught and C while it is not equal to the true state but they may not be too far from the true state as well they are arbitrary. Now I am going to define a functional again look back in all of the developments in inverse problem there is always a cost functional which is very basic to the solution of the inverse problem the cost functional is essentially the is denoted by G F C it is simply the weighted sum of squared errors E K C we have seen in the previous slide given by the equation 4 is the forecast error this must be R K sorry this must be R K so this is the weighted sum of square of weighted sum of square of forecast errors if I substitute for E K from the previous slide I get this expression this is the forecast error this is the forecast error transpose and this is a very familiar equation 5 that describes the cost function is a familiar formula that we have used repeatedly in almost all of our treatment the reason why this relation occurs again and again because our approach is rooted in least squares the common theme of our course is least squares approach to solving inverse problem solving data simulation problem. So, this is the least square cost functional so what is our goal our goal is to be able to find a perturbation delta C such that if I change C to C plus delta C J of C plus delta C has no deterministic component and is purely weighted sum of terms of the kind which are induced by the errors that means I would like to be able to add delta C to the control delta C consist of delta X naught for the initial condition delta C and delta alpha for the for the parameters. So, how do I add corrections to the initial conditions of the parameters so as to annihilate the deterministic part of the forecast error leaving behind only the weighted sum of squared errors of Gaussian random variables. So, please remember V K is Gaussian V K transpose R K inverse V K it is the weighted sum of square is a normalized sum of squares of Gaussian random variables pictorially we are going to start from C this is the forecast X K is the forecast resulting from C I would like to be able to change C to C plus alpha C plus delta C consists of delta X naught and delta alpha. So, this is the change in initial condition delta C so this is the new control from the new control I can get a new state which is X 1 plus delta X 1 delta X 1 is the change in the state at time 1 X 2 delta X 2 delta X 2 the change in the state at time 2 X K plus delta X K is the change in the state at time K all the delta X K is induced by two changes one delta X naught the initial condition to delta alpha in the parameters I must say this is delta. So, delta X naught belong to R n delta alpha belongs to R p these are increments in X naught and alpha delta X K is the induced increment in X K at time K so what is our goal our goal is to be able to find delta X naught delta alpha such that when I use this new state X K plus delta X K which is the perturbed state and apply to the function H it will give the observation model of the noise that means it will match the deterministic part of the noise what is the deterministic part of the noise this is H of X star H of X star that is essentially that is essentially the deal here that means my model predicted observation will match the actual observation model of the noise that is the equation 6 and that is the goal with which we are going to be that is the goal we are going to be working towards and this equality is to be interpreted in the least square sense in the least square sense. So, I again we would like to be able to understand the structure of the forecast errors to that end so what is that where are we now we have already assumed I have perturbed the control perturbed control leads to perturbed state the perturbed state is given by X K plus delta X K so Z K must be equal to so we would like to be able to find delta X K we would like to be able to force delta X K such that Z K is equal to H of X K plus delta X K I can express this in the first order Taylor series leading to this expression which is H of X K plus the Jacobian of H at X K times delta X K if I using 4 let us go to 4 for a moment now so in 4 I have E K of C is equal to H of X K C minus H of X K star C minus V K so if I brought that in here if I brought that in here this part the this part sorry this part which is the increment is given by this equation this is equal to Z minus H of X K Z minus H of X K is essentially minus E K C star going to equation 4 you can readily see I would like to show it do you once again equation 4 that is sorry equation 4 so Z equation 4 tells you E K is equal to H of X K C minus E K therefore Z K minus H of C is minus E K so that is the relation that we have here the Jacobian as you know is a matrix of size m by n E K C is a matrix of is a vector of size m so you can think of this like H of X is equal to Z where H is a m by n matrix X is n by 1 and this is m by 1 that we saw in our in our lectures on static problems so you can see delta X K is so X is related replaced by delta X K H is replaced by the Jacobian of H and Z is replaced by minus E K Z so once you make this correspondence between the static problem and this problem our job is to be able to find delta X K such that it satisfies this equation. This equation represents an equation of the linear least square type that we have discussed in earlier modules we also know the E K is a random vector its covariance is R K so I can determine X K by solving 7 as a weighted linear least square problem as a weighted linear least square problem so let us let us summarize what we have done you run the model forward from an erroneous control I would like to be able to compute or characterize the increment in the state delta X K at time K so what is the increment we would need at time K which is delta X K such that it will annihilate the forecast error such an increment at time K is obtained by solving this linear least square problem as a weighted least square problem because we already know that E K is a random vector whose covariance is R K inverse so this is how now once I characterize this a solution of this linearly square problem we already know delta X K is related to delta X naught and delta alpha using forward sensitivities I am going to now relate delta X K to delta X naught and delta alpha so that is the next step in our analysis. So recall X K which is the solution at time K is a continuous or a smooth function of X naught and alpha so X K depends on X naught and alpha smoothly smoothly means what X K is differentiable with respect to X naught and alpha those of you who have taken differential equations you know that under smooth so if I have a differential equation X dot is equal to F of X of alpha if X is smooth smooth in the sense that F has is not only continuous but possesses continuous derivative with respect to X and alpha of several orders the solution X of T of X naught of alpha this solution is a smooth is also smooth it you can compute the derivative with respect to T you can compute the derivative with respect to X naught you can compute the derivative with respect to alpha that means if F is smooth in a differential equation X naught is equal to F of X of alpha if X naught is the initial condition the solution at any time T is equal to X of T when what we say as X of T is in fact X of T of X naught and alpha in differential equation theory one can show if F is smooth X is also smooth with respect to X naught and alpha in the case of discrete time dynamics a similar conclusion also follows if my model map M is smooth the model solution is also smooth with respect to the initial condition and alpha this smoothness essentially allows us to be able to compute the derivative of X K with respect to X naught the derivative X K with respect to alpha now let us denote the ith component of X K as X K comma I I am now going to compute the first order derivative of X K I with respect to X naught J so this is the Jth component of the initial condition this is the ith component of the solution at time K I am going to denote this as U K I J U K is a matrix I J I refers to the component at time K J refers to the component at time 0 this derivative is called the forward sensitivity of X of X K of I with respect to X 0 of J so as you vary I and J in the interval 1 to N you get a matrix that matrix is called U K so U K can be thought of is simply as a Jacobian of X K with respect to X naught this Jacobian matrix is called the forward sensitivity matrix of X K with respect to X naught so this is where the forward sensitivity of the solution with respect to X naught comes in so what does this say if you make a small change in the initial condition at time 0 what will be the effect of that small change in the initial condition on the solution at time K that is that is what this matrix of forward sensitivity captures likewise I can define the forward sensitivity with respect to alpha so V K I J is again the X K I is the ith component of the solution at time K alpha J is the jth component of the control vector the partial derivative is the I jth element of a matrix V K so that is called the forward sensitivity of X K I with respect to alpha J so you can again think of V K as a Jacobian of X K with respect to alpha this is called the forward sensitivity matrix of X K with respect to alpha alpha so V K the matrix V K the matrix V K is a N by P matrix it represents the forward sensitivity to the solution at time K with respect to the P parameters is called parameter sensitivity U K on the other hand is the forward sensitivity solution with respect to the initial conditions both these sensitivities essentially tells you how a small change in the parameter value how a small change in the initial condition would be amplified at time K so you can think of the sensitivity as a kind of an amplification factor that helps to amplify the initial error so having defined the concept of forward sensitivities now I would like to use U K and V K to be able to characterize X K please remember why are we interested in that we have already said the way to correct the forecast error is to determine delta X K as a solution of the linear least square weighted version of the problem so that is satisfied. Now we have to relate delta X K to delta X naught and delta V K this is where the role of U K and V K comes into being so let X K be the model solution at time K from first principles the variation so from this equation we can derive this so X K is a function of X naught and alpha therefore the first variation of delta X K induced by delta X naught and delta alpha is essentially given by the product of delta X K the product of the forward sensitivity of X K with respect to X naught times delta X naught plus the product of the forward sensitivity of X K with respect to alpha times delta alpha is a very basic principle that comes from Taylor series expansion first variation and the notion of a differentials in calculus. Now we have already seen that this matrix is V K this matrix is U K and this V K so this matrix is a n by n matrix this matrix is a n by p matrix therefore the induced increment is related to delta X naught via U K and delta alpha via V K please understand I need to be able to control I need to be able to change the control change in the control means it will change the solution so everything must be related to time zero so this is how we relate the change in the state at time K to the change in the state at time zero and change the parameter at time at time zero so now we have achieved one of the things that we need to be able to relate delta X K delta X naught and delta alpha. But to understand but to use eight to characterize delta X K now we need to be able to characterize what is U K and what is V K U K please remember how the solution at time K varies with respect to initial condition V K is how the solution at time K varies with respect to the parameters these are called forward sensitivities. So we need to be able to explain or we need to be able to find out a way to characterize the evolution of U K and V K please recall these are matrices so we are going to collectively talk about the evolution of all these of these two matrices in time. So the dynamics of evolution of U K is our next topic consider the component of the model dynamics in one so please remember the model dynamics is given by X K plus one is equal to M of X K of alpha. So if I consider the at the component on the left hand side I should be able to consider the at the component on the right hand side and that is what equation nine is all about consider the at the component of the equation the model equation in one. So this is the equation in one which I have written for us to be able to recall. Now I would like to be able to differentiate both sides with respect to the jth component to the initial condition. So the differential coefficient of the at the component at time K plus one with respect to the jth component time 0 that is the left hand side. Now I can use the chain rule alpha does not depend on X naught X K depends on X naught MI depends on X K and X K depends on X naught so I have to use the chain rule in standard calculus. So the left hand side is equal to partial of MI with respect to partial of the Qth component of X K and the partial of Qth component of X K 2 the partial of jth component of X naught so the product of these two partial derivatives. Now Q is an arbitrary one Q can take the value 1, 2, 3 all the way up to n. So I have to sum it up over Q so this is the fundamental relation that relates the dynamics of evolution of the forward sensitivity. Please recall this is the forward sensitivity of the ith component with respect to the jth component time K plus 1 time K plus 1 and this is the sensitivity of the Qth component with respect to the jth component I can rewrite this in the form of a inner product that is the row that is the column you can convince yourself very easily this is the ith row of D K M this is D K M and this is the jth column of U K that is being that is being that is a fundamental interest right now that is a fundamental interest right now so let us go back now. So one sensitivity component is given by the product of the ith row and the jth column if I want to consider all the sensitivity matrices simultaneously there are n square such sensitivity relations the set of all n square such sensitivity relations is captured in the form of a matrix dynamics. So U K plus 1 is a sensitivity of time K plus 1 U K is a sensitivity of time K U K plus 1 is a matrix U K is a matrix D K M is the D K M is the sensitivity I am sorry it is the model Jacobian with respect to X K so here I am using a simplified notation I want you to remember that instead of writing D of X K of M we are simply write D K of M D K of M so D K of M is the model Jacobian so this is a dynamical equation in discrete time I need an initial condition U naught is given by partial of delta X naught with respect to delta X naught that is an identity matrix and I have already talked about the change in slight change in notation for the better. Now please remember 10 is a linear equation 10 is a non-autonomous equation it changes along the trajectory it is a homogeneous recurrence relation it is a homogeneous recurrence relation please recall U K is the sensitivity of X K with respect to X naught therefore equation 10 tells you how the forward sensitivity of the solution with respect to initial condition evolves starting from the initial value of I so I have to compute all these along the trajectory you may recall in the context of non-linear version of the 40 bar we talked about the tangent linear system you can think I am going to challenge the reader to be able to compare this with the tangent linear system and verify this is very similar to the tangent linear system that we talked about in the context of 40 bar in the context of 40 bar so 10 is a forward dynamics it is called the forward sensitivity now I can iterate 10 starting from I therefore weak I can iterate sorry I can iterate 10 sorry this is U K I can iterate 10 and so U K is the given by the product of the Jacobian along the path this can be written as the product succinctly like this which can again using the notation that we have already used is the product of the Jacobian along the trajectory pi denotes the reverse product from K minus 1 to 0 of the Jacobian of m along the trajectory please realize this must be U sorry we will correct that this must be U therefore U contains information about the behavior of the trajectories in what way does this contain the information about the trajectory if I perturb the initial condition by a small amount how sensitive the solution is going to be at a future time that is why it is called forward sensitivity. So, we not only derive the forward sensitivity equation 10 but also have solved for the forward found an expression for the forward sensitivity it is simply the product of the Jacobian of the model map along the trajectory. Now we are going to concentrate on deriving a similar dynamics for VK please remember VK is the sensitivity the solution with respect to the parameters again I am going to consider the ith component of the model equation that was also the starting point previously earlier we found the derivative of this with respect to the jth component of the initial condition here we are going to differentiate this with the jth component of alpha therefore the left hand side is simply the derivative of the scalar with respect to alpha the right hand side it is slightly different from the previous derivation XK depends on alpha so MI depends on alpha in two ways MI depends on alpha directly MI also depends on alpha indirectly through XK so one is the direct term another is the is to be obtained by chain rule this is the term that comes from the chain rule to accommodate the implicit dependence this is the explicit dependence so the sensitivity of XK plus 1 the ith component of it with respect to the jth component of alpha is essentially the sum of two terms the first one is an implicit dependence second one is an explicit dependence again expressing so that I varies from 1 to n j varies from 1 to p so there are n p such relations by collating all these n p such relations and arranging them in the form of a matrix we get a matrix recurrence this is the model Jacobian this is the model Jacobian with respect to parameter with respect to the parameter so this is the so dm alpha is the Jacobian of m with respect to alpha now what is the initial condition for this initial condition is delt is the partial of X naught with respect to alpha the state of the initial term is X naught alpha is the initial settings of the parameters the initial condition has no bearing on the parameter value and the parameters has no bearings on the initial condition therefore the initial condition is identically 0 it is a 0 matrix so this is the starting initial condition so 13 is a recurrence relation that starts from a 0 matrix whereas the previous one started from identity as the initial condition so we have derived recurrence relation for the evolution of both the forward sensitivity with respect to initial condition and parameters equation 13 is as in is again a linear non-autonomous non-homogeneous recurrence relation the previous one was linear non-autonomous it is homogeneous now I have a forcing term dm of alpha is the forcing term we are now going to simplify the solution of this we are going to change the notation let ak be dkm let bk be that in this case the previous relation 13 can be rewritten as 14 vk plus 1 so vk plus 1 is equal to ak times vk plus bk I can substitute back that is a very good interesting exercise I am going to ask you to do this as an as an exercise by solving 14 one can verify that the solution vk is expressed as 15 where v naught is the initial condition but please recall v naught is a 0 matrix therefore the first term vanishes the second term is the solution therefore vk is given by the sums of products of matrices a's and b's a's are the model Jacobian with respect to with respect to the state b's are model Jacobian with respect to this is with respect to the state this is with respect to the alpha so it is the products of Jacobians model Jacobians both with respect to the state and the parameter the sum thereof is a complicated expression much more complicated than the uk but however we have an explicit expression for the sensitivity of the solution with respect to the parameters I think we have we are almost there to state the final version of the problem. So please recall from 7 we know the delta xk that will help to annihilate the deterministic part of the error is given by 7 I have quoted the 7 back again here from 8 we know okay now let us let us talk about the change so xk is equal to x of k of x naught and alpha therefore delta xk from basic differential equation basic theory of differential is that delta x by delta x naught times delta x naught plus delta x by delta alpha times delta alpha so this is the matrix this is the matrix and this matrix is u of k of delta x naught plus this is v of k plus delta alpha and that is what equation 8 is all about therefore we are able to x delta xk in terms of delta x naught and delta alpha. Now I can substitute this delta xk from here to the equation 7 if I substituted this and simplify a little bit I get d of xk h times uk comma d of xk h times vk times delta xk delta alpha must be equal to minus e k minus e k of c that is equation 17. Now let us look at the left hand side I know the Jacobian of h I know the forward sensitivity because I have already solved the recurrence relation again these two are the same so I know this this is again computed from solving the forward sensitivity e k is known so now we can see the left hand side is a matrix that is the vector that is the vector we know this we know this we do not know this part we can essentially see I got a classical linear least square problem why this is linear the unknowns or the unknowns what are the unknowns delta xk delta alpha they occur to their first degree therefore we have converted the forecast correction problem to one of being able to compute the increments in the initial condition delta x naught and delta alpha and the these increments are going to be decided by the solution of this linear least square problem and we will formulate it as the weighted linear least square problem because the right hand side is stochastic and I know its variance is archaic so to further simplify the notation so the forecast error correction as a as an inverse problem I have already alluded to this and that is what we are going to be emphasizing again. So I am going to define hk of one as the product of these two matrices hk of two as the product of these two matrices I am going to define a new matrix which is hk which is the two matrices put together side by side so this matrix hk is a m by n plus p matrix so combining this with 17 we have please recall delta x naught and delta alpha is the vector of delta c this is of size n this is of size p so the whole thing belongs to Rn plus p therefore by solving this linear least weighted least square problem with rk inverse as the weight using the methods that we have already discussed using the methods I do not think the module 6 is correct I will correct that applying the corrections to correction delta t to c and I can then compute delta c by the solution that is the whole idea of the forward sensitivity method. Now you can see how the forward sensitivity method gets into the matrix hk on the left hand side of 19 so I am going to talk about an algorithm which is called the forward sensitivity method fsm is an acronym start with the initial value of the control c start with the initial value of the control c compute the model trajectory x naught through a xk let zk be the observation at time k that is given to you that is that is zk is given to you compute the error which is h of xk so you know h you know xk so you know h of xk h of xk minus e is the error compute uk vk by solving the forward sensitivity dynamics the matrix recurrence relation assemble the matrix hk 1 hk 2 that gives you the matrix h then solve the resulting least square problem as a weighted linearly square problem with the weight rk inverse once you compute delta c you can add delta t to c I get a new control. Since we have used the first order method while the new control will be better than the old control in annihilating error you can expect the error will not be annihilated in its entirety because we have used only first order recurrence of first order Taylor series expansion in our in our in our theory therefore what is a way to further reduce the forecast error you repeat the whole procedure again starting with c new and do whatever you did with c so if you iterate this several times each time you are going to keep adding increments to the control that continuously keep moving the the the the the forecast error towards purely random part it annihilates continuously the deterministic part that is the idea that is the idea. So in all these discussions we essentially said if I had only one observation how I can use one observation to be able to to to be able to compute the control in principle it may have multiple observations so I am going to provide a quick extension of this let there be n times let there be n observations the observation that these n times be given by zk1 zk2 zkn for each ki now I simply need to repeat what I did for one observation to each of these observations and then collate them all together that is all the idea there is nothing more because because each observation is going to give me give me information about the increment so I have to repeat as many times as I had so I need to be able to compute hki1 earlier I had hk1 here I have hki1 hki2 which is given by this which is given by this ukivki are the sensitivities of the solution at time k xkih xkih are the Jacobian of the forward operator at time ki therefore hki is the concatenation of these two matrices side by side now for each time ki I have this matrix now I have n such times now I have to prepare another gigantic matrix so create a newer matrix h where I stack hk1 hk2 hkn this matrix is of size or nm cross n by p likewise I also stack all the errors forecast errors at time k1 k2 kn and that is going to be giving you a large vector of size nm so if you pull them all together I have now a new linearly square problem please remember delta c has not changed only the matrix h has changed only the vector e has changed in this case I have I think it is it should be minus e minus e I can now solve this as the weighted linearly square problem the weight matrix is r inverse r inverse is simply a diagonal matrix r inverse is a matrix of size nm cross nm so each of the yeah each of the n matrices are put along the diagonal of a matrix r this r will be a collection of all the covariances of observations at each time so is a is a gigantic matrix and you can formulate this as the linearly square problem since we have solved the static linearly square problem weighted and weighted version in detail in previous modules we are not going to describe that part but one can see that if you know the linearly square static problem you can solve this dynamic problem simply an application of those concepts here so what is the interesting thing here is that even though it is a dynamic data simulation problem we convert the dynamic data simulation problem to a static deterministic inverse problem and that is the essence of forward sensitivity method we can also talk about continuous time versions we simply for the sake of simplicity used the discrete time version now I am going to present a a quick example of the forward sensitivity method using a continuous time model this is a model that describes a motion of cold air over a hot ocean so this is called the Lagrangian forecast in in meteorological circles that temperature is x, xs is the temperature of the sea surface xs is greater than so in united states we have the Gulf of Mexico during winter time cold air sweeps from the north the waters of the gulf are very warm in October November it holds lot of the heat from the summer so when the cold air moves over the warm gulf there is a heat transfer from the warm gulf waters to the cold air so the air becomes warmer and warmer so I am now equation 22 gives you the dynamics of evolution of the temperature x of the air column k so x is the sea surface temperature k is the turbulent mixing or a turbulent heat transfer coefficient and we are assuming x k, k, t are all non-dimensional in other words originally I had a dimensionalised version assume that I have non-dimensionalised it we have already done that so this is the non-dimensional version of the model equation the solution of this model can be given explicitly because simply a first order differential equation we can readily solve from standard experience in calculus so this is the solution this is the form of the solution x0 is the so you can see the solution depends on x0 the solution depends on xs which is called the boundary condition and the solution also depends on the parameter k so the control here c consists of x0, k and xs so the control has 3 components one is the state of the system one is the initial state of the system another is the parameter another is the sea surface temperature the solution is monotonically increasing and xt tends to xs as t goes to infinity that means as the water as the air moves over the water the heat transfer takes place the air temperature rises the heat transfer stops when the air temperature is equal to the sea surface temperature that is the very simple dynamics so because the solution is known I can compute the sensitivity is rather explicitly I do not have to use the differential equation to compute the evaluation of the sensitivity so that is the initial condition sensitivity boundary condition sensitivity parameter sensitivity. So once I know the sensitivity I am ready to compute the matrix H but before that I need to be able to compute the observations or have the observations so using the observations starting from initial condition make the forecast compute the errors compute the forward sensitivity you know the error using the forward sensitivity I can compute the matrix H we have the standard linearly square problem. Here are the examples for this model the model solutions this is the model solution x of t this is the model solution this is the sensitivity of the model solution with respect to x0 this is the sensitivity of the model solution with respect to the boundary condition this is the sensitivity of the model solution with respect to the parameters. So you can see solution and three sensitivities are graphically given here I am now going to discretize the model equation using standard Euler scheme this is the standard Euler scheme so if I simplify this 25 I get this equation this is again a linear equation because we started with the linear model beta is k times delta t delta t is the time interval for time discretization I can solve this model equation explicitly and this is the discrete version of the model solution that is given in 27 we have already given the continuous time version of the model solution earlier. So what is that we are going to ask you to do we are going to ask you to be able to that is what problem 2 is all about verify that the solution of the recurrence is given by this we also want you to be able to solve the recurrence relation that are given in the previous slides when we did that I also want you to be able to derive for the continuous time dynamics with that is problem number 1. I would like you to be able to differentiate the continuous time dynamics and derive the dynamics of evolution for the forward sensitivities both with respect to the parameter and the initial conditions. This FSM method was developed by ASS it was published in a paper in 2010 entitled forward sensitivity based approach to dynamic data assimilation which appeared in advances in meteorology in the volume in 2010. In this paper we describe both continuous time as well as discrete time the example that I talked about we discuss in complete detail we would like to refer the reader to this paper and I would also very strongly encourage you to be able to produce the results in this paper to make sure that you have a total and a thorough understanding of this methodology. With this we conclude our discussion of data assimilation using FSM forward sensitivity method you can really see this is another way to do this in our next lecture we will talk about the relation between 40R and FSM I strongly encourage you to be able to perform a data assimilation using the forward sensitivity how do we do that you assume certain values of the parameters you generate the solution you generate observation and then change the initial condition create erroneous forecast then use the generated observation to be able to control the erroneous forecast and that is what is called the twin experiment which we have been doing all along why what is the best way to be able to benchmark your algorithm is to be able to generate data artificially using the model and if the model if the method is capable of being able to recover the true initial condition then you have shown the proof of concept using these artificially designed observations that essentially gives you a clue that it should also work in real situations so that is the basic idea of the whole approach with this we have completed the discussion of FSM we will in the next lecture talk about the relation between FSM and 41 thank you