 Now, that we have derived the equations for Kalman filter namely the forecast equation, the forecast mean, the forecast covariance, observation, the analysis step, the analysis mean and the analysis covariance. In this case the system is linear, the noise is Gaussian therefore the forecast is a Gaussian random variable. The observations have Gaussian distribution, analysis have a Gaussian distribution as we had observed several times in the previous lectures that Gaussian distribution is the only one that is decided uniquely by the mean and the covariance. So, if I compute the analysis covariance and the analysis mean I essentially characterize the entire probability distribution. Against this now I am going to describe couple of simple example to illustrate the dynamics. First I am going to talk about scalar dynamics with no observation that means this is called stochastic dynamics. So, let A be a scalar positive, WK be a scalar Gaussian random variable with mean 0 and variance Q, Q is the variance and X naught is M naught P naught. So, P naught in this case is a scalar, P naught is a scalar. The dynamics is given by a simple scalar linear dynamics XK is equal to A times XK minus 1 plus WK. So, we have talked about the value of A WK X naught the solution for this linear occurrence can be given by this I would like you to verify the correctness of this equation by substituting back into the by using the method of substitution that is the best way to describe it. So, from here we get this solution. Now I can take the mean of both sides please remember the mean of WJ is 0 and WJs are temporarily uncorrelated. Therefore, if I took the mean the second term does not contribute anything the mean of the state at time K is 8 to the power of K M naught. The variance of the state XK at time K is PK that is equal to variance of X A times XK minus 1 plus WK. If you multiply a random variable by a constant you multiply the covariance by the square of the constant variance of the sum is the sum of the variance since the two quantities are not correlated. Therefore, the variance of the state at time K is given by this and this XK is essentially the forecast because I am simply using the model. So, it to the power of K MK is the mean of the model forecast the A square PK minus 1 plus Q is the variance associated with the model forecast. Please understand this variance has two components one coming from the initial covariance the distribution of the initial condition second coming from the model noise. This is the scalar analog of the vector forecast covariance we have already derived within the Kalman's within the Kalman's framework. And if you now substitute PK minus 1 in terms of PK minus 2 PK minus 2 in terms of PK minus 3 and so on and open it up and simplify PK depends on P naught A to the power of 2 K P naught plus Q times A to the power 2 A to the power 2 K minus 1 divided by A square minus 1 again I would like you to verify by solving this simple linear recurrence relation. So, for a given M naught P naught and Q what is M naught the mean of the initial condition P naught is the covariance of the initial condition Q is the variance of the observation I am sorry is the variance of the model noise. I would like to be able to now analyze the behavior of all the moments what are the moments the first moment and the second moment of the forecast that simply depends on A now that simply depends on A because all the other factors are fixed for a given M naught P naught and Q the behavior of the forecast moments the first moment and second moment depend only on A when A is less than or equal in the region A greater than 0 less than or equal to 1 the model is stable what do you what do you mean by the model is stable the model solution without the model solution does not explode to infinity in fact it can be shown from the solution of the model equation in the previous step XK is equal to A to the power of X naught plus that we can readily verify that limit XK as K cones is 0 then P K when the limit of P K is also given by that. So, the limit of XK is given by this the limit of P K is given by this and I would like to be able to tell you that this essentially comes from the first equation I would like to call it star which I would like to call it star. So, if this is star if this is double star in the next equation both of this comes from star and double star. Now if A is less than if A is finite but greater than 1 the model is unstable that the complementary part the model is unstable the limit of XK goes to infinity the limit of P K also goes to infinity then A is equal to 1 the model defines a random walk. So, in this case XK plus 1 is equal to XK plus WK plus 1. So, it executes a random walk on the real line in this case XK is equal to X naught plus WK WK is the sum of all the noise. So, E of XK is M naught P K the variance of XK P K is equal to P naught plus K Q. Now you can see even when A is equal to 1 while the mean remains the same its covariance increases linearly as P naught plus K times Q. So, this is simply the analysis of the behavior of the solution of the stochastic linear dynamics given by XK plus 1 is equal to A times XK plus WK plus 1. Now I would like to bring in the data into the picture and that brings I continue the same example I am now going to talk about Kalman filtering. So, without data I simply make predictions with model alone we talked about forecast mean forecast covariance we analyze how the forecast covariance varies for different regimes when A is positive in between 0 and 1 when A is positive and greater than 1 when A is equal to 1. So, we divided the range of values the parameters into three sub regions one stable another unstable another corresponds to random walk in two of the three cases when A greater than 1 or A is equal to 1 we see that the variance go to infinity the random walk model is very interesting and we may have occasion to talk about it later XK so now let us continue XK plus 1 is equal to XWK plus 1 WK plus 1 is again the noise with 0 mean and variance Q ZK is equal to H times XK plus VK H is a scalar VK is a scalar VK is 0 mean is a Gaussian random variable with a 0 mean and and and variance R now I have to distinguish between forecast and the analysis therefore forecast state is equal to A times the analysis of time K in other words I am going from time K to time K plus 1 this transition is what we are talking about earlier we talked about the transition from K minus 1 to K absolutely there is no difference except for the values of the indices so PK plus 1 F is the forecast covariance at time K plus 1 is equal to A square times the analysis covariance at time K plus Q the analysis itself is given by the forecast plus Kalman gain times ZK minus HK HKF the Kalman gain is given by this formula which can be simplified as which can be simplified as PK hat H R inverse and then the analysis covariance is given by this expression PKF minus something I am trying to subtract a positive quantity from PKF therefore the analysis covariance becomes less so PKF R so analysis covariance is equal to forecast covariance times R divided by H square times PKF plus R inverse so this is the very simple expression for the analysis covariance as a function of K all these things all these things arise from the derivation of the Kalman filter except that we have substituted the corresponding formula so now given I have the equations for the analysis analysis covariance forecast forecast covariance I can now talk about the stability of the analysis part in order to understand the stability the analysis part in other words what do I want to find does the analysis go to infinity as time goes to infinity does the analysis comes down to 0 what happens to the analysis error what happened to the forecast error as a function of the model parameter these are some of the things that we would like to be able to understand to analyze the stability of the filter so in this case forecast is given by the forecast at time K plus what is equal to 8 times 1 minus KKH KKH XKF plus 8 times KKZF ZK the analysis at time K is given by this I would like you to go back to what we are doing look at this now I can substitute the in this equation how do I get this I can substitute the analysis expression into the forecast expression that is exact I can also substitute the forecast expression into the analysis expression so that analysis at time K plus 1 can be expressed in terms of forecast at time K I am sorry analysis at time K that means I can get a recurrence in the analysis analysis value at time K and K plus 1 I can also get the forecast expressions connecting the forecast at time K plus 1 to forecast at time K that is the important part of the recurrence in here analysis depends on forecast forecast depends on analysis by mutually substituting each other I can express forecast depend on forecast analysis depend on analysis at time K plus 1 to time K that is the recurrence we are talking about so by substituting this I get one recurrence for the forecast the one recurrence for the analysis so if I have a forecast I can compute the forecast error if I have analysis I can compute the analysis error we have already seen the forecast is equal to forecast minus the state given by the model analysis again analysis error can again be computed as we have already seen how to handle the analysis error forecast errors I also have the expression for the forecast covariance in terms of analysis covariance and this case covariance is essentially the variance the forecast variance is a square times the analysis variance plus Q here again the analysis variance depends on the forecast variance again I can substitute each other I can substitute this in here I can substitute this in here I can then relate forecast at time K plus 1 to forecast at time K analysis time K plus 1 to analysis at time K if you do that the example continues now after that substitution I get this equation yes you can see this is heavily there is a lot of heavy algebra now divide both sides by R if I divide both sides by R I get this relation so what does it tell you this tells you forecast covariance at time K plus 1 is related to the forecast covariance at time K now look at the expression that I have saw the forecast covariance occurs both in the numerator and the denominator so this is the non-linear recurrence relation this is a non-linear recurrence relation therefore I am now going to change the notation I am going to define pK is equal to pK f by R if I did this this one in view of this one you get this relation where alpha is the ratio of Q over R this is an interesting ratio what is Q Q is the variance of the model noise or the variance of the observation noise so the ratio of the two devices is alpha so if you look at this normalized forecast covariance see that is what I want to emphasize in this is the dynamics of evolution of the normalized forecast variance what is the normalization I have normalized this with respect to the variance of the observation I have normalized this with variance of the observation so if you look at this relation you can see A is a constant pK plus 1 depends on pK H is again a constant alpha is the ratio so you dip this expression on the right hand side depends only on A H and alpha this type of recurrence relation in mathematics in the theory of difference equations has come to be called Raccati equation it is the first order equation why this is the first order equation K plus 1 depends on K it is scalar because we are concerned with only with the radian variances is non-linear obviously because the right hand side depends on pK in the both in the numerator as well as the denominator therefore it is the first order non-linear scalar recurrence the particular structure has been around for a long time it is due to Raccati an Italian mathematician there is a Raccati equation both in ordinary differential equation as well as in difference equation here we are concerned with the difference equation analog of the Raccati equation this equation is not easy to solve because it is non-linear so now I am going to talk about the asymptotic properties of the forecast covariance so let us see how we got here once more I substituted the the quantities because forecast depends on analysis analysis depends on forecast I mutually substituted them and made forecast dependent on forecast analysis dependent analysis likewise we did for the covariances as well once we did the covariances I am now trying to single out the forecast covariance I normalize the forecast covariance by the updurational covariance R and rewrote the equation that resulted in a normalized forecast covariance dynamics which is a difference equation it is a first order scalar non-linear difference equation now I am going to look at the asymptotic properties of this first order non-linear scalar equation why what is the aim the aim is the following I would like to be able to understand the regimes excuse me namely does the under what condition does the covariance grow under what condition the covariance the forecast covariance died out that relates to analysis stability of the forecast covariance now once I analyze the stability of the forecast covariance the corresponding properties of the analysis covariance follow immediately because forecast covariance depends on analysis covariance analysis covariance depends on forecast covariance so behavior of one asymptotic behavior of one will imply the asymptotic behavior of the other so to that end I am now going to assume h is 1 without loss of generality what if h is just a parameter that relates to converting the state into the observation so please remember zk is equal to h of xk plus vk in this case I am assuming h is equal to 1 I am assuming h is equal to 1 so in that case my equation now becomes simpler like this a square pk divided by pk plus 1 plus alpha I would like to be able to understand the behavior the long term behavior of this equation this important equation star here in order to understand the long term behavior of this what do I do I am trying to look at the increment pk plus 1 minus pk that means how much pk plus 1 differs from pk at the kth step if I substituted pk plus 1 in terms of pk and did the algebra I get this relation I get this relation which can be now written as a ratio of a polynomial divided by pk plus 1 the polynomial in pk is given by the expression of the numerator which we can readily identify yes if you are trying to read through there is ton of algebra and I think the only way to be able to do it is to be able to hit all the major developments and that is what I am trying to do leaving behind the details of the derivation of the algebra to the reader. So when when does this recurrence relation on pk converges that means pk plus 1 is equal to pk is the condition for convergence pk plus 1 is equal to pk is the condition for convergence at which time lambda delta k is 0 at which time delta k is 0 delta k is the equilibrium if delta k is 0 then g of pk must be 0. So now you can see how we have changed the variable from pk to delta k express delta k as a ratio of two polynomials in the normalized forecast covariance pk we have already now identified delta k being 0 is an equilibrium point at which k is pk plus 1 is equal to pk that means pk does not change it has come to a stable value because delta k is the ratio of two polynomials in pk the numerator polynomial must be 0 for delta k to 0 and that is where we are. So this calls for analyzing the behavior of the solution of the numerator polynomial the numerator polynomial is equated to 0 it is rewritten it can be rewritten by changing the variables. So I am going to concoct a new variable beta, beta is equal to a square plus alpha minus 1 please understand a is the model parameter alpha is the ratio of the two variances model noise to observation noise. So I can concoct a new symbol beta to this term. So if I did that my polynomial becomes this this is a second order polynomial in pk I can now apply the standard rule for finding the roots of the second order polynomial there are two roots p superstar and p sub star these expressions are now dependent on only beta and alpha please recall beta depends on a and alpha. So I have a numerator polynomial g of alpha which is a quadratic I have solved it. So I now know the value of the equilibrium at which point the four casco variance will settle down there are two equilibria p superstar and p sub star now we would like to be able to understand the behavior of the solution around these two equilibria to see whether the slope of it is increasing or decreasing such as it is like this or it is like this this corresponds to this corresponds unstable this corresponds to stable why this corresponds to stable if I am here this is this is pk pk plus 1 is smaller if I am here this is pk so from here it pushes here from here it pushes here. However if I am here the in this case if I am here it grows bigger if I am here it grows bigger so it goes away and it goes away therefore this is unstable that is stable stable means if I am to the left of it it pushes to the right if I am to the right of it it pushes to the left that means the stable equilibrium is an attractor in the neighborhood an unstable equilibrium is a repeller if I am to the right I move to the right if I am to the left I move to the left so this is the repell here attractor. So to be able to see something is a repeller on attractor we have to get the slope of g of alpha I am sorry g of pk in the at the point where the equilibrium occurs where the equilibrium occurs therefore the general expression for the gradient of g g prime of pk is minus 2 pk plus beta now we are going to evaluate by setting pk is equal to p super star super star is the star is a super script in this case the derivative is negative. So please understand here the derivative is negative here the derivative is positive so the derivative negative corresponds to an attractor the derivative positive corresponds to a repeller so you can readily identify this is p star this is p sub star so p sub star is unstable p star is stable so we need to consider only again I want to reemphasize one is an attract one is a attractor another is a repeller stable and stable therefore in the limit pk will go into the attractor when pk goes into the attractor at the attractor p star is equal to a star a square p star divided by 1 plus p star plus alpha. So you can readily see this must be the expression for the forecast normalized forecast covariance at the point at the equilibrium at the equilibrium you can you can solve this equation for p star and you can get the exact value I would like to now give strongly recommend that you plot g of pk versus pk you know g of pk is a very simple expression and also would like you to plot delta k versus pk and verify all the claims that we have thus far I think these are very important exercises to understand thoroughly the long term behavior of the forecast covariance when the model is scalar when the model is scalar. Now you can see the p star and p super star and p sub star depend on beta beta and alpha beta depends on a so you can find the regions in the parameter space that gives rise to stable behavior that gives rise to unstable behavior I hope I hope that is very clear so what does this tell you again I want to reemphasize this this to essentially tells you that no matter where you start pk plus 1 will come and settle at this point depending on the values depending on the values of a alpha and so on. So using MATLAB I would like you to be able to verify all these conclusions and that is an important part that is an important part of the of the analysis that is an important part of the understanding of the Kalman filter dynamics. Once I know it converges I think it makes sense to ask yourself the question how fast does it converge that relates to rate of convergence this is not the first time we are talking about rate of convergence we are talked about rate of convergence when we talked about gradient methods and especially when iterative methods like gradient method so in any iterative algorithm there are two questions one should ask does it converge if it does at what rate. So we are going to quickly indulge in the in the calculation for the rate of convergence let yk be equal to pk minus p star so what is pk minus p star pk is the current value of the normalized work as covariance p star is the asymptotic value at the stable equilibrium so I would like to be able to measure the difference between where I am and where I will hit sooner or later so I also know the equation for pk plus 1 so we would like to be able to compute the rate at which we would like to be able to compute the rate at which we converge so yk plus 1 is equal to pk plus 1 minus p star so I am going to substitute pk plus 1 p star simplify use the relation yk this is this must be pk sorry p sub k pk minus p star is yk this is also pk sorry this is also pk p sub k p sub k so if you this is also p sub kms so if you can now see we have already gotten an expression for yk plus 1 relating to yk and what is yk yk is essentially the difference between pk and p star. So if I consider 1 over yk plus 1 that is given by this expression so this expression is essentially coming from here which I can rewrite like this again this is yk sorry y sub k now I can I can I can rewrite this expression as a sum of these two terms again a little algebra will give you so 1 over yk plus 1 is equal to 1 over yk times a constant plus another constant so I am getting a recurrence for 1 over yk plus 1 so please go through the algebra now yk is the distance between pk and p star I am trying to express yk plus 1 which is the distance between pk plus 1 and p star in terms of yk so I am trying to get a recurrence in yk instead of trying to get a recurrence in yk I can equivalently get a recurrence in 1 over yk why we would like to be able to consider a quantity which is easy to analyse that is all what the matter here is. Now I am going to change the variable once more you can see how many different ways in which we can look at it so 1 over yk is zk that essentially tells you the previous equation at the bottom of slide 30 now becomes a linear equation the linear equation is given by zk plus 1 is equal to c times zk plus b where c is a constant and b is a constant that is the k so let us go over this quickly once more I have here a Carti equation I am going to change the variable for the Carti equation I have a Carti equation for pk now I have a Carti equation I have a corresponding nonlinear equation for yk plus 1 while I have defined yk I am trying to rewrite it as a recurrence in 1 over yk it turns out the Riccati equation after making these transformation these two transformation namely yk is equal to pk plus 1 minus p star 1 over yk is equal to zk the Riccati equation becomes linear linear equation can be very easily solved I can iterate the linear equation so this is the solution for the linear equation which can be like this so zk is given by this expression that is an important expression so 1 over zk is yk yk is equal to 1 over zk defined by this so let us not worry about this part in this equation b divided by c minus 1 is a constant even so when c is greater than 0 and c is I am sorry when c is greater than 1 I am sorry when c is greater than 1 ck to the power c to the power k goes to infinity therefore yk will tend to 0 yk tend to 0 so you can readily see that this expression is equal to c to the power minus k times a constant therefore when c is greater than 1 this yk tends to 0 at exponential rate I think that is the importance of this so c is equal to 1 plus p star by a whole square and that must be greater than 1 so under under under the condition that c is greater than 1 which in turn relates to 1 plus p star divided by 2 square if greater than 1 we can really see yk converges at an exponential rate so this is an exponential convergence that is an important that is an important conclusion so what what is that we have accomplished we have accomplished the following by converting a sequence by converting the Rackard equation to a sequence through a sequence of transformation we converted it a non-linear equation to a linear equation which we then solved and found conditions under which this linear equation solution to which go to go to infinity which in turn means zk goes to infinity zk goes to infinity when c is greater than 1 when zk goes to infinity when c greater than 1 yk tends to 0 now please realize why the definition of yk the definition of yk is pk minus p star yk refers to the distance between the current value the forecast covariance with asymptotic value so that distance goes to 0 at an exponential rate and this gives you values of the parameters under which the colvin filter forecast covariance not only converges but also converges at a exponential rate excuse me the rate of convergence continues once we have again we have assumed h is equal to 1 once we have the convergence of pk we can now conclude the convergence of pk hat which is the analysis covariance the expression for pk hat divided by R is given by this equation if you took the limit of this pk hat tends to a particular limit because pk f tends to a particular limit and this is the limit of the analysis covariance so analysis covariance converges with forecast covariance they work in locked step now I am going to go back to the analysis of what is called stability of the filter stability of the filter h is 1 this is the forecast covariance sorry this is the forecast covariance I am sorry I say forecast covariance that is wrong this is the forecast error the forecast error can be simplified to be this the forecast error recurrence has two parts the homogeneous part and the forcing part the homogeneous part is given in the following the forcing part consists of two error terms akk kvk plus wk plus 1 so this is the stochastic part this is the deterministic part the deterministic part is also the homogeneous part because this is the forcing please recall kk the Kalman gain is given by this so from here I am going to get 1 minus kk as this so the Kalman gain kk is given by this formula which is pk divided by pk plus 1 from that you can really infer 1 minus kk is equal to 1 over pk plus 1 substituting these and simplifying it can be verified that the e bar f of k plus 1 is given by this recurrence substituting the value of p star it can be verified that e bar fk plus 1 is equal to 1 over square root of c times e bar fk since c is greater than 1 1 over square root of c is less than 1 that means in going from time k to time k plus 1 the error reduces therefore if you compute the error ek bar f with respect to the state at time capital N ek bar f is given by 1 over square root of c to the power k minus n times e bar f by n so n is can be thought of as a starting time since 1 over square root of n 1 over square root of c is less than 1 as k goes to infinity the term 1 over square root of c to the power k minus n tends to 0 therefore the overall error goes to 0 this in turn means the filter is stable so we have analyzed all the properties of the Kalman filter and have illustrated the derivation as well as the nuances with respect to several properties of the Kalman filter equation using a simple static linear dynamics thank you.