 So we have been looking at Kalman filter and in my last lecture I did a derivation so those of you who could not attended should have a look at CD webpage for listening to the lecture. So what I will just take a very quick recap of what we have shown in the last lecture. The one thing is if you make an assumption that state noise measurement noise and the initial guess for the state all of them have Gaussian distributions state noise and measurement noise are Gaussian white noise processes and initial estimate or initial at time 0 the estimate has a Gaussian distribution then what we have shown is that Kalman filter gives you a maximum likelihood estimate it also gives you a maximum a posterior estimate the value that or the vector that maximizes a posterior density function for nice thing about Gaussian distributions is that if state noise and measurement noise are Gaussian white noise processes and initial condition is a white noise then all the conditional densities of xk given yk-1 or xk given yk they are also Gaussian processes so they also have Gaussian densities they are not Gaussian processes but they are Gaussian densities you can show that distribution of them is Gaussian then Kalman filter can be derived without making any assumption about density so you do not have to make an assumption about Gaussianity you can derive Kalman filter just using arguments of minimum variance okay minimum covariance so trace of covariance matrix we minimize with respect to the gain matrix and we still get the same thing so one of the beauty of this result is that it can be derived by multiple viewpoints and you get the same result if you start from just optimization viewpoint just minimizing a scalar objective function minimize the trace of the covariance matrix you get exactly the same estimate which you would get if you have to start with Gaussian assumption and then looking at you know which estimate maximizes the likelihood function or which estimate gives you maximum of posterior density all of them just give you same result so different viewpoints just collapse into one single result okay why this result is important because it is recursive it is it systematically takes into account how the noise or uncertainty in the state dynamics and uncertainty in the measurement are systematically taken into account yeah here we have when we come to stationary observer we need some assumptions of that sort but here the stability can be shown without requiring observatory assumption so observatory assumptions are required if you if I am not wrong only required if you look at stability by not ignoring I have looked at stability only one limited viewpoint only error in the initial state I think when you go to input to state stability you may require observatory condition or observatory condition may be required for existence of the Kalman gain that I have to check where the observatory condition comes into picture here so nice thing about Kalman filter is that you only work with first and second moments of the distribution whether it is Gaussian or non-Gaussian you just keep working with first moment second moment and it is very easy to do calculations and this is much easier than doing pole placement because how do you choose the poles if you happen to know WK and VK to be Gaussian white noise processes is not clear okay so that part is not clear so this gives you very systematic way of handling you also looked at convergence of errors and then looked at it as a deterministic linear system and then what I showed was that you can construct a Lyapunov function and you just consider the error in the initial condition and it will converge the difference between the true state and the estimated state will asymptotically go to 0 and this we did using you know this is the just summary of the Kalman filter but here I am not looking at WK and VK yet I am using same update equations which are used for the Kalman gain here you can look at Q and R to be two tuning matrices which are positive definite okay and then you can actually show that one can construct the error dynamics is given by this equation this we have already derived earlier and this difference equation actually asymptotically stable that I have shown using this matrix inversion lemma and then constructing a Lyapunov function which is so construct so we have defined this Pi K here Pi K is inverse of covariance matrix using inverse of covariance matrix we have constructed Lyapunov function and then shown that this Lyapunov function is negative definite moreover under certain assumptions you can show that this is a asymptotically stable system so observer is stable okay so actually in this case we designed an observer first for the performance how was the performance defined performance was defined using minimum variance and then now I am showing that the designed observer turns out to be stable okay just going back here to this difference equation why did I have to use I had to analyze stability of this difference equation this particular difference equation here combined difference equation now for this combined difference equation I could not use the condition that eigenvalues inside the unit circle why what was the problem why I could do that because it is time varying system this matrix here Phi times LK C is constant Phi is constant LK is not constant LK is time varying so this is a linear system but with time varying time varying matrix Phi Phi means if you call this whole thing as some Phi tilde this combined matrix you cannot look at its eigenvalue because it is not constant with time it is changing so you cannot use eigenvalue analysis here I had to resort to use of Lyapunov functions okay so indirectly it also demonstrate how Lyapunov functions can be used very powerful theory so that is why I did not talk about spectral radius less than 1 eigenvalue less than 1 because here you know it is this matrix combined matrix Phi times I- LK C is changing with time okay so each one of them will have different eigenvalues and then you cannot say that if all of them are inside unit circle you cannot prove that then system is stable that cannot be proved okay so such a condition does not exist okay if this matrix is constant it does not change with time you can use that condition of inside unit circle that is that condition holds for linear time varying and discrete time systems okay so that is why I had to resort to Lyapunov functions and then I just showed that I can construct a Lyapunov function using covariance inverse or distance major using covariance inverse and then if you see this VK is error transpose covariance inverse into error actually this term appears also in the Gaussian density function e to the power same term so I just showed that this is VK plus 1 minus VK is always negative definite and this means that the error asymptotically goes to 0 so we have at least under limited conditions we have showed asymptotic convergence of Kalman filter okay I have not touched upon here what happens I have made a simplifying assumption that WK and VK are 0 to treat it under Lyapunov framework when WK and VK are not 0 is possible under certain modifications there is you can in Lyapunov function theory or Lyapunov stability theory there is something called input to state stability so you can also talk about unforced see this is we talked about unforced stability here you can also talk about stability when there are external disturbances like WK and VK but then the match becomes much more complex I do not want to get into that right now so at least for limited case I have shown that it is stable okay so now I just want to move ahead from here and what I want to do now is to make a connection with time series models okay again when I started looking at these two things Kalman filter and time series models well somehow the books written on time series models and books written on Kalman filter are by two different proponents one kind of people who more believe in you know first principle models and then Kalman filter is used there with mechanistic models whereas time series models come from completely different angle and there is a deeper connection and many times this is not highlighted in the sufficiently in many of the books but actually if you see what is the connection then you will be able to you know appreciate what is happening much better if you remember when we developed all those ARMAX ARX you know those kind of models attempt was to capture effect of unknown disturbances unmeasured disturbances okay in Kalman filter to you have two unknown disturbances WK and VK okay and attempt is to capture the effect of WK and remove VK and you know so something similar is happening we need to connect and see what is the connection okay now I want to first talk about this concept of stationary Kalman filter now stationary Kalman filter is what you typically see is that when K goes to infinity this PK given K-1 and PK given K do not change with time they go to a stationary solution okay and this stationary solution I am denoting it by P tilde I am calling it P tilde because this is K given K-1 and I am calling this as P infinity okay so those see these are when you go to Kalman filter these are different equations if you look at these equations these are different equations okay these are connected different equations this and this they are different equations in time okay and then just like a linear system will have a stable solution this is a linear difference equation you know it will have it is a linear difference equation and it actually asymptotically under certain conditions the solutions asymptotically converge to steady state okay the steady state I am denoting by P tilde infinity P infinity and L tilde sorry L star infinity now L star infinity is called as you know this L star is called as steady state Kalman gain which means after some time when you do programming now my next computing assignment is that for the given system you are supposed to implement Lohenberger observer and Kalman filter input to be given is PRBS again okay start from a initial condition which is for the observer start from a initial condition different from the plant and see whether the observer converges to the truth okay so this is the next assignment I will put it on the moodle today but now the idea is that once we have this simulator you can do everything that we have learned in the class so just now next submission is implement Lohenberger observer and Kalman filter on that particular system on which you are studying okay and there you will notice this when you program to see that if you actually keep track of how gain is changing after some time this gain becomes constant okay gain does not change with time and that particular gain is this now actually you can write the steady state equations okay these are called as algebraic Riccati equations the original equations which we use to find the covariance matrix and the Kalman gain are called as Riccati equations and these are called as algebraic Riccati equations because there is no time involved here in algebraic Riccati equations I am just writing the same equations at the steady state so these are coupled matrix equations okay and there has been lot of literature and how to solve this particularly when the dimension become large it becomes a complex problem and probably large part of 60 70s lot of work went on how to solve algebraic Riccati equations how to solve so actually now my prediction and update is prediction of course is given by this there is no gain involved here update is given by a constant matrix which is now I am just showing you quadruple tank system okay which we have been looking at as a theme example the I have taken covariance Q to be 0.01, 0.01 so Q is a diagonal matrix I assume that there is uncertainty in each of the states not entering through any input but general uncertainty in each of the states whose covariance is given by this particular matrix and R is the error in the measurement whose covariance is again given by 0.01, 0.01 I have just taken some sample values it is not that so that sufficiently there is large unknown component now we just put final steady state solution because it is difficult to show all possible LK varying as a function of time there are many LK varying as a function of time as K goes to infinity you get this probably we can see that it actually Kalman gain goes to let me just show you first these results you can see here that this is the estimate this is the perturbation model initial estimate I gave for the perturbation model to be 0, 0 okay the true state is 2-2 see I have the true state is 2-2, 2-2 okay initial value initial value or initial guess for the observer I have to give initial guess for the observer I have given it to be 0, 0 okay the noise is there okay and then I have to you know I have to compare since I am doing an experiment in my computer I can compare the truth with the estimate okay now these two level one and level two are measured level three and level four are not measured okay so level one and level two you know there is no wonder that this starts following each other but look at level three and level four level three and level four are not measured so this is the comparison between the estimate this green is an estimate and this is the truth okay so this gives you a least square estimate you have to understand you are never going to recover the the truth this is the estimate that minimizes variance or covariance matrix okay so the green is an estimate of level three and level four if I want to use it for control I can use it because after some time the mean value is exactly following you see this so here you get a least square estimate minimum variance estimate so it is not a true value okay I just told you that covariance when you do an update the covariance of the updated state is smaller than the covariance of the predicted state all that I have done here is I have plotted the spectral radius of the covariance matrices updated covariance matrix and predicted covariance matrix okay you can see that when you do update the covariance reduces spectral radius reduces which means the covariance reduces okay because for a four cross four matrix to show that it reduces is difficult so I use the spectral radius as a major because this is a positive net matrix all I can value the positive okay smaller I can smaller spectral radius means the covariance is reducing so this is a demonstration of let us see how the code looks like and let me see if I can demonstrate that Kalman filter actually goes to steady state after some time P matrix goes to steady state so now for the observer case I am not going to post my code okay you can see it here but you have to write it on your own understand and write it okay so my Kalman filter this is slightly older program but what you have to understand is that at kth instant when I get a measurement I do all the calculations for Kalman filter so this is my Pk1 is my predicted covariance update this is my Kalman gain calculation Pk1 into Cmat into inverse of Vk then I do a correction here oh this is the prediction I compute the covariance I compute the innovation ek is yk is the measurement which is coming from the plant and Cmat into x hat k is the estimate and this lk times ek lk is the Kalman gain which has been computed here lk times ek is where you are doing the update and this is the prediction this is the updated covariance Pk is function of Pk-1 so these are the calculations that are done for Kalman filter okay this is see you have to understand how a digital control system works to implement this at kth instant when I receive a measurement immediately I do this calculations it is assume that it is almost instantaneous at least relatively instantaneous compared to the gap between the two samples okay so these calculations have to be done and then I send the whatever input that goes to the plant this is plant simulation and this is observer simulation so this is observer and this is plant this is plant okay and then we can actually check how these matrices change as a function of time so let us start tracking them so let me let me say here to begin with we will put or we will do it from the here it is Pk-1 then Pk and lk and I will put a pause so you will see that as we run this program this is the observability matrix and its rank is 4 okay so I am just showing you how Pk is changing with time and lk is changing with time you see that right now Pk okay let us do it let us put them side by side Pk and Pk-1 let us put them okay so you see how this covariance is change with time so Q and R matrices are given and then okay so this is my so this part is my initial Pk given k-1 this is Pk given k this is updated covariance this part and this part is predicted covariance and lk is the Kalman gate you will notice that these values will start changing as a function of time so this again this is the predicted covariance this is the updated covariance and so on and asymptotically this equations will if you see the values here let us just look at a diagonal values the diagonal values are reducing and that to the diagonal values of the predicted covariance are higher than the diagonal values of the updated covariance okay so every time you do update the covariance reduces the error reduces okay and Kalman gain is changing as a function of time but after some time it will go to a steady state now if you see this matrix and this matrix just look at diagonal values they are not different from the previous time this lk is not changing so within some 20 samples the Kalman gain has converged to lk star or lk infinity okay the covariance is have converged to their respective steady state values okay so this is so after some time see this change in covariance is occurring only in the initial period after some time you know it is going to a steady state so one might say that why do you want to program you know if I am running a Kalman filter for a very long time only suppose the initial 100 to 100 samples Kalman gain is going to change okay instead of doing this programming for you know time varying Kalman gain which is going to change only first few instances just take the steady state value and use it practical approach it can do it so you can just solve algebraic equations get the steady state Kalman gain and use it and you will get good okay so instead of doing this programming in which covariance change lk change and then they go to steady state you could directly choose to solve steady state Kalman gain equation the MATLAB gives you solution if you want for the steady state algebraic equation and you can directly get the Kalman gain and use it for so let us move back to is everyone clear about what is done so the observer is run parallel to the plant okay so when you code when you write program you have to run at kth instant you have to get a measurement write the observer k to k plus 1 you get new measurement created here when you move from k to k plus 1 then this measurement this is a loop this is loop comes back to observer again okay new measurement comes you do estimate and you know it is running parallel to the plant okay that is what that is how you implement this okay and that you can see from this graph also see what is the meaning of if you see here initial period this spectral radius is changing after sometime it has stabilized okay spectral radius is not changing which means pk given k minus 1 and pk given k have become constant after sometime they are not changing with time they have just stabilized it is a stable solution okay now what if your model is not correct there is some mismatch in the model what if there is some disturbance or one of the simplest problem is suppose a and phi and gamma matrices are correct and c matrices but and the measurement noise typically is a Gaussian noise measurement noise is not problem but the state noise being a Gaussian white noise process or a white noise process is a very simplistic assumption sometimes or oftentimes the noise entering the state dynamics is colored okay it is like a drifting signal and so even though you have very elegant solution when you assume it to be a white noise okay mathematically very convenient see many times we make simplifications so that you know you get some mathematically elegant solutions but then you start from that you get some understanding there and then you start going towards the reality you know you relax some other assumptions and then you so one of the things that is done is that suppose this wk we assume to be white noise we model the unknown disturbance using two components we model this using another component here I am another additional I would say artificial state neta here sometimes this is artificial sometimes you could relate it to a real problem real disturbances and then this gamma neta is some kind of a tuning matrix so this omega is again artificially introduced this omega is a white noise so what you are saying is that there is another disturbance which is like integrated white noise integrated white noise is random walk model see this is neta k plus one is old value plus a white noise so this is like an integrated white noise this is like a drifting behavior this is the most commonly used model when you want to model disturbances which are unknown but drifting simplest model to use okay and so this kind of a model is used to take care of the fact that the state disturbances need not be white so this is white noise and this is called as integrated white noise why integrated white noise because this is an integrator neta k plus one new value is old value plus noise this is simple integration last value is equal to see integration I do not know whether you have done this derivation for you know PI controller discretizing a PI controller see I suppose I am just giving you a parallel another example so that you will appreciate why this is an integrated see suppose I have this term you know say u t is equal to integral 0 to t ft dt okay so this I can approximate as summation i going from or k going from 0 to say n there are n samples from 0 to t fk and dt I take sampling interval to be t okay so this integral I can if dt is nothing but the sampling interval and fk is the value this function takes at discrete time points and if these samples are very close I can write this integral like this okay but then what you can do is that u t plus t is 0 to t plus t which is ft dt and I can approximate this as summation k going from 0 to n plus 1 fk into t right I have advanced by one sampling interval okay t is my sampling interval I advanced by one sampling interval so this is integral from 0 to n where n is equal to t by t by okay n is equal to n samples takes you to time small t and n plus 1 samples take you to time t plus t okay so this small t plus t so now one way to write this is to see suppose you take this as equation 1 and you take this as equation 2 okay I can subtract you know subtracting 1 from 2 you will get u t plus t minus u t is equal to see this is summation up to n this is summation up to n plus 1 so if you subtract what will remain only one term will remain okay so that will be f of f at k into t so I can write this equation as u t plus t is equal to u t plus fk into t or in other words u k plus 1 is equal to u k plus something so integrator integrator in discrete time can be written like this okay this u is an integration of all the values in the past so I am writing an integrator in a different way. So coming back to our equation here this is an integrator okay neta k plus 1 is equal to old value of neta plus correction now this correction is artificially introduce white noise so it is like a drift okay so drifting behavior so if I want to model state disturbances which are not simplistic white noise model then you can augment this is the common trick because you have to be able to model those unmeasure disturbances there are ways by which you can do it slightly differently and we will see that soon so this q is then you know you augment this system you have new state space model xk plus 1 neta k plus 1 together become a new state and then you can work with Kalman filter for the augmented system so you can design a Kalman filter for the augmented system and typically in such case this q neta is treated as a tuning parameter to make sure that the drifting disturbances are modeled and then how to use this tuning parameter will require some experience it is not straight forward so w term will be there w will get modeled through q so I am saying that it is addition of white noise plus integrating white noise w here w neta that is it artificially introduce tuning white noise no it is not same as wk yeah so this is something which is artifact of a modeling it is just trying to capture a drifting noise as a combination of integrated white noise and a white noise this is integrated white noise this is white noise and then attempt to merge the two into a okay I will just let us go over the Kalman filter summary we have to have initialization we have to have initial guess when you start you have to kick off your Kalman filter you need a initial guess of the initial state you will also have to give a initial covariance this is typically given some alpha times i where alpha is a large value okay you covariance large initial covariance means you are not sure about the state guess that you are giving if you are pretty sure about a guess that you are giving you can give a small value of this alpha times i this alpha can be small so this is kind of a tuning parameter when you actually start a Kalman filter what should be the initial covariance you have to give a guess and this p0 actually will tell what trust you place in your guess okay if you give large p0 which means you do not trust your covariance is large means you do not trust your initial guess covariance if you give a small which means you have a reasonable guess okay then the at kth time step the thing that you have to do is to compute Kalman gain then do this recursive estimation and did I miss oh I am talking about the prediction estimate here one minute let us revisit this well sometime back if you remember I talked about two different types of estimators we when we yeah when we talked about you and my observer we first develop this prediction estimator if you remember and then I said there is a filter estimator prediction estimator is using one there is one time delay between the estimate and why okay and then we had this current state estimator current state estimator was prediction step and then the correction state right so there are there in the observer I first derived the prediction estimator and then I showed how you can use the correction filter estimator in Kalman filter I first derived the filter Kalman filter I can also derive Kalman predictor one step predictor okay so I am not giving you the derivation I am directly giving me the final form here you can look at Astroman Wittenmark for derivation which runs very very similar to what we have done slight algebra is different but the final form is more or less the same I mean the ideas in the derivation are more or less the same and final form of the equations is slightly different okay so this is so in this case prediction estimator I compute the Kalman gain okay I have a prediction estimate which means K plus one is LP star just like we had prediction Leunberger observer this is prediction Kalman filter Kalman predictor this is Kalman predictor okay and the update there is only one covariance update there are no two different there is nothing like you know prediction correction there is only one step so again you will ask why will you use this and why not the Kalman filter it depends upon how much computation time you have you have to understand that you have to do this calculations of matrix multiplications and you know gain update inverse or whatever is involved in real time and it depends upon which system you are trying to use this observer if your system is very very fast if you have to do millisecond calculations and if you have a processor which is you know onboard processor is not so powerful then you might resort to prediction because in prediction you can do this calculation between the two samples keep gain ready and when the measurement comes you can just you know or you can at the instant when you want to use observer you can just use the prediction estimate and do not wait for calculations to be over at a given sampling instant so that is where the prediction estimator is useful another reason I am talking about prediction estimator will become clear very very soon why am I but is the idea clear is what just like we had prediction Leunberger observer this is prediction Kalman predictor okay this Kalman predictor and there is only one covariance is only one covariance because we do not have prediction correction step is only prediction step okay and this is the this is how you update the this is the new covariance from the old covariance this is how you calculate the Kalman gain for this case the Kalman gain and for the prediction case and the Kalman gauge for the filter case only differ through this five okay if you look at the if you compare the values if you have the notes you can find out and see the how the values compare only this five will come extra here so I am just showing you how a Kalman predictor would look like this is for the CSTR example the reactor example this is my model I have developed Kalman predictor here the steady state predictor will look like this and I am trying to estimate concentration from temperature and very quickly the observer error goes to zero and since I am doing simulations I can actually compare the truth with the estimated value and you can see here that the estimated concentration and the true concentration are almost you know sitting on the top of each other so you get very good estimates very quickly. So the error is you know initial error is large but after that it converges this is like a zero mean error and you get a mean value estimate through this so the point is see now in the reactor I want to control the composition okay now once this observer has settled I can develop even before you go to the advance control I can develop a simple PID controller okay that will measure concentration what do you know measure concentration actually physically you are measuring a temperature then you give the temperature measurement to this observer observer will reconstruct concentration estimate and then based on the concentration estimate you do the control so you close the loop using estimated state okay so this Kalman filter is actually acting as a soft sensor okay. So actually in some sense after these microprocessors have become you know so powerful and you can use calculations online the combination of real sensors which give you you know fast measurement of some physical variables which can be measured together with these model based sensor you know has expanded the possibility of measurement far beyond what was possible say 30-40 years back when you could not use on board computer now you can put on but computer on a you know on a device and you can have these soft measurements coming up so model based sensing I would call it model based sensing is a very very rich area used in many different fields of engineering okay now question is under what condition the solutions of Riccati equations exist and when will you get stable solution so you can show that under weak conditions if this pair Phi and square root of Q is stabilizable what is stabilizable I am going to talk about it in today's class but right now just except this theorem and if Phi and C are detectable weaker form of observability then you can show that the solution of Kalman predictor always exist and you will get this is about the steady state solution so you can actually come up with algebraic criteria this is for the Kalman predictor just like we can have stationary Kalman filter you can have stationary Kalman predictor and this gives you conditions under which the solution or you can find the stationary Kalman predictor which is stabilizing so that means the error will go to 0 okay so right now all the things that are needed to understand this I have not covered but I am stating it here for the completeness maybe as you progress will be able to collate and understand this the reason I am talking about the steady state Kalman predictor is this now this is my steady state Kalman predictor as K goes to infinity PK given K- will go to P infinity let us call it P infinity here this P infinity is the solution of this algebraic Riccati equations these are two coupled equations if you see here this P infinity is a function of P infinity plus LP but LP itself is a function of P infinity okay so you can actually eliminate and combine into one equation one big equation and solution of this is the steady state Kalman predictor and steady state Kalman predictor when does the solution exist for this particular this is a tough equation to solve because just remember these are matrices all these are matrices okay and then you have a problem here because this P infinity P infinity appears here nicely as multiplicative but when you go to LP there is a inverse of P infinity coming up here okay so solving this problem getting a solution for this problem is not trivial okay fortunately now MATLAB gives you solution of I think there is a subroutine called ARE algebraic Riccati equation okay so you just give phi and C and say ARE it will give you Kalman gains okay so you do not do much now to solve those algebraic Riccati equations you just give PQ phi C okay and say ARE and if you do not have MATLAB you can use so which is a you know which is a public domain software and you can install it on your machine and then you can get going you can design a Kalman predictor so it is not a so anyone of these standard or you can write your own tiny code and you saw that it goes to steady state so even if you do not have ARE okay you write your tiny code for the observer after some 100 samples you will get the steady state solution okay so no problem you are in business if you just have some computer with you and routines for matrix multiplication and well also inversion mind you these things I mean these things can become complex when you miniaturize you want to have embedded observer it can be complex because you know we on a computer assume that inversion can be done very easily if you want to do it on some micro controller writing and where floating point you know computation support may not be available or you know may not be so powerful then doing matrix of computations then doing matrix inversion is not a joke using a computer you know but computers are relatively slower okay and difficult to use for very fast systems so where so on embedded computing even though these things look very simple on a computer PC laptop they are not easy when you go to you know embedded control but if I want to see what is the advantage of designing this LP infinity what is the computation advantage all online there is no every time calculation you compute this once offline you design this LP 0 and then you know you this step involves only this computer so online matrix computations all are gone and anyway you know after sometime it stabilizes to LP infinity use that LP infinity and you know solve the problem so straight forward way of so this is much more practical in to have stationary Kalman filter and then use it but in the assignment you are not going to do stationary Kalman filter you are going to use very change with function of time okay now what I want to do is I want to connect I want to show that stationary Kalman predictor is nothing but box Jenkins model nothing but a form of box Jenkins model or time series model so you know we just close the loop and other way of looking at it is that box Jenkins model or RMAX model is a parameterization of stationary Kalman filter that is under view point okay so look at this stationary form of the Kalman filter now instead of calling it L star I am just see this is what we had right to look at this last equation x hat k plus 1 is ? x hat k given k-1 ? uk plus some constant gain matrix into ek where ek is actually yk-c x hat k given k-1 so I have written it in a different form it is the same equation okay now I am going to just remove this you know k plus 1 given k k given k-1 I am just going to remove all that complex notation I am going to use the same notation that we use at the end of system identification okay so this actually this form of model is called as innovation form of state space model here you have 5k xk plus ? uk plus L times ek is defined by this equation yk is equal to C xk plus ek this kind of a model this kind of a model form is called as innovation form of state space model it is driven by we have shown that ek is a white noise sequence we have shown that ek is a white noise sequence and so ek is a white noise sequence ek in our case are innovations we have shown that innovations is a zero mean white noise sequence we have proved that earlier okay so it is a zero mean white noise sequence with known covariance we also had an estimate of p how to compute p so if I just take Laplace transform if I take q transform of this I will get nothing but you know we have done this earlier I just take q transform of this this is nothing but the box Jenkins model okay so these two are one and the same so I arrived at it completely differently earlier what we had done we developed we developed box Jenkins model first and then we developed state realization now I am coming to box Jenkins model to another root I developed Kalman filter then I developed stationary Kalman filter and then I said if I take its transform you get box Jenkins model okay so it is a loop once I started from this end and came to this end now I am starting from this end and going back here okay so I am just showing you that the stationary Kalman predictor is nothing but a form of box Jenkins model okay it is a form of box Jenkins model. So there is a deeper connection between time series models particularly when you are modeling unmeasured disturbances okay and now will you understand it within this course it is very difficult it will take some time to see pin it took me many years to understand these connections unfortunately the books which talk about time series modeling do not talk about Kalman filters the books which talk about Kalman filter do not talk about time series modeling and then you have to read both and make connections Leung's book it does make these connections but he makes it in an exercise so unless you happen to go and solve exercise problems you know you do not see those connections and but just keep this in mind that when you are developing a time series model actually it is a way of developing a form of Kalman filter and other way around a Kalman filter stationary Kalman predictor or Kalman filter can be shown to be related to the box Jenkins models so why I am saying box Jenkins it is generalized form it could be R max it could be RX so all those are just parameterizations but so identifying these models is somehow equivalent to so one could say that Kalman predictor could be directly identified from data and I do not have to go through first principles that WK VK characterization then Kalman gain capitations I can directly develop box Jenkins model and then I have a Kalman filter that is under viewpoint it is completely data driven way of getting a Kalman stationary Kalman filter or stationary Kalman predictor so you can always instead of doing this you can always directly identified from time series data how do you go what do you do when the model is non-linear okay that is done using I am just going to mention it in two minutes we do not have time to go over this if you have a see we did linearization okay and then we develop the observer using linearized models but in reality most of the systems are not linear they are non-linear and then you need to use non-linear models okay so actually Kalman's development I have uploaded by the way those papers the seminal papers which are given price to this entire area so Kalman's paper talked about linear systems that problem was I do not I would not call it it was a low hanging fruit it was very difficult to come to that point where you could come up with those recursive solutions which are so elegant but after that that triggered research on what do you do when the model is not linear okay and that is given rise to entire area of state estimation so the simplest thing that came up first was called as extended Kalman filter extended Kalman filter is where you try to use same update formula or gain calculation formula as Kalman filter but through local linearization of the non-linear model and I have just summarized it here very very quickly so you just do the these equations for gain calculations remain same except that Phi and gamma is computed by locally linearizing the non-linear model at every time point okay so kind of merging the two by using some approximations I have uploaded one very nice review paper by Chen which actually talks about entire development of state estimation from say 50s to now a very very exhaustive review and that probably you should attempt if you want to do research in that area one of the very nicely written reviews. So EKF will just showing you an example if I do EKF now with linear model what happens is you can use it only in a small range around the operating point where you are linearized non-linear model you can use everywhere I am just showing you here over a wider range I can use state estimation using this extended Kalman filter in extended Kalman filter I am going to use non-linear differential equations directly in my prediction step in the update step look similar see what is different the update step is same okay the prediction step here in extended Kalman filter is nothing but directly using non-linear differential equations okay that is what it means I have written it in a little bit abstract form but this prediction step instead of using linear difference equation linearized model we use the non-linear differential equation directly using OD45 or whatever whatever okay. So this is estimation of another example a tubular reactor this is a system in which a reaction is carried out from some A goes to B goes to C and typically in such systems what you want to know is how the concentration is changing as a function of length and as a function of time this is a distributed parameter system concentration of the reactants A, B and C are changing as a function of time you can only measure temperature okay but temperature is also changing as a function of time along the length. So this is a problem which is more complex than the full time problem so you can place only few temperature measurements along the length of the reactor this is like a think of it as a tube in which the reaction is being carried out okay now I want to know how that profile inside changes as a function of time I can use partial differential equations in my prediction step these are coupled partial differential equations that talk about how concentration and temperature are related and so on if you solve them together you can develop a software sensor that actually shows you profile inside just based on temperature measurements okay. So I am merging temperature measurement data with the model predictions online in a computer okay and then this is some mtech students work where we actually use this to generate concentration profiles inside some details are given here and there are unknown disturbances in the feed concentration and feed flow rate and then this is the exit concentration I am not I am interested in what is the purity of the product but I can only measure temperatures I cannot measure concentrations so I have used this partial differential equations online solution of partial differential equations merged it with the temperature measurements through Kalman filter Kalman gain calculations and I am just estimating the concentration this is how the profile of concentration changes as a function of time so this is at time 1 minute this is at time 5 minutes and the profile goes to a steady state of a sometime so using this model based technology you can actually see what is happening inside you can see as you can reconstruct a state of see ultimately all these things are used in all kinds of image reconstruction and you know because you only get certain measurements you have a model and then you reconstruct what is inside okay so this is one example where we have this is a experimental work you know where this setup I think I had shown you earlier there are two tanks in series there is a heater here and I am mixing hot water and cold water this is a simple system that you would have in your bathroom you have a heater water heater and then you are mixing you know water from cold tank and hot tank and you want a temperature particular temperature for your water for bath okay so this is my problem is to control temperature and level in this particular tank the problem is here one problem that we faced while developing the state estimator here is that the heat transfer from this tank wall to the outside changes as a function of time in the day time it is different night time it is different it is different when the fan is working when the fan is not on if 5 fans are on if 2 fans are on if 1 fan is on the heat transfer to the atmosphere is different okay and the model changes so what we did was we wanted to monitor online the rate at which the heat transfer is changing it is not a directly measurable parameter okay but through model what we have done is that see this is the Q is the heat transfer model that we have developed under one particular condition we multiplied by multiplied this by an effectiveness factor some alpha and we say that is alpha can change with time okay and with this added this alpha as an extra state that can be estimated using this measurements and then you know I am showing you here this effectiveness factor heat loss parameter we called it okay and this heat loss parameter I can track okay so when the heat loss parameter was wrong there was a difference between the estimated temperature and the temperature after some time the so observer is able to estimate things that are not directly measurable that is what I want to point out here this is experimental work so again the same mtec student experimental work so there was a mismatch here and after the mismatch was closed okay after I corrected for the wrong parameter in the model online I could get good estimates of temperature so there are many issues here in state estimation what to do how to deal with plan model mismatch I am doing all these things under the assumption that the model is perfect typically the model is not perfect and what do you do even if it is perfect in the beginning when you start your observer after some time the model can become bad see for example in chemical plants what happens is there is fouling okay some deposits take place you know what is fouling if you have used the immersion heater if you have ever used the immersion heater in the beginning it is you know very nice and no deposits but after some time if the water is bad there are deposits and the rate at which the heat is transferred to the water changes okay so there is a model plan mismatch if you develop a model in the beginning that will not work after some time you need to change the model how do you estimate the measurement noise and state noise golden question okay recently one of my students completely is PhD on this on this part how do you estimate noise from data then there are issues like observability how many extra parameters you can estimate from given set of data what if data is missing okay what if there is multirate data okay in wireless sensors packet losses occur okay in some situations for example some concentrations they cannot be measured at the same rate as you can measure temperature so there are multirate systems okay so you can modify all this for dealing with multirate systems for dealing with packet losses for reconstructing when measurements are lost all these things can be done using filtering ideas and it is a vast area of research even though the research has been going on in this recursive estimators for 50 years I do not think we have those solutions there is enough scope for doing lot more okay yes there are some fundamental results at least for linear system theory you can prove it for nonlinear case there are no proofs yes because there is a rank condition which violates if you have one measurement and if the system is observable then you can only estimate one more extra state you cannot estimate to I can give you a reference where this is of this is but how does it translate when you go to a nonlinear system is not here okay because local observability what it means in terms of a global observability for nonlinear system is not very perfectly linear system linear system you can prove this that you cannot estimate more particularly well I will just put one rider particularly when you model the extra states as an integrated white noise if it is not an integrated white noise I do not have a clear answer if it is integrated white noise that is how typically it is modeled extra states the parameter that we estimated that effectiveness parameter we model it as an integrated white noise. So we have two measurements we can estimate two extra parameters not more than that if you try to estimate in the sense what you can uniquely estimate see if you start estimating more parameters you will get some values but it is not unique the answer is not unique I am talking about unique estimation or unique reconstruction this is independent of whether it is Lienberger or Kalman filter this is a filtering issue generally this is a system property for linear systems first of all dynamic model based observers can be used to reconstruct unmeasure states from frequently measured outputs and Kalman filter of course one of the most elegant solutions when you know the noise models it can be used extended Kalman filtering can be used when the system is nonlinear and actually these are these belong to the class of so called Bayesian state estimators it is a very white class of observers the new names or the new buzzwords in this area are particle filtering and ensemble filtering and so on these are used everywhere filtering is now used in robotics it is used in target tracking radar applications speech recognition image reconstruction so actually what I taught you here is not just limited to process control or to control this is developing in soft sensor for a states that cannot be directly measured but you have a model for it how do you reconstruct how do you merge observations which are available online with a model and create a software sensor or soft sensor I should not call software sensor a model based sensor popular word to industry would be a soft sensor or software sensor for you guys it should be no state estimator on observer okay so well these are the nice references Soderstrom I would if you want to go deep into this area Soderstrom book is excellent this is recent book the book by Gelb is also one of the standard references for Kalman filtering and Astroman Wittenmark of course gives you so this brings us to end of this lectures on state estimation and I will just preempt you now with where I want to move next well where I want to move next just take 5 minutes to talk about this will start these all the lectures for the next time I am going to talk about now control so now just take a look at what we have done we started by saying you know we want to develop control relevant models we had two ways of developing control relevant one if I have a mechanistic model I linearize and develop a control relevant model which is linear difference equation if I do not have I start from data and then derive a linear difference equation states model which I can use for developing a controller then in between before we went to controller design we talked about this observer observer is using just the measurements reconstructing the state vectors okay now I am going to complete this whole thing so remember one thing when you did modeling you had an interface with the real world because model is you know trying to mimic the real world moment you come to observer and controller design you are in the world of models this is the imaginary world okay here first of all you are looking at linear models okay all linear results are valid linear algebra you can use so it is a very nice universe where you can do manipulations you can place poles you can do all kinds of things and then you design the controller in this linear space linear model space and then the problem comes is how do you implement this that is realization through hardware that is done through a computer so you go back to the real world by implementing the observer controller into the so now why I talked about observer why I talked about now controller afterwards I am going to talk about what is called as state feedback controller so state feedback controller assumes that is a state measurement available full state measurement is available in practice what is available is only some measured values but then I have shown you how to reconstruct the states so now assume that the state can be reconstructed I have state measurements available now how do I design a feedback controller if all the states were measured okay that is the next question that I want to answer okay so what is the outline of this I am going to talk about motivation then there is going to be this part I can go a little faster than observer because you will find that this is like a 2 you know mirror images there I talked about observability here I will talk about something called reachability okay which will be a very similar concept there the idea was can I estimate states from the measurements now I will talk about can I take the state from anywhere to anywhere that is called reachability okay so I will talk about reachability then we talked about pole placement observer I will talk about pole placement controller same idea transform the system to controllable canonical form there we did observable canonical form I am going to transform it to controllable canonical form and do controller design okay so just a parallel it is very very straight forward well when we went to multiple input multiple output systems we did not pursue with pole placement we went to optimization based formulation I am going to go to optimization based controller design which is linear quadratic optimal control okay so this is going to be the parallel there okay so just one to one mapping you can do then I am going to talk about this technology model predictive control which is a practical technology which is implemented in the plants and we will also see whether it is my friend who actually implements this he can give one or two lectures of actually how he does it practically on real very very complex plants so that you will get to know that how it is done and so this is complete cycle so this course you should know at the end starting from scratch even if you do not have any model you can develop model development from data you can develop state estimator combine it with a controller and then you know how the whole thing is that the entire cycle so we are not missing any point in the life cycle of controller synthesis from model development to controller synthesis we will do all that and then we will start talking about these developments so the only one slide I will show here before I close this so given the state space model and controller design is that we assume that all the states are measurable okay and design the controller okay then we design a stable estimator actually we did the step two first it is I was stable estimator and then what we do is we combine the two we marry the two so estimator with the states using reconstructed states you implement the control law okay so and then we will have what is called as separation principle where you can show that you can independently design observer and controller to be stable the joint system will be stable so that I will show so why I am working with state space because I can work seamlessly talk about single input single output multiple input multiple everything can be done under one framework okay and it is all linear algebra so it is very very easy to implement so this is what is going to be my next say three or four lectures I will complete this and if you ask me what is the next computing assignment we implement the observer and you know Kalman filter what is the next assignment we implement LQG controller and whatever we do we just go and study on that particular system okay.