 Hello, welcome to this lecture on digital communication using GNU radio. My name is Kumar Appaya and I belong to the Department of Electrical Engineering at the Indian Institute of Technology Bombay. This lecture will be a continuation of our discussion of suboptimal equalization techniques. Over the previous couple of lectures, we have been looking at zero forcing equalization. In zero forcing, what we do is that we try to come up with an equalization approach or filter that zeroes out all the interference or the inter-symbol components of the interference. The problem with that approach as you saw both in the theoretical discussion as well as our GNU radio simulation is that it can result in a significant amount of noise enhancement. This noise enhancement can prove costly especially when you have a low signal to noise ratio and therefore the zero forcing equalizer is not a good choice when you have very very low signal to noise ratio. In this lecture, we are going to look at one more suboptimal equalization approach called the linear MMSE equalizer. And here the MMSE stands for minimum mean squared error equalizer. What we do over here is that we take a statistical approach and we try to minimize the expected squared error between the detected symbol and the transmitted symbol and that essentially gives us our equalizer. Just like you may have seen in other discussions on squared error, the use of squared error gives us an easy handle to compute the equalizer because you can essentially differentiate and set the filter coefficients to zero and an equation essentially gives you your filter coefficients. That is what we are going to look at now. Now as the name suggests, we have to minimize the mean squared error. So this J is like an objective function, the J here refers to the mean squared error. This mean squared error is a function of our filter coefficient C. C has the same meaning as earlier, C essentially corresponds to a vector containing filter coefficients. And we define the error as expectation modulus C hermitian R k minus B k. R has the same meaning as earlier, it is essentially a grouping of the symbols that you are that you have received at the receiver. It is a vector of the same size as C. Now if you expand the J of C, you get this particular form. I will actually show you this in more detail, let us do it here. So we have C hermitian R k minus B k. Now B k here is a symbol, it is a complex number like you know QPSK symbol or something while C and R are vectors therefore, I give the underscore. Now C hermitian R can also be written as R comma C that is also another notation that you will see sometimes elsewhere. Now when you deal with C hermitian R squared, one way to handle these kinds of expectations is to simply write the mod square as that into that hermitian. Just keep in mind that this is a number, but we will still write it as conjugate or hermitian it is the same thing. This is equal to expectation of C hermitian R k minus B k times C hermitian R k minus B k whole conjugate or hermitian it is the same thing. I am deliberately using hermitian because that will give us a way by which we can swap this order of R and R k to get a convenient kind of representation. This is equal to expectation of C hermitian R k minus B k. Now I will apply the hermitian operator, I will get R hermitian C minus B star because B is a number, its hermitian is merely going to be the conjugate. Now the hermitian operator when you apply on C hermitian R k is the same as conjugation, you can write it as C hermitian R k the whole conjugate that will be the same as R hermitian C you can verify that. Now if I expand this, I will get C hermitian expectation. Now the reason I can take C outside is because C is a fixed quantity and is not dependent on the exact value of R. It is a fixed filter based on the statistics, but not dependent on R and B themselves. I will write this as R k, R k hermitian. Why? Because C hermitian R R hermitian C, I can take the C hermitian and C outside the expectation. Then the next I will take B and B star as a plus mod B k star. The next thing I am going to do is I am going to next take the cross terms that is I am going to do C hermitian oh I am sorry I should put an expectation here. Let me just address that. So, expectation mod B k square because the expectation B is a statistical quantity because it is random symbols and we have expectations inside. Next the cross terms. So, we have plus expectation of C hermitian R k B star minus B k R hermitian k C yeah that is it ok. So, now we have these this particular cross term this C hermitian R k B k star minus B k R hermitian C ok. If you now look at what we have on a slide essentially this is what we have written except that we have written this as 2 times real part ok. The 2 times real part comes in very handy because you can essentially write this as if you see C hermitian R k B k star the whole conjugate is B k R hermitian C k. So, I can write this as I can also put the expectation outside does not matter it is the same thing. Now the key idea for us is that this is J and if you look up how to differentiate this J with respect to C. So, if I write something like dou J by dou C is equal to 0 essentially I am dividing I am differentiating with respect to the first entry of C second entry of C third entry of C setting each to 0 and this is the vector notation. This is something you can check any linear algebra or matrix calculus reference this is going to give me C hermitian R k R k hermitian if I now call this R ok and if I now call let me just write it and then you will see then you will get 2 times R times C ok 2 times R times C and minus we will get P is equal to 0. What is this P? Now if you differentiate this 2 times real part of E power minus I am sorry this is 2 P 2 times real part of expectation of C hermitian R k B star k this is again something which you can carefully perform the differentiation you can differentiate with respect to the real part of C imaginary part of C and so on. So, what you need to do is you need to expand each entry of C as the real part and imaginary part real part and imaginary part perform the expansion differentiate with respect to the real part of the first entry imaginary part of first entry real part of secondary imaginary part of secondary and write them equal to 0 and you can show that you will get this form where P is believe this should be a point I am sorry yes where P is of the form expectation B star k R k. Now let us actually just understand this this matrix is let us say you take a group of k entries is k cross k is k cross 1 k cross 1. So, you have to solve the equation R C equal to P and if P if your R is invertible your C is equal to these kinds of equations are very common in communication and signal processing ok. You can show several properties that R has that make evaluating R very easy you know all those you know if you look at the entries of R they have certain patterns it may be it will be of a topolitz form all those kinds of things are there R inverse P if R is invertible is going to be C. Now if you evaluate dou square j by dou C square you will find that you can that the resulting the resulting numbers that you get will indicate that this C is indeed the minimum that is you are now getting at a global minima based on all values of C. Therefore, this C is also referred to a C MMSC you can choose any filters, but this C MMSC is that filter which will minimize the minimum mean squared error j. So, coming back over here. So, C Hermitian R C plus mod B k square by the way we were silent about B k square we generally you know B k square is essentially expectation B k square is ES I should have I should put an expectation here as well expectation B k square is ES that essentially does not depend on the realization. So, that derivative with respect to C is also 0. So, if we differentiate you will get 2 R C minus 2 P is equal to 0 and R is equal to expectation R R Hermitian P is equal to expectation B star R k Hermitian this is also called the cross correlation matrix. Therefore, the minimum mean square equalizer is C MMSC is equal to R inverse P where R is expectation R R Hermitian and P is expectation B star R k. Now, if you remember our model from earlier we had this particular structure where we grouped several symbols together and we were able to write in fact, we group 5 of them together for our running example and we were able to write R k as U B k plus W k. Now, for this U B k for W k let us evaluate what these quantities are. So, I am going to write R k here this B is a vector this B is a vector column vector containing you know 5 elements if you remember sorry 3 elements if you recall in the running example you had 5 rows 3 columns and B had essentially B k minus 1 B k and B k plus 1 that is what we had. Now, let us evaluate R that is expectation of R k R k Hermitian remember we are assuming that the processes are wide sense stationary and this will result in the by the by definition U B k B k Hermitian U Hermitian plus W k will have give me I will just write it as C W that is expectation W W Hermitian plus the cross terms, but the cross terms are 0 because we are assuming that W k and B k's are uncorrelated. We will assume that the W k's we will assume that the W k's and B k's are uncorrelated. So, that means that this essentially simplifies to R is equal to remember U does not depend on statistics U times and what is expectation of B k B k Hermitian expectation of B k B k Hermitian will have in the diagonal entries have B k square B k minus 1 square B k square B k plus 1 square in modulus. The off diagonal will be B k B k minus 1 B k B k plus 1 since we assume IID signaling it will be E s times I. So, I am just going to write it as E s times I where E s is what expectation of mod B k square U E s is a number plus C W. So, I can also write this as E s times U U Hermitian plus C W where E s is the signal energy. Now, if you look at what we have in the slide that is essentially what we have written over here. Now, how do we write this as the second part? For the second part all I do is I can just expand this as column wise. So, if you remember how matrix multiplication works U U Hermitian can be written as first column first column outer product the second column second column outer product and so on. I can write this as which is exactly what we have over here. Similarly, my P now B k remember R k has R k is essentially U B k plus W k and in this B k you have 3 entries you have 3 entries and you have only that one entry corresponding to the B k. So, in other words this is actually equal to expectation of now if you substitute this particular quantity over here the W k goes away because W k and B k are uncorrelated. Now, if I write U times B k U times only B k term survives it should be star sorry the rest all terms go to 0 the rest of the terms go to 0 that means my P is only going to pick that column of U that corresponds to B k. So, this is going to be we will call it U 0 the column that corresponds to B k that is think of it this way the columns of U correspond to each entry of B k minus 2 B k minus 1 B k B k plus 1 B k plus 2 only that column that corresponds to B k is going to get picked right. So, that is essentially what is happening over here. Now, the next thing we have is if we now put this together we are going to get our C M M S C to be equal to remember it is R inverse P. So, I am going to write it as this is great. Now, I am going to play a small trick ES is a number I can take it inside by just doing 1 upon ES. So, this can be written as U U Hermitian C W upon ES times U 0. Now, if your C W is essentially identity times N naught by 2 then it has a nice form. Here is the interesting part the key difference between zero forcing and MMSE is that this 1 upon ES term essentially weights your equalization filter basically weights meaning it applies a factor depending on the noise. Let us actually just play a small kind of mind game if your C W which is the noise is its entries like its variance is much, much larger than ES then what is it saying if you invert this matrix C W upon ES you are going to get something very close to zero. So, if you are in a very, very low SNR regime then the problem is that the C M M S C says ok I am just going to zero out your signal you are not going to get any information. However, if your C W is suppose you are in a very high SNR regime then your C W is going to be really, really close to zero. In that case there is an important realization this approaches Z F as let us say C W or let us say SNR tends to infinity ok. Now, this may be a little confusing as to why this is something we will see when we do it in the GNU radio as well, but the intuition is that the C W's entries are much smaller than ES then this U U Hermitian Ohl inverse turns to the pseudo inverse which is exactly what we had in the case of the in the case of your zero forcing it will become that and this is something we will show numerically also. The key however is that this M M S C offers you a slider kind of I mean intuitively it offers you a slider that slider is picked automatically as to how much weightage you give to the received signal. If the SNR is high it will just do the same thing as zero forcing the SNR is low it will say ok fine just do not invert the channel blindly because then you are going to amplify the noise will strike a tradeoff where you try to get the minimum mean squared solution maybe that works better that is the hope well let us just see how that works. So, this essentially is what we get this is what we just showed now in what sense is it better in the sense that the signal to interference ratio S I R is maximized for the linear M M S C equalizer in other words in our notation this is the signal to interference ratio how let us check. So, if you look at this picture maybe I will just yeah I will just take this picture I just take this one all right. So, now over here right let us write this in two parts this is equal to E S times let us not let us not do this ok I am sorry let us actually take the original let us take the original signal. So, you have R k is equal to U B k plus W k now let us write this in two parts this is going to be B k notice that I am writing B k without the underscore which means this is that number times U 0 plus summation over j such that j is not equal to 0 B j U j plus W k. Now, blindly speaking this corresponds to our signal power so to speak right because this corresponds to how much signal you are essentially going to get. Now, if you do C M M S C Hermitian R k I am going to get B k times C M M S C Hermitian U 0 plus summation over j such that j is not equal to 0 B j C M M S C Hermitian U j plus C M M S C Hermitian W k. Now, over here just keep a note this corresponds to signal and this corresponds to let me not write it that way interference due to the neighbouring symbols plus noise. So, if I want to compute the signal to noise ratio I must take the variance of the I mean the essentially the expectation square of the first part divided by the expectation square of the second part. So, if I want to write the expression for the S I R the signal to interference ratio I am going to just write this S mod and if I take expectation I expectation B k square is going to be E S. So, I am just going to write E S times magnitude C M M S C Hermitian U 0 square divided by again this mod B j square is going to give me E S and B j times B j plus 1 B j minus 1 B j minus 2 all have cross covariance 0. So, if you carefully write this out you will get E S summation j such that j not equal to 0 magnitude C M M S C Hermitian U j square plus. Now, C M M S C Hermitian W k if you write the expectation of that square you can write the C Hermitian M M S C W k plus W k times W k Hermitian times C Hermitian. So, this you can show easily will be C M M S C Hermitian C W C M M S C that is it. So, this is the signal to interference ratio so to speak and if C W goes to 0 you can see that the signal to interference ratio just becomes the ratio of these, but remember what happens in the case of zero forcing in the case of zero forcing your C was chosen in such a way that C Hermitian times U j for all the other use was 0 right because you chose the C that aligned with U 0, but not with the other use. So, this inner product was 0 in the case of zero forcing in the case of M M S C it is not 0, but you want to essentially take into account the impact of the noise also. So, this is the signal to interference ratio which I have written over here as well the C should be C M M S C I use the angle bracket notation otherwise it is the same. This signal to interference ratio is maximized when you choose C S C M M S C and among all the linear equalizers I am not showing this, but among all the linear equalizers it can be shown that this C M M S C maximize the signal to interference ratio it turns out that minimizing the mean squared error is the same as maximizing the signal to interference ratio. Now, the other thing is when SNR goes to infinity then the M M S C equalizer becomes the Z F equalizer. Now, intuitively it makes sense which I just discussed with you because of the fact of that those inner products if you are you know C Hermitian C W C is essentially close to 0. You can choose the C that orthogonalizes with respect to others and just has inner product with respect to u 0 and signal to interference ratio becomes you know essentially infinity, but this is intuitive we can also just show numerically also that this works and we will do that in the next lecture also. Now, one aspect that is an extension of these techniques is the concept of adaptive equalization. See in the case of practical systems right the channel impulse response or frequency response whichever way you look at it changes with time that is let us say you have your phone and you are essentially moving around the environment around you changes which means that the channel or the impulse response that your phone sees essentially starts changing. So, that means that you may require frequent retraining or recalibration. An alternative is that you can actually rather than you know just retrain get the channel use it then the channel goes bad retrain get the channel an alternative to this is that can we find some way to track this channel. Now, this tracking the channel is something where you learn a little bit about the changes in the channel as you go on. So, for example as you are walking while taking a call with your friend the phone essentially starts just learning the changes in the channel as opposed to learning it afresh. This kind of approach is called an adaptive approach it can either be training based where the base station or the transmitter sends some information to your phone or to the receiver using which it can find the changes in the channel or it can be blind blind means if the receiver knows only what data is being received, but just does not what let us say what modulation or what you know what kind of time what data rate it is, but does not know the exact sequence which is there there is no training it can still try to find out what the signal is. The adaptive approaches can be either training based or blind there is a good mix of both in practical systems you can check those out. Some examples of these are the so called LMS or least mean squares algorithm then there is recursive least squares and then there is something called decision feedback equalizer all of these build on top of the equalization technique that we have just discussed so far and these can essentially be employed to not just equalize the channel, but continuously adapt the equalization so as to improve the performance and not necessarily be stuck at you know one particular channel and then retrain and so on. So, they are very very efficient reduced in several standards as well. So, to summarize our discussion of equalization so equalization is necessary whenever your impulse response affects the symbol detection. Now, the optimal strategy is to use the maximum likelihood sequence detection, but this may be prohibitively complex because you know when you have a very very large number of samples on which you want to essentially start detecting the exponential search is just impossible of course as you saw the Witterby algorithm signifies sorry simplifies the evaluation significantly, but again you may or may not be able to implement the Witterby algorithm because that also requires tracking of state having some channel related you know changes and all those kinds of issues exist. So, the one approach to not have not do the optimal thing, but hope that it works is to use a suboptimal approach in this case of course you can use zero forcing MMSE or a host of other approaches to perform the equalization they are much simpler to implement than the Witterby algorithm, but there is a trade-off because you will suffer in terms of performance especially if you have a in the lower SNR regime these will not perform very well when compared to the optimum maximum likelihood sequence estimation approach. Finally adaptive equalization is at an extension of all these techniques wherein rather than just train learn the channel once you essentially have an approach where you train and then when the channel changes you try to learn the changes in an incremental manner it's much more efficient much more effective because you can just learn on the go as opposed to stop train redo stop train redo and these are a combination of these are used in several standards and implementations as well many of which you are using every day. So, this was our discussion on equalization in the next lecture we will implement the maximum minimum mean squared array equalizer on GNU radio and see the differences between zero forcing and then move on to other topics such as OFDM wireless and so on. Thank you.