 Hello, so welcome to this lecture on digital communication using GNU radio. My name is Kumar Appaya and I belong to the department of electrical engineering at IIT Bombay. In this lecture we are going to look at demodulation and cover some of the preliminaries and gain some of the tools that will be useful when we perform demodulation that is to get back your symbols from the waveforms. Also as an introduction you can recall that modulation is the conversion of messages into signal waveforms. So, as you are aware in the case of digital communication our aim is to communicate a finite set of messages and these messages have to be converted to waveforms and these waveforms as you have seen are designed with bandwidth and power constraints. The reverse operation is demodulation wherein from the waveforms you want to infer or conclude as to which message or messages were sent. So, the task of recovering messages from received signals is demodulation. Here the tricky part is that since we have waveforms and these waveforms can be very complicated and they can actually undergo variations. As an example in the case of let us say amplitude modulation let us say that you send one of 1, 2, 3 or 4 levels, but these levels are actually translated to some waveforms. So, for example you can have this is one level this is another level this is another level. Now because of noise actually what happens is that these things get little bit affected and they do not resemble the original waveform. So, the question is when your waveform undergoes modification under channel noise or other distortions what do we do? Remember that this is a very practical and typical effect that you have to handle as a communication engineer because the environment in which you are going to send your waveforms is definitely going to cause some modifications. The other thing is that whenever you receive your signal at your receiver or your cell phone or your laptop or wherever it is it is definitely going to be affected by noise. So, the challenge which we are going to consider in this lecture are to find out what are the optimal methods to recover messages with minimum errors. So, we are silently introducing the aspect of error also that is it could so happen that you send a particular symbol, but because of the variations caused during the transmission or during reception it could be misinterpreted as some other symbol. See after all you are sending a waveform you are receiving a waveform what if the received waveform is so far apart or so different that you actually conclude that it is a different waveform you will end up making an error minimizing these errors and ensuring that you get a reliable communication is part of what we call the detection problem. The detection as opposed to estimation refers to the fact that we want to conclude which one of the finite messages you have sent. The ingredients that our toolkit for demodulation has are basically random processes because both you know largely we are going to consider noise here and noise is a random process and it has certain spectral properties that will come in handy. The other is hypothesis testing wherein you have to find out which one of these particular hypothesis is most likely given the observation which is actually another form of or another presentation of the detection problem. Now let us revise some basics of the Gaussian random variable. Before we actually get to why I am straight jumping straight into the Gaussian random variable the reason is because Gaussian random variables appear naturally as noise sources and you know some channel modification effects as well which we will cover later primarily due to the central limit theorem. Now if you may recall the central limit theorem essentially says that if you have several IID random variables like X1, X2, X3 and so on. So if you add these random variables then the resulting random variable is actually Gaussian irrespective of the prior distribution that your X1, X2, X3 had. For example you can take X1, X2, X3 as uniform and add 10 independent and identically distributed uniform random variables and if you actually plot the histogram surprise, surprise you will get a bell shaped curve. So whenever you have variables which appear as a combination of or as a sum of several independent identically distributed random variables it is more or less accurate to consider them to be Gaussian and this is confirmed by experimental observations also. Now since we are going to consider noise significantly in this particular lecture let us also have an idea as to why this thermal noise or circuit noise is actually generally treated as Gaussian. So if you look at something like a wire or you know whatever you know medium that you are going to consider for sending your signal typically if it is something like a wire where you are sending signals as voltages and currents the carriers are electrons. So you have these little electrons but these electrons are going to be thermally agitated that is because of the effect of the heat and you know the effect of heat on these electrons there is a random process which happens. Some electrons are going to essentially jump or you know go faster or slower and get missing and things like that that is they essentially superpose some variations onto the current or the voltage that you are going to observe at the receiver. So since these little tiny variations are all treated as independent it is a reasonable assumption and they all superpose that is they add up and because of that you can assume that the thermal noise at the circuit level is typically Gaussian. And this is also borne out by experiments that is simple thing you can do is connect a wire just from ground to the oscilloscope in your lab and we then just make the oscilloscope go to the you know most you know like the least possible vertical resolution and take those samples and then transfer it to your PC and histogram it you will get a bell shaped curve. So circuit noise is typically Gaussian. Now the PDF of a Gaussian random variable with mean mu and variance sigma that is this is the mean mu and the variance is sigma square. So this is something you have to be careful about the variance is sigma square standard deviation is sigma the mean is mu. This is the PDF it is 1 upon sigma root 2 pi e power minus x minus mu the whole square by 2 sigma square by the way here x is a real number. So the notation which we typically use is n mu sigma square indicating that we have a Gaussian random variable that is this random variable is distributed in a normally distributed that is why we use n or Gaussian mean is mu variance is sigma square. Now of course the PDF you would have seen is bell shaped the median for a Gaussian it is interesting that the median which is where the 50 percent distribution lies is mu. The mean is also mu the mode if you do not recall what a mode is the mode is where the probability density function essentially has a peak this is called a unimode unimodular distribution because it has a single peak and this peak is at mu. So the median mode mean all for a Gaussian are mu. The other thing is that the variance the sigma square controls how fat your Gaussian is that is how much variability there is if the Gaussian is narrow then many if you start generating samples of this Gaussian that is if this variance is small you start generating many samples of this Gaussian they will be very close to mu but if your sigma is large that is your variance is large then you will have variables all over the you will have the random variable realizations all over the place and a wide variance signifies that there is a wide variation you can see. So it is very typical to as you know to say that people typically say that most of the Gaussian is concentrated within mu plus or minus 3 sigma ok. If you calculate the probabilities that will be close to 99 percent and so on and you can just do those some people say 5 sigma, 6 sigma and so on but mostly it is concentrated within let us say 6 sigma or you know width of interval with centered around mu. So that is how a Gaussian looks like. Some other basics of course these some of these are true for random variables with any distribution affine transformations naturally preserve Gaussians this is not true for all random variables of course but still for Gaussians it is true. If x is distributed as Gaussian with mean mu and variance sigma square y is equal to A x plus b then y is distributed with mean with as Gaussian with mean A mu plus b with variance A square sigma square. So this is not difficult to see because if you look at expectation of y that is equal to expectation of A x plus b which is A mu plus b. Now similarly for variance if you look at the variance of y that is equal to the variance of A x plus b and I hope you recall that any addition of any constant does not affect the variance. So this is variance of A x and multiplication by a constant essentially results in the variance getting squared and this results in A square times variance of x which is A square sigma square ok. So be aware that you must be able to quickly do these transformations the reason is because often when we analyze problems related to detection you may want to scale you know if you have y is equal to you know x plus noise or something you may say oh let me multiply everything by 10 so that it becomes easier. So then you have to make sure that all the distributions are scaled accordingly sometimes you may subtract some value so then you have to adjust the mean. So make sure that these affine transformations and these computations are very clear. Another thing is this conversion to standard form. So this N01 is called the standard normal distribution or standard Gaussian distribution. That is it is a Gaussian random variable with mean 0 and unit variance that is the variance of this random variable is 1. Now the key idea is that any Gaussian random variable there is an affine transformation that will convert this to standard normal. What is it? It is essentially x minus mu by sigma. So why does this work? So you have the result on top ok this is actually like if you want to say use this y is equal to ax plus b you essentially have a as 1 upon sigma and b as minus mu upon sigma that is what you have right because it is x by sigma minus mu by sigma. So that is a is 1 upon sigma b is minus mu upon sigma. So it is very evident that a mu plus b ok so that a mu plus b is essentially mu by sigma minus mu by sigma which is 0. Similarly let us also look at the variance let me erase this. So if you have as 1 upon sigma bs minus mu upon sigma. Now if you want to calculate the variance like I mentioned earlier this b does not play any role in the variance. So we can essentially ignore this a is 1 by sigma and if you just say a square sigma square that is sigma square upon sigma square which is 1. Now this particular conversion to standard form used to be quite popular in the olden times when we did not have means of computing the Gaussians in a very easy way. So what would happen is that there would be essentially a booklet or table with lots of these Gaussian probabilities encoded in them and you would essentially have those probabilities encoded only for n 0 1. So suppose you had a mean of 10 and a variance of 25 you would then convert that whatever Gaussian probability wanted to find out for a standard normal by transforming those variables and then just looking it up on the table. So this used to be popular in the olden times of course today computers can essentially calculate these things very very easily. Another important aspect that we need to be comfortable with is the CCDF that is the Complementary Cumulative Distribution function of the Gaussian which we also for convenience call the so called Q function. This Q function appears very frequently in the context of digital communication because calculation of symbol errors and bit errors often involve computations over Gaussian of the form this that is probability that your value is larger than you know. So as we will see shortly or maybe in the next lecture or so you typically have to demarcate your received symbols into decision regions ok so or decision intervals. So an error happens if your received value falls outside an interval. So decision interval is something like saying z is greater than x and z is greater than x that is just you are integrating and in this case I am using the standard normal. Standard normal is often used because you can always normalize all your computations to make it have the noise to be something like n01 that is something you will see shortly. So P of z greater than x this is the Complementary Cumulative Distribution function for a unit variance 0 mean Gaussian random variable is integral x to infinity e power minus t square upon 2 by root 2 pi dt naturally you can see that this is the same as the Gaussian PDF except that you have substituted mu as 0 and sigma as 1 and this for a standard normal is the Q function. Remember one thing the Q is always defined only for a standard normal distribution not for any you know anything which has nonzero mean or a variance other than 1. Now even this Q computation actually is you can see is a you know it is an integral so you will have to compute it numerically or you know do a look up a table like I mentioned in fact if you are interested in looking at the function in MATLAB octave and you know python and all those you can look for this erfc function erfc is not a precise in the sense precisely the same function because erfc I think does not have some scaling like you know the 2 and 2 pi so you will have to do some very minor mapping but the erfc does this computation. So this has been implemented for you in many computational software so you can use those each software has a different way of implementing it some of them do a computation some of them do a table look up and interpolate between values and so on but if you really want to just do a quick back of the envelope calculation and approximation you can get some reasonably tight bounds for larger x ok. So these bounds are very very useful because you know some sometimes when you are as a designer you want to just get rules of thumb and you want to you know just get quick values you can use some approximations to estimate this Q very very easily. For example 1 minus 1 upon x square times e power minus x square upon x root 2 pi and is one of the you know like Q of x is always larger than this and Q of x is always smaller than e power minus x square by x root 2 pi. You can see that these two are the same and this just slides slightly lower so you can see that when x starts becoming values like 3, 4, 5 and so on then this becomes a very very tight range of values within which you can sandwich Q. So Q is essentially approximated by using these. Now it is worthwhile to see how these can be done you do not need anything more than elementary calculus to do these kinds of computations. So for example I will show you how you can get one of these bounds. So you start with Q of x ok that is the definition is that it is remember it is the complementary cumulative distribution function for a standard normal. So a standard normal is e power minus x square by 2 by root 2 pi I will write t because I am integrating from x. So I will write e power minus t square by 2 by root 2 pi it goes from x to infinity that is to visualize it is this part though I should not remember I am so sorry Gaussian cannot go to 0 this is standard normal ok yeah it is it is this part that part ok always have that picture in mind it is the part to the right of x ok. Of course you can draw x to the left also I mean I agree but so in general you always take x to be a large positive value so it is the right of x. So integral this integral you play a trick you multiply and divide by t you multiply and divide by t should not be a problem because you know even if you say oh what happens if my integrating you know my integration limits have this 0 within it no problem still 0 by 0 does not cause any problem because there the limit is 1 so it does not matter. Now if you play this trick you can just write this as d of e power minus t square by 2 upon t root 2 pi and you must remember integration by parts that is integral u dv is uv minus integral v du. So u times v in this case is minus e power minus t square upon t root 2 pi you write x and infinity over here that is you substitute those values minus v du in this case v is you know like this you write this as e power minus t square upon t square root 2 pi ok. The reason is because you have this 1 by t and this t that is why you get 1 by t square you can check this out. Now if you substitute over here in the case of infinity there is no problem it is 0 in the case of x it will be e power minus x square by 2 upon x root 2 pi which gives you this part minus integral x to infinity e power minus t square by t square root 2 pi dt. Now you can clearly see that e power minus t square by 2 by t square root 2 pi is a non-negative function therefore for any value of x this integral will be a positive number since you are subtracting a positive number you can bound this by writing less than or equal to e power minus x square by 2. So this gives you the left hand side bound over here for the right hand side so sorry this gives you the right hand side bound rather yes this is the right hand side bound for the left hand side bound it is not very difficult all you need to do is you need to just continue over here you multiply again by t over here and t over here and if you perform this integral you will get something and you will get something plus. So you can actually write the greater than or equal to sign and you will get that 1 by x square very beautifully. So once if you keep going by parts again and again you will start getting tighter and tighter bounds of course typically we just restrict ourselves to these two bounds that is the most useful of course emphasizing that these are useful when you want to do hand calculations you know but with a calculator but typically nowadays computers can evaluate these very very efficiently. The next thing that we have to focus on are collections of Gaussian random variables. So often in digital communication we have to deal with collections of random variables for example suppose that you are getting multiple symbols at once and for some reason you know there is some relationship between these or some noise related correlations are there and all those kinds of things can happen. So we do not deal with Gaussian random variables or random variables in isolation we also deal with collections of random variables. In the most general form whenever you have a collection of random variables we group them into a vector a column vector typically and call such a vector a random vector. Typically of course there is no need for x1, x2 to have the same distribution but typically it is convenient for you to group together random variables that have the same distribution not just same in same way but the same distribution of course there may be correlations and there may be of course there may be minor variations for example x1 may be Gaussian with mean 1 variance 2, x2 may be Gaussian with mean 1 variance 3, x3 may be Gaussian with mean 10 variance 50 and so on. Those variations can be there but typically we group together Gaussians in a Gaussian random vector. One important definition or a characterizing aspect of random variables is a so called covariance matrix. This covariance matrix Cx is defined as expectation x minus expectation x minus sorry times x minus expectation x Hermitian. So almost like you want to take the outer product. So x is a column vector x Hermitian is a row vector this is the outer product. Now this covariance matrix is a very important quantity that tells you a lot about the joint relationship between the entries in the random vector x that is Cx is actually an n cross n matrix whose ijth entry is the covariance between xi and xj and the diagonal entries the ith entries are the variance of the corresponding x i's. So what does this mean? So effectively if you look at the off diagonal entries of C, Cx ok these are the variances and this particular entry is the covariance between xi and xj that is the expectation of xi minus mu i times xj minus mu j. So this is the covariance. Now covariance of course you would have seen in your past courses but to give you an intuition covariance measures both the variance of the individual random variables as well as their correlation. So how does that work? So if you look at the next one there is a Cauchy Schwartz inequality that says the entries of this covariance matrix have to satisfy mod Cx ij should be less than or equal to root Cx ii times root Cx jj ok. This is something which Cauchy Schwartz inequality says there are many ways to prove this but more importantly if you look at Cx ij divided by root Cx ii root into Cx jj that will be the correlation coefficient it will be a number between minus 1 and 1 if you remember the model will be a number between minus 1 and 1 and that gives you the correlation coefficient between xi and xj. Now this is important because x obtaining correlations tells us that how much we know about one particular random variable if we know information about the other. Now in a very similar fashion this particular Cauchy Schwartz inequality also places some constraints on the kinds of entries that can go here. You cannot fill in arbitrary entries and just hope that this becomes a covariance matrix. In other words this particular covariance matrix must satisfy the fact that if you have fixed diagonals the off diagonal entries must satisfy this particular condition ok. So in fact there are some other properties these covariance matrices are by definition you know if for real random vectors they are symmetric matrices for complex random variables they are Hermitian symmetric. You can just put a Cx Hermitian over here and you will find that if you just use the relationship A B the whole Hermitian is B Hermitian A Hermitian you will get the same Cx back. So there are many many properties of these we will explore all those as and when we get to them but for now a covariance matrix characterizes both the variance of the random variables in the random vector and it also characterizes the correlation coefficient. The correlation coefficient is something which is very useful in deciding how the x i's and x j's are related. So let us look at collections of random vectors. So when we look at x in this collection of Gaussian random vectors let us say x is distributed as normal and we use the notation Mx Cx where Mx now is a column vector Cx is a matrix. So let us say x has n entries Mx has n entries Cx is an n cross n symmetric or Hermitian matrix. Now once again if we perform an affine transformation let us say y is equal to Ax plus B then y is just going to be distributed again as Gaussian with mean A times Mx plus B and covariance A times Cx times A Hermitian. So I think it is evident at least the mean part should be evident you can just put an expectation over here and get A times expectation of x plus B that is A times Mx plus B the covariance must also be very evident but if you really want to just do it if you write y minus expectation of y Hermitian sorry Hermitian should not go here times y minus expectation of y expectation of this Hermitian then if you just substitute for y as Ax plus B then over here A Mx plus B comes in. So what you will get is you will get expectation of A times x minus Mx times x minus Mx Hermitian A Hermitian and if you bring the expectation inside you will get A Cx A Hermitian. This is something which you can verify it should not be too difficult. So this is the corresponding affine relationship for the case of Gaussian random variables or the case of the Gaussian random vectors you know like this is the analogous relationship that you found in the random variables case for the random vectors. So let us before ending let us look at some special aspects of Gaussian random variables. If you are given the mean and the covariance the distribution is fully specified like that is okay I mean this you know that is something which is special for Gaussians but it is oh it is something which makes sense. So for example if you look at fx of x1, x2 up to xn joint distribution of these x1, x2, xn which are Gaussian random variables its joint distribution is going to be 1 by root of 2 pi power n determinant of Cx times exponential of minus of x minus Mx whole Hermitian Cx inverse x minus Mx whole Hermitian that is this is something which is you know special this is again the matrix or vector version of the corresponding Gaussian this is like instead of 2 sigma square you have a Cx inverse over here you have x minus mu, x minus mu on both sides this should be no Hermitian here sorry x minus Mx times x minus Mx both sides. So this is the Hermitian should be here and not here apologies yeah so that x minus Mx Hermitian Cx inverse x minus Mx gives you a number. So this is the joint distribution of Gaussian random variables. So there are some couple of properties that is independent implies I mean independent is same as uncorrelated and correlated implies independence and joint Gaussian is preserved and refined transformations these are a couple of properties. So this and some more properties of Gaussian random variables we will cover in the next lecture. So what have we seen so far we have seen that Gaussian random variables have some special properties and especially when you look at Gaussian random vectors then the distribution is specified when you specify the mean and covariance in the next lecture we will cover a little more about the relationship between these jointly distributed random Gaussians as well as hypothesis testing. Thank you.