 Let us continue our discussion on convergence of sequence of random variables. We have talked about different notions of convergence. In the last class we looked at Cauchy criteria. What is the other thing we looked in the last class? Other than Cauchy criteria. So we looked at some other means of other conditions or other properties that one can verify to check whether a sequence convergence to a sequence to a limiting random variable, right? One was this correlation property of the random variables. So we move on now, like we want to now study fine. We have a sequence of random variable whether they converge to some limiting random variable. So often instead of looking at the random variable itself, we may be interested in looking whether the means of this random variable converges to some value. So when I look at the means of each of these random variables, right, I will end up with a deterministic sequence. After taking expectation of each of this random variable, I will have a deterministic sequence and it converges. So and if it converges, now I want to understand when the mean will converge or are there any property that will help me to infer from the properties of convergence of the random variables in other what we have already studied like convergence in probability. So let us say that I have xn converging to x in probability, okay? So what does this mean? This means basically limit as, sorry, so this basically we said the probability that is supposed to 0. Now suppose if this happens, can I say that expectation of xn goes to expectation of x or what I am basically asking is, so what is x here? Expectation of x is nothing but expectation of limit as xn and this limit is involved in the probability sense whether this happens or what I am being seen crossing whether this is the same as limit as infinity to infinity of expectation of xn, right? When I said this, this is what exactly I meant, right? Whether expectation of xn converges to expectation of s is basically saying whether expectation of n as n goes to infinity equals expectation of xn, yeah? Yes, we have said but we are doing this, we are asking this question, right? Whether this holds? We are not right now saying that this is true, we want to see whether this holds, if at all it holds when does this hold, okay? So when we studied some examples, when we are studying convergence of random variable in probability sense and the mean squared sense, we came across an example where it said that it converges in probability but it did not converge in mean squared sense, right? We had an example like that because convergence in probability does not always imply convergence in mean squared sense. That happened in that example because it so happened that even though the example I had there, xn's, they are putting values on a smaller and smaller intervals as I put n goes to infinity but their amplitude was also exploding. So you remember like we had this example where, so it was like this and this was some sequence, what was we called it an. As n increases, this was putting on a smaller, smaller interval but it was amplitude is also increasing, right? And we had argued that unless a n grows much smaller than n, this guy will diverge. I mean the mean squared error will diverge and this will not converge in the mean squared sense. So similar behavior happens here, like that is what we want to capture. If you are looking at the expectation of the random variable, even though it may be putting mass on a smaller and smaller interval but it may take a very large value, right? Even though because of that it may converge in probability sense in this fashion, the limit and this may be almost same everywhere except for a small interval differing by epsilon or like they may be different at only some small interval but on that interval my xn may be taking a very large value because of that expectation could be very large there. And in that way we cannot say that always this kind of behavior happens. To make this more concrete, let us look at an example. So let us say u is a random variable. Now I am going to come, so let us say I have random variable u, I have a sequence of ai's which are events and these events are such that their probability goes to 0 as n goes to infinity. And I am also going to look at this sequence bi. These are another set of sequence of numbers, some numbers that are given to me. Let me see, so and I am going to assume that they are nonzero. So this is I am starting with and I am going to define a sequence of random variable based on this in the following fashion. So this u does not depend on any n, it is just like one random variable u. I am going to define for each n a new random variable which is the sum of these two random variable u and this bn is the deterministic quantity which is coming from this given sequence and it is multiplied by this indicator function which depends on an. And an is the event that I am assuming it will be such that its probability goes to 0. Let us say I have such a setup. Do you understand what I mean by indicator of an? Fine. Now, if I am going to look at its expectation, what its expectation is and I will also assume that with finite mean plus what is this, what will be if I take expectation, this will be bn into, it is going to bn into expectation of the indicator function. What is then? It will be probability of an. So it looks like its expectation for any n is going to look like this. So now let me ask this question. I have a sequence of random variables like this. Does this converge in probability? So to verify that what I need to do? Probability that what you guess it should be converging to, if at all it converges? u. So let us say our guess is, let us assume I am going to take this to be u. What is this quantity? If I can show that as n go to infinity, this probability goes to 0, then I know that xn converges to u in probability sense. Now, check that or see that this probability is same as xn is not equals to u. See when I say xn is not equals to u, that means the difference is going to be some nonzero quantity. So it could be either greater than epsilon or it could be much greater than that. That is why this is going to be an upper bound on this phi so far. Now, come back to this. Now what is the probability that xn is not equals to u? It is going to be, so probability that xn are not going to be equals to u, that is enough that if this quantity happens to be positive, then this is going to be different. And when this is going to be positive, then this is true. And what is that probability? That probability is nothing but the probability of event an. So the way I have constructed this, xn not equals to u is nothing but probability of an. So the probability that they are not differing is that this quantity is going to be positive, this quantity is positive is nothing but this an has to be, this indicator function has to, this condition of this indicator function has to satisfy and that is going to be satisfied probability an. And what is our assumption? We have said that I will come up, my assumption is that this an is such that this guy goes to 0. So then we, what, so what we have shown then? So what we have just shown is that xn converges to u in probability sense. Now let us come back to this. So if xn converges here to u in probability, fine. Now I wanted to ask the question, can I also do this and can I also say that expectation of xn converges to expectation of x, x in this case is going to be u, right? Because that is the limiting distribution. But here if you focus on this, even though this probability of an's are going to 0 as n goes to infinity, I could choose bn's such that this product does not go to 0, right? So in that case this expectation of xn need not be the same as expectation of u as n goes to infinity, right? They can be different. So as you see that even though xn can converge to some distribution in probability, but that does not automatically allow us to take expectation of the sequence to be the expectation, the limit of the expectation of the sequence to be the expectation of the limiting random variable. So then the question now is when does this happen, right? We want to now look at the conditions when does this happen. So an indicator of an event, expectation of indicator on an event is what? It is the probability of that event itself, right? How can you write this? Just write it into 0 and just expand it so expectation of this. Only the one term remains, the other term is 0 and the one term that remains is exactly this, 1 into vr of a n. This one, let us look at this event, xn not equals to u, right? That means the difference between them is not 0. It is other than 0. It could be positive or negative. When you are going to look at the absolute value that is going to be, the difference is going to be greater than 0, right? What you are saying all you are asking is this difference is going to be greater than 0. That could be either greater than 0, greater than epsilon or it could be much more than that. Exactly. So this event is going to be subset of this event. That is why I am going to get an upper bound. So then I am going to now state couple of results which actually helps to do this kind of interchange. What I mean by interchange is exactly this. The first one is called bounded convergence theorem. See, some of these results you may not be able to directly relate it to where they come in practically. But some of these results are useful to state some results. Some of that you are going to see later in the class today. That gives us some intuition about some of the practical things. So these are like some intermediary building steps that we need to understand to understand some larger result. For example, later in the class today I am going to talk about central limit theorem. That is one of the important result and which has many practical implications. So to understand that we have to understand these results. So at every result do not try to connect it with the practical things then you may be lost. But just try to follow what we are trying to define and then eventually where we are going to connect it. Let this be a sequence such that there exists some L. So this theorem tells you such the first condition. It says that suppose you have a sequence and such that you have some L which is finite and all my random variables, their absolute values is bounded by this random variable with probability 1. That means then convergence in probability to a random variable x implies that their expectations of the sequence of random variable converges to the expectation of the limiting random variable. So this is kind of this is a very intuitive statement. So that is why it is called bounded convergence. What you are saying is your random variables are always bounded. That naturally means that even your limiting random variable cannot be unbounded. So if you have everything bounded then there is no way that in the sequence you will incur a case where something explodes. So because of that there is nothing observed behavior when I have this boundedness and everything goes well. Let us look to prove of this. So when I make this kind of assumption that all the random variables are bounded like this where I made such a such case again. Did I use such an assumption earlier also? Where? Yeah, so in which type of convergence I used it in mean squared sense. When I had a mean squared sense I wanted all the second moments of all the random variables to be bounded there. So if I have a random variable like this and if L is finite and if I take the second moment will it be still bounded and that limit is going to be what? Instead of L it is going to be L square. So this condition and I had when I said convergence in probability implies convergence in distribution. But convergence in probability does not always imply convergence in mean squared sense. But I said that convergence in probability implies convergence mean squared sense under some condition. What was that? Yeah, so if my sequence are bounded by some random variable. So here is it, can I say that here convergence in P already implies convergence in mean squared sense? Yes, right, because that y is nothing but L in this case. And we have already said that just L square, L square should be finite. Right, fine. So and we have already said that so under this condition we know that if xn converges to x in probability it already converges in mean squared sense. And in the mean squared sense when I discussed it I already said that the limiting random variable the second moment is already finite. That was a consequence which I asked you to verify. I do not know you guys did. So that came from triangular inequality. So by the same logic we can also say that here this x is going to be bounded. Its expectation is going to be bounded. So this is also going to be bounded. Now let us understand. So you can verify that I am just leaving you this that x is going to be also going to be L for probability 1 whatever that limit is. After this let us focus on this inequality. Take any epsilon greater than 0. Now my claim is this x minus xn I can write upper bounded in this fashion. So this is the main inequality I need to make this final claim. So let us see we understand this inequality. So take an epsilon greater than 0. If it is such that this x minus xn this absolute value is greater than epsilon greater than equals to epsilon if this is true or like let us say this is not true. That is this difference is upper bounded by epsilon. This term is already 0 and then this condition is already true like x minus xn is upper bounded by epsilon that is what I have already assumed. In the other case where let us say this holds where x minus xn is greater than epsilon it is going to be at least any pi epsilon it is going to be plus something else and what is that something else we can always write x minus xn as upper bounded by x plus x of n and both of them are upper bounded by 1 with probability 1. So I could write them as upper bounded by 2L with probability 1. So just verify that is why I could write this will not be greater than 2L at most. I mean see like this x minus xn I could have already written this is upper bounded by 2L but I am expressing in terms of this epsilon because this gives me a tighter bound that is if this condition holds yes 2L is there 2L is there if this condition does not violate then epsilon is the bound that is our tighter bound. Anyway so I have this inequality here. So now let us see what I want I want to now I want to find expectation. So what is the meaning of expectation of xn goes to expectation of x that means if I am going to look at the absolute difference of these expectations this should go to 0 as n tends to infinity or alternatively what I can show f if this is upper bounded by epsilon for some n which is sufficiently large for all then after some point. So now fine this is there this I could write it as expectation of x minus xn why I could do this because of the linearity of the expectation and now I could do that now I have taken this absolute term inside the expectation is this true because the differences I have made absolute so my expected value is going to be larger fine now I am going to bring in this bound which I have applied here and what is this is going to be so if I use this bound this is going to be epsilon plus 2L and expectation of this indicator so that is going to be now I know that this guy goes to 0 this probability goes to 0 right and I know that because of that then I can say that there exists some n naught such that for all n greater than or equals to n naught I can replace this guy by maybe there exists enough for all n greater than n I can write it as this is going to be less than or equals to I can always write it as so you give me epsilon I will take epsilon by 2L and I will look after which point this guy is going to fall below epsilon by 2N I can always there always exist such an n right so now you should do that for sufficiently large n this guy is going to be I can upper bound by 2 epsilon this I know this guy goes to 0 as n goes to infinity whatever epsilon it is for whatever epsilon you again it goes to 0 now what I will do is whatever epsilon you give me I will be looking at epsilon by 2L and I know that after some n naught this guy is going to be below epsilon by 2L it is because this guy is going to 0 right so I will just plug in this quantity here for all n which are greater than this n or 0 then it becomes 2 epsilon it is just like yeah you can take anything I just want to bring it to simplify in all this 2 epsilon now it is clear right like now I am more familiar territory now I can say that okay I have shown that this difference is upper bounded by 2 epsilon for all n greater than n naught and this n naught is a function of epsilon and now this if this holds if I can do this for every epsilon then what is that what does this imply this epsilon is arbitrary right so then this is exactly the definition that this xn converges to x so this implies so I am saying that this implies this are you convinced or not so what I could show okay now think of this as some this was an a n sequence and this is the limit what I have shown is a n minus a is going to be upper bounded by 2 epsilon for n sufficiently large and this I am doing for any epsilon that you have given to me so that means that is what the definition of the limit right a n converging to a and I could do that because I am exploiting the fact that I already know that xn converges to x in probability I am just using that convergence to claim that this guy converges so fine so this is one simple thing like if I know that my bound random variables are bounded and if that may and my random variable convergence in probability then I can do this interchange of the expectations yeah so what is the other things when is not necessary it is not only if and only if condition if this holds then we say we can do this but if we can do this at all I do not know this it is that always implies that such a thing implies so let us say these other things which tries to generalize this bit see when I try to state this I wanted this xn to be bounded right and this bound was a deterministic bound but you can relax this and say that instead of the deterministic one I can have this bound with some using a random variable so l I can replace by random variable y if such a y exists which always dominates xn then also it is possible so let me make that so this is called so this is just a slight generalization of this bounded convergence instead of boundedness now you look as domination we are going to say that this sequence are such that they are all dominated with probability 1 by random variable y and this dominating random variable is such that its mean is bounded and then if I have this sequence converging to some x infinity in probability then I can do this interchange okay so it is same as this except for the fact that I am replacing this l by this y which dominates the sequence but that why I will further want that it has a finite expectation so if I can find such a random variable which has finite mean and it dominates my sequence and then I can interchange this limit if my sequence converges in probability so we are not going to prove this but just take this like we should understand how to apply these results so before you apply you should better check all these conditions so the last one called monotone convergence theorem so this is as the name indicates this is an analog version of monotone convergence we already come across deterministic sequence right so what we know about a deterministic sequence when it is monotonically increasing so let us have a sequence deterministic sequence let us talk about deterministic case if it is always increasing like a of n plus 1 is going to be greater than or equals to a of n what happens to limit a n the limit exists it converges it depends on n y so any sequence which is monotonically increasing necessarily converges right either to a finite one or it could be like even the limit could be unbounded it at least it does not diverge right so if I have a monotone let us say I have let us say these are a 1 a 2 1 2 3 4 like that right my and this is my sequence if my sequence is like this increasing it is increasing increasing that either it blows up and goes to infinity or like it saturates at some point right so it always converges so similar analog so this is the convergence in the extended domain so we allow the limit to be infinity as well so it is not necessary that when we talk about convergence the limit has to be always finite even the limit could be infinity right it is just like that it is just that it does not happen that while it is going going like at some point I come down so this violates the definition of my convergence right so if it is always increasing increasing at some point either it goes to infinity in that case it is infinity is the limit or it saturates at some point then the finite in that case that saturation point is the my limit okay let me take this I have this n n greater than or equals to 1 this is a monotonically increasing sequence right where does this converge conversion this is the extended notion of converges yeah no this is the deterministic case also even I am giving you a deterministic case here right I have this n why like this sequence a n this a n is simply n this is converges but it converges to infinity okay fine so I am now defining this sequence in this form so I have my random variable sequence such that if you are going to fix a sample point on that sample point this forms a monotonically increasing sequence okay and then define my limiting random variable for each omega to be the limit of this so now this xn of omega is a deterministic sequence right and since this xn of omega are monotonically increasing this indeed exists maybe it could be limit could be infinity what is the term for that with possible value of infinity so if this happens then I can always say my expectation I can interchange like this and here I really do not need to first check I mean I do not really need a convergence in probability if this monotonicity property holds that is it like if I can check this then I can always yes but a provided that sample point has non-zero probability if that omega has 0 probability even if it is infinity I mean that become bit we have to define right like this has infinity but its probability could be 0 but in that cases we have to define its value to be the product of 0 and infinity to be 0 and then we can continue with that okay so fine so these are the three theorems called bounded convergence dominated convergence and monotone convergence you should know and understand when you can use them.