 We were discussing hierarchy of convergence. So, we were talking about various notions of convergence and which of these notions imply the other and so on, hierarchy of convergence concepts. So, the main result we had was that almost sure convergence and earth mean convergence they both imply convergence in probability and convergence in probability implies convergence in distribution. So, generally none of these none of the opposite implications are true in general none of the reverse arrows hold generally speaking. However, if you have so in this for this reverse arrow for example, does not hold generally, but if your limit is a constant then convergence in distribution implies convergence in probability. So, if the limit is a constant x is a constant then these two notions are equivalent. Then we also proved this bit. So, we are done with that bit we are done with that bit we give a counter example to this. So, you can construct a counter example to this it is actually fairly trivial we will do it and then we constructed a. So, this proof we have done. So, almost sure implies convergence in probability we will do today. So, it is a non trivial proof and the reverse arrow is not true also we have given a counter example. So, I have to give you a counter example for this for this I gave you and for between these two I have to give you a counter example right neither way is the implication true right. So, where we left of last time was if you consider. So, you consider the example where x n is equal to 1 with probability 1 over n 0 with probability 1 minus 1 over n. We showed that x n converges to 0 in probability, that x n does not converge to 0 almost surely right. So, here x n are independent. So, this is a counter example we gave right to show that convergence probability does not .replique convergence almost surely we used in fact Borel Cantelli lemma 2 to assert that it is not true that beyond the point. So, even no matter how far out you go with there will be some x n which is equal to 1 right which means x n does not go to 0 the argument clear. So, this Borel Cantelli lemma also gives us a sufficient condition to check for convergence almost surely. So, if you think about convergence almost surely it is conceptually easy enough. So, it just says that you go look for all omegas where this convergence happens if the set of omegas where this convergence happens has probability 1 you have fine right then convergence almost surely. But this is not a very practical definition right you have to go on hunt for all where you have to see which are where where where this the convergence happen where the convergence not happen and look for its probability right. It is not a very practical way of ascertaining whether the convergence almost surely occurs or not. So, what we will do is today we will derive first we will derive a simple sufficient condition for convergence almost surely and we will derive a necessary and sufficient condition for convergence almost surely. So, sufficient condition for checking whether x n converges to x almost surely if for all epsilon greater than 0 sum over n equals 1 to infinity probability then x n tends to x almost surely. So, this is a easy condition to check. So, if you notice this. So, if you notice just that object it looks like it looks like. So, this is what this term if you prove that this term goes to 0 then it is convergence in probability right. So, if this term goes to 0 you have convergence in probability for every epsilon greater than 0 this condition is a little bit stronger it says a little bit more as n tends to infinity not only does this term go to 0 for every epsilon it goes to 0 fast enough that the sum converges to some finite value is it clear. For example, if this probability where to go to 0 as 1 over n you will have convergence in probability, but 1 over n does not sum to any finite value is infinite. So, if it goes to if this term goes to 0 not just goes to 0 as n tends to infinity, but goes to 0 fast enough to keep the summation finite then you have convergence almost surely is that clear. So, this is a very easy way to check for convergence almost surely, but this is only a sufficient condition. If this holds you are guaranteed almost your convergence, but even if it does not hold sometimes you may have almost your convergence. So, it works only in one direction how do you prove this. So, it is a fairly easy proof. So, you have to use Borel cantilelemma if you see a summation like that you have to use Borel cantilelemma right. So, you call that event let us call this event a n of epsilon a n of epsilon is the event that the absolute value of x n minus x x e it is epsilon by Borel cantilelemma 1 only finitely often does a n of epsilon occur for any epsilon correct. So, by Borel cantilelemma 1 we see that for any epsilon greater than 0 a n of epsilon occurs only finitely many. So, what I want to say is for a epsilon greater than 0 only finitely many a n of epsilon occur with probability 1 right you agree with that. So, I have sum over n equal to infinity probability of a n is finite. So, only finitely many a n will occur by Borel cantilelemma and this is true for any epsilon greater than 0 why because that is what my if for all epsilon greater than 0 I have this condition. So, for all epsilon greater than 0 only finitely many a n of epsilon occur. So, what does that mean it means that for any epsilon greater than 0 there exist some n naught beyond which none of the correct right. So, which means beyond that n naught absolute value of x n minus x is less than epsilon. So, x n has converge to x and that with probability 1 correct you see the proof. So, this Borel cantilelemma is very powerful sometimes improving this almost sure convergence. Thus for any epsilon greater than 0 the difference the difference x absolute value of x n minus x. So, what I want to mean. So, I am not saying this in a sentence difference remains less than epsilon for all large enough n right. So, I am saying this. So, there all these x n right. So, this is let me just consider x n minus x as my what I am plotting right. So, and if this is my I am considering this band of width this is epsilon this width is epsilon what I am saying this they will only be. So, this. So, if I am plotting here the difference x n minus x right. So, then may be. So, there may be some excursions right as n goes, but what happens is there must be some n beyond which it stays within it forever stays within this epsilon band right never exceeds this epsilon band. And this is true for every epsilon greater than 0 which means almost surely x n converges to x you see this picture. So, this is the difference that can jump around. So, you this is your epsilon band you fix any epsilon you want 0.001. And then it keeps jumping around, but the Borel cantilemma says for with probability 1 there exist an n beyond which this excursions never occur. Because only finitely many excursions occur outside epsilon and if finitely many excursions out late epsilon occur. So, this sequence is dooms to stay within this epsilon band forever no matter how small you take your epsilon eventually will get in and never get out which means almost surely convergence is occur right, but this is only a sufficient condition. So, it is. So, I leave it as an exercise to you to figure out a situation where convergence almost surely happens, but this situation is violated this condition is violated. So, if this condition is violated your guaranteed almost sure convergence, but you may have almost sure convergence without this condition holding you see what I mean you create such an example actually you already know that example you just have to recall it what I am trying to say is try to break the converse of this theorem converse in hold great. So, that is a very simple sufficient condition to check for in a similar spirit next I am going to do as necessary and sufficient condition for convergence it will essentially exploit this picture necessary and sufficient condition for almost sure convergence. So, before I move to this I want to make sure you understood this sufficient condition because the necessary sufficient condition is little more involved if you do not understand this you will have hard time with what I am going to do this theorem. So, I am going to call same a n of epsilon is the event that this excursion happens right. So, same as this let a and b that event and b m of epsilon is equal to union n greater than or equal to m a n of epsilon. Then x n tends to x almost surely x m converges to x almost surely if and only if probability of b m of epsilon goes to 0 as m goes to infinity. So, this is a necessary and sufficient condition. So, whenever something is necessary and sufficient it is like equal statement right x n tending to x almost surely is equivalent to saying that my b m of epsilon has a probability that is going to 0 see note that a n of epsilon probability of a n of epsilon going to 0 is convergence in probability correct I am saying a little more I am saying a little something is. So, I will help you interpret this also that this one mistake I made I think b m of epsilon tends to 0 as m tends to infinity for all epsilon greater than 0 this is true for every epsilon. So, now I will help you with the statement. So, a n of epsilon is clear a n of epsilon is the event that there is a there is at least an epsilon excursion of this difference at the n th value correct. This is the n th term of the sequence. So, this difference exceeding epsilon means a n occurs a n of epsilon occurs. Now, what is b m of epsilon it is the union n greater than or equal to m a n of epsilon. So, can you explain in words what b m epsilon means you fix an epsilon no problem then what is b m of epsilon sorry. So, this is union over n greater than or equal to m. So, you are uniting over m m plus 1 m plus 2 and so on right. So, in words then b m of epsilon is the event that at least one excursion happens after m m or after right correct. So, a n of epsilon is the event that at n. So, if you are looking at a particular n right. So, this is the n axis if you look at figure right. So, you are looking at the event that this difference has exceeded epsilon right x n minus x is a epsilon that is your. So, a n epsilon just looks at n whereas, b m of epsilon says you fix an m fix an epsilon and then you say b m occurs if there is even one excursion m or after is that clear. This just means there exist an n greater than or equal to m for which the excursion happens the greater than epsilon excursion happens. Any questions on this this is very crucial everybody with me. So, what this theorem says is if the probability of this even goes to 0 then you have almost here convergence. So, if the probability of the event that. So, x n tends to x almost surely if if an only if beyond m no epsilon excursions ever occur. If you find some m right this m can be very large, but if you find some m beyond which no epsilon excursions ever occur after that m then you have almost here convergence and vice versa. It is an if and only if condition it is not a sufficient condition it is necessary and sufficient condition. Any questions we repeat anything. .Yes, no that was the sufficient condition. So, a n of epsilon. So, you understand what this figure is I am plotting. So, this curve is the this whatever I have drawn here is the difference x n minus x I am plotting it versus n. And I am saying that let us say this is my n if the difference between if this difference x n minus x exceeds my epsilon band then I say. So, this is actually not absolute value of x n minus x then this is only x n minus x fine. So, if this difference exceeds epsilon I say a n occurs a n epsilon occurs. Now, we know that if see a n epsilon probability of that going to 0 is convergence in probability right that is clear to everyone, but convergence almost surely needs a little more. It not only needs that the probability of a n of epsilon goes to 0 a n of epsilon is the event that there is the at n there is an excursion right. But we are saying that b m of epsilon has to have probability going to 0. Now, this b m of epsilon is the event that there exists an there exists some there exists some m beyond which no excursions occur. So, b m of epsilon is the event that there exists. So, b m of epsilon is the event that there exists an epsilon excursion m m plus 1 beyond m correct. If the probability of an excursion at m or beyond if that probability goes to 0 you have almost your convergence and vice versa it is a necessary in the future. So, this m will be related to that n of right. Yes that is what we are saying. So, from here sufficient condition is going back there. Yes sufficient condition see sufficient condition yeah. So, this essentially tells me that there are only beyond some. So, beyond some m there are no more incursions that is what there are no more excursions what this guy says right. Now, I am saying this is a little more strong. Just put some constraint before no no no no see this is a sufficient condition this is a necessary and sufficient condition. So, this will be a this will say more than that. Before n naught by b c l m m beyond n naught it does not occur. After some. So, this results hold this result holds because by Borel Cantelli m r there exist some m beyond which there are no further in the excursions right. Therefore, it converges here what we are saying is it is necessary and sufficient that B m of epsilon has the probability tending to 0. No it is not. So, I will prove this I will say this is probably the I mean. So, this is probably the most challenging or demanding proof you will have to understand. I mean there are harder theorems than this which we will not prove. But, I think you should understand this much under the convergence almost surely you should really and appreciate what is going on. So, I am doing this proof proof. So, I will prove the direction. So, I have to prove that one direction right. So, if limit m tending to. So, this is one direction other direction is the I should say that if x n tends to x almost surely I have to prove this happens. So, first I will prove this direction. Let a of epsilon equal to integral sorry intersection m equal to 1 to infinity union n equals m to infinity a n epsilon. So, this is a familiar object right this is the event that. So, this what is this event. So, this is nothing but this is the event that a n infinitely occurs infinitely often correct this is an old four we encountered this in world cantilever proof right. So, now I am going to say. So, what have I assumed I have assumed this limit is 0 is it not you know what I am actually trying. So, sorry I am actually what I am trying to prove is actually the converse I am sorry. So, I am proving that if sorry. So, I started the wrong proof. So, if x n converges to x almost surely then I will prove this first. So, if x n converges to x almost surely then clearly probability of a epsilon equal to 0 for all epsilon greater than 0 let us see if you agree with that. So, I am assuming that x n converges to x almost surely. So, but x n converges to x almost surely then I am saying that for any epsilon I must have only finitely many excursions. So, I must have only finitely many epsilon excursions because you are you have convergence almost surely. So, at some point you should be always within that epsilon band only then you can you say the probability you have to be within the epsilon band. So, which means you have to have only finitely many excursions because if this is not true then you will not have convergence with probability 1. So, we have thus probability the probability intersection m equals 1 through infinity. Now, you look at that what is this guy what is inside this box is b m epsilon that is what I have defined it you will agree with that I have simply rewritten this bit. Now, I can do something now I can. So, I do have. So, these b m epsilon are nested now you have to tell me whether it is nested decreasing or they are nested decreasing b m epsilon are nested decreasing they are Russian dolls you see why because b m epsilon is the even that there exists some epsilon excursion m or beyond. So, if you have some excursion m plus 1 or beyond you are guaranteed to have an excursion m and beyond. So, b m are nested decreasing they are Russian dolls which means I can invoke continuity of probability measures by continuity of probability measures. We have limit m tending to infinity m tending to infinity probability of correct. So, one way m done so I have proven this statement. Now, I have to prove the converse. So, little bit more tricky I have to think about differently it is not necessarily more tricky. So, the converse I have to prove that if limit m tending to infinity then x n converges to x almost sure. So, I mean if. So, this is all for for all epsilon greater than 0 everywhere it is for all epsilon greater than 0. Now, see the key to this proof everywhere is that see throughout. So, it is actually this is not just if then it is actually equivalent may be I should. So, this condition that I have said x n converges to x almost surely is equivalent to saying that there are only finitely many excursions correct for any epsilon greater than 0 that is what I am going to use right. So, more formally see be the event that x n of omega approaches x of omega I want to prove that probability of c equal to 1 when this is satisfied. So, I have I can use what I will prove is the probability of c complement is 0. So, let me just keep this figure. So, I am going to prove that probability of c complement is 0. So, what is c complement mean c complement means that x n is not converges to x which means that no matter how big an n you look at there is some excursion right. So, that is what we will use probability of. So, I will write it like this. So, c complement is the event that there is no convergence does not happen which means there must be some epsilon for which a of epsilon occurs. Now, what is a of epsilon are there in finitely many excursions right. So, if convergence does not happen see convergence is the same as saying beyond some n you are within the epsilon band for any epsilon correct. If you do not have convergence then for some epsilon you have infinitely many excursions correct. So, which means there exists some epsilon for which. So, you can write it like this. So, c complement is the same as saying there exists epsilon for which there are infinitely many epsilon excursions agreed. So, a of epsilon is the event that there are infinitely many excursions. So, now. So, there exists some epsilon greater than 0 for which there are infinitely many epsilon excursions. So, now little this is a little bit of a problem. So, we never encountered a situation where you have a uncountable union this is uncountable union right. So, what you do is you convert it to a countable union. So, this is a standard trick in measure theory probability 3 and so on. So, you say union over m equals 1 through infinity a of 1 over m you see why these 2 are the same. Because if for some epsilon which is not of the form 1 over m let us say there is an excursion for. So, let us say this occurs for some epsilon greater than 0 that epsilon need not be of the form 1 over n. But you can always find a large enough n for which 1 over n is smaller than that epsilon correct. So, if you have infinitely many excursions for epsilon equal to 0.0028 or something you can always find an integer let us say m such that 1 over m is less than 0.0028. In particular I can take m equal to 1 over 0.002 right something like that. So, you can always do that going from an uncountable union to a countable union is not a problem fine. So, let us say epsilon is suppose you have an excursion suppose you know that a epsilon there are infinitely many excursions for epsilon equal to you give me a number. So, let us say it is like 1 over some big number right. Let us say it occurs for 1 over 10000 pi just for the sake of it for epsilon equal to 1 over 10000 pi you know you have a excursion right. So, you can always take. So, how is this is like 3000 something right. So, you take m equals 4000 right. So, you have infinitely many excursions for that epsilon. So, you definitely have infinitely many excursions for epsilon equal to 1 over 4000 correct. So, there is no loss of generality in doing that right this is something I want you to appreciate. So, now it is easy right now you know how to deal with this stuff right. So, this I can use union bound correct. So, I have to prove that probability of. So, if I have the summation of the probability that there are. So, there are all these terms right I want to prove that each of these terms is 0 probability of each of these terms I want to prove is 0. How does that follow it follows from. So, it follows from here right I have assumed that. So, probability of a epsilon is equal to limit m tending to infinity of this guy right. So, I want to show want to show each term here is 0 as long as this is equal to 0. So, I think I am using the same m that is causing a little bit of confusion let me see. So, I may maybe I should put a k here instead of m let us see. Now, I want to look at probability of a of consider probability a of 1 over k this is equal to the probability that intersection m equals 1 through infinity. So, yeah throughout you just write k just for just. So, that we do not mix up with our b m them there just write k here let us just write k all this is k this is also k this is equal to you agree. See probability of a of epsilon is nothing, but the probability that it is the probability of a of 1 over k is the probability that there are infinitely many 1 over k excursions which I am writing like that agree. So, this is equal to by continue probability this will be equal to limit m tending to infinity probability of b m of 1 over k. And this is equal to what why is the 0. So, that is what I assumed correct. So, this is equal to 0 by assumption correct thus probability of a of 1 over k is 0 for all k greater than or equal to 1 correct. So, this summation will be equal to 0. So, if you are summing an infinite number of 0's you get you get a 0 that there be no confusion about it some k equal to 1 to infinity 0 is 0 there is no confusion about that. Which means probability of c complement is 0 there were probability of c is 1. So, that clear. So, go over this theorem properly this really this proof of this theorem really brings out the qualitative nature of this almost your convergence. So, what this theorem saying is almost your convergence is equivalent to saying that there exist some m beyond which there are no more epsilon excursions for any epsilon greater than 0. All your epsilon excursions end at some point and after that your sequence is constrained to lying within that epsilon band. I have used some probability right. So, you are saying that you can just replace your initial proof. So, the only step involved there is that you have to go from here to here that is exactly what I have done actually right. So, you have to go from an epsilon to some countable index that you have to reason. After that that is exactly what I have done then this union bound also helps. So, there in lies the qualitative difference between convergence in probability and convergence almost surely convergence just probability just says that a n of epsilon probability goes to 0. So, convergence in probability always looks at just 1 n forget about the rest of the sequence only you look at a particular n and sees what is the probability of an excursion look at fix an n look at the probability of an excursion and if the probability of the epsilon excursion goes to 0 then you have convergence in probability. For convergence almost surely you are not looking for a particular n what you are doing is you fix a particular m and you are saying that beyond m there are no further excursions. Is that clear yes with me. So, you see even that the 1 over n example we did right x n equal to 1 with probability 1 over n see the probability of x n exceeding epsilon may namely x n being equal to 1 has very small probability 1 over n. But, if you look at after m there will be some excursions with probability 1. So, you do not have almost your convergence correct now are you able to convince yourself that convergence almost surely implies convergence in probability. So, this here this hammers everything it proves that simple implication of almost your convergence in playing convergence probabilities. So, trivial consequence of what we proved right. So, we have proven much more you see why because our theorem necessary and sufficient condition theorem says beyond m there are no further epsilon excursions which means at m there cannot be for any the probability of excursion is going to 0 you have therefore, convergence in probability. So, that is fairly easy. So, corollary x n converges to x almost surely implies x n converges to x in probability. So, this is because see x n. So, x n converges takes almost surely is equivalent to saying that limit m tending to infinity probability of b m epsilon equal to 0 right. But, so b m epsilon is contained in a m epsilon you see why see b m epsilon means m and beyond there are there is an excursion I think it is other way is it. So, maybe I missed it up b m epsilon means there exists an excursion. So, b m epsilon says there exist an excursion m and beyond a m epsilon means there exist an excursion at m right. So, I wrote this is incorrect. So, I should really right. So, I should really write that correct agreed. So, thus so this implies limit m tending to infinity probability of m epsilon equals 0 why b m is a bigger event right that has probability going to 0. So, you have convergence in probability of course, the converse we know is not true right we may have this probability going to 0. But, you may have some excursion popping of for right. So, you may we saw the situation where this probability goes to 0. But, this probability does not go to 0 the 1 over n example we gave right. But, this direction easily follows. So, this result is a trivial consequence of the necessary and sufficient condition theorem we proved. So, I will stop here.