 Welcome back. So, today we will discuss Borel-Cantelli lemmas. So, Borel-Cantelli lemmas say that if you are given a sequence of events, it gives conditions under which only finitely many of these events will occur. I will stated properly now. So, as usual you are given a probability space. Some omega f p is given to you. So, there are two Borel-Cantelli lemmas that we will study. There are actually a few more versions, but we will study the two most famous ones. The first Borel-Cantelli lemma says the following. If a 1, a 2 dot dot dot is a sequence of events such that sum over n equals 1 through infinity probability of a n is finite. Then almost surely or with probability 1 only finitely many a i's will occur. So, a 1, a 2 dot dot dot this is a sequence of events. So, they are all f measurable. So, all these events will have some probabilities associated with them. So, if it so happens that the sum of the probabilities of a n, n equal to 1 to infinity is finite. Then the Borel-Cantelli lemma 1 says almost surely that is with probability 1 only finitely many of the a n's will occur. So, I will help you digest it a little bit better. First I want to. So, there are only two possibilities for this summation. So, probability of a n is a non negative. This is like a non negative sequence. P of a n is a non negative sequence. So, if you sum a non negative sequence n equals 1 to infinity it is well defined. It is always well defined. The question is whether it goes to something finite or goes to plus infinity. There are only two possibilities. So, the first B C lemma talks about the case when it is finite. The second Borel-Cantelli lemma talks about the case when it is infinite. We will first deal with in this finite case. So, I will help you parse what this finitely many a n's occurring means. The second Borel-Cantelli lemma I will just state both and then explain both of them. So, this is like a partial second Borel-Cantelli lemma is like a partial converse to the first B C lemma. So, it says that if this is infinite then infinitely many a n's occur, but not quite. You need independence. Here you do not need any independence. Then almost surely infinitely many a n's occur. So, these two are the statements. First Borel-Cantelli lemma says if the summation of the probabilities is finite only finitely many a n's will occur. The second says if the summation is infinity and you have independence then infinitely many a n's will occur with probability 1, both the probability 1. So, let us try to understand the first Borel-Cantelli lemma first. So, you are looking at. So, you can look at this guy. So, probability of a n as some sequence p n some numbers p n with some 2 something that is finite. So, what you have really is a sequence of events whose probability is going down to 0. Because if you know that if a series sums to something finite the nth term has to go to 0 that you know, but the converse is not true. So, you have a sequence of events which is becoming more and more unlikely. So, in that case. So, it is not only true that p n the probability of a n is going to 0. It is also true that the sum is finite which is saying more than that nth term is going to 0. So, you have the sequence of events which are becoming more and more unlikely and they are becoming more unlikely rather fast. So, that the summation is finite. So, which means so this implies that beyond a certain point beyond some finite let us say n naught none of these a n's will occur almost surely. There exist some n naught beyond which none of these n all these a n's will fail to occur because they are becoming so unlikely. So, that is what this means. So, there so finitely many see finitely many a n's will occur means some a n's will occur, but beyond the point there exist some n naught beyond which none of these a n's will occur that is what this means. Similarly, for this part if you are talking about infinitely many a n's occurring that means what. So, you go as far as you like as begin n as you like, but beyond that n there exist at least some a n that occurs no matter what and you look at that is what this finitely many a n's and infinitely many a n's occurring means. This is intuitively everybody with me. So, I will introduce some notation to make this more formal. So, I want to talk about see I am both this lemmas talk about some event of only finitely many a n's happening or infinitely many a n's happening. So, I will actually denote that. So, there is a particular notation for it. So, event that infinitely many a n's occur this is what I want to characterize. So, I want to write this as some unions and intersections of a i's and that is done as follows you write n equal to 1 through infinity union i equal to n through infinity a i. So, it is a nested is an intersection and a union this event is called. So, this is denoted often by a n infinitely often this is simply notation for this event. This is an event that we will encounter a few times. So, this is just denoted by a n i o what this means is just this. I will help you pass this in a minute, but you will agree that since a i's are events and I am only taking countable unions and intersections what results will also be a event. Because, this is just countable union and intersections of these a i's. So, it has to be f measurable. So, this is also an event and I can genuinely refer to it as some event for which I can assign probabilities to it is that clear first of all. So, what the Borel Cantile lemma 1 says is that this event has 0 probability in this case and has probability 1 in the second case that is what the B.C. Lemma says everybody with me of course, I have to explain to you what that all these intersections union means. So, I will help you pass this as follows. So, let us make a definition let B n equal to union i equals n through infinity a i. So, then I have a n i o is equal to intersection B n correct. I am just so this guy here this guy here I am calling B n to help you pass it. So, whenever you have this nested unions and intersections actually in your homework you will see three of them in one homework here there is just two. What you do is you start from the most in a most union on intersection and then work your way out. So, I am going to call that B n I am indexing it by n because I am starting from n here. So, what is B n can somebody tell me what in words what B n is at least. So, B n is the event that at least one of a n a n plus 1 a n plus 2 etcetera occurs that here at least one of a n a n plus 1 a n plus 2 etcetera occurs that is B n. So, for that reason B n is sometimes referred to as the nth tail event nth tail event right it is saying that from n onwards at least one of the a is occurs a n a n plus 1 a n plus 2 one of them at least one of them occurs. And then what is a n infinitely often. So, it is just the intersection of these nth tail events right. So, this is the event that all the B n's occur for every n B n occurs correct by definition. Now, B n is again the nth tail event which means for every n at least one of a n a n plus 1 a n plus 2 etcetera occurs correct. So, which means no matter how big your n is I will have at least one of the a n's after that n right occurring is that clear. So, no matter how far I go no matter how big let us say my n naught is beyond that n naught one of these a i's will still occur correct this clear to everybody. So, the first barrel cantile lemma says that a n infinitely often this event has zero probability in this case. The second barrel cantile lemma says this event has probability one in this case are there any questions at this point before I prove these results. The proofs are actually fairly elementary once you manage to understand what it is that the lemma says the proof improving it is not very easy difficult. So, a i's so this I have defined B n as the union from n to n to infinity. So, this is B n is the event that at least one of a n plus a n a n plus 1 a n plus 2 etcetera occurs right for the given n right. So, it is called the nth tail event. So, given n it is the event that at least one of a n a n plus 1 a n plus 2 etcetera occurs and this is the event that each one of the B n's occurs which means for every n. So, I will say that this event has occurred if each of the B n's occurs which means for every n I must have a at least one of a n a n plus 1 a n plus 2 etcetera occurring correct for every n I need at least one of a n a n plus 1 a n plus 2 etcetera occurring any questions proof B c lemma one is very easy to prove. See remember that first Borel continue lemma does not need independence or any other further constraints right. If it so happens that sum over n equal to infinity p of a n is finite then this will apply only finitely many a n's will occur. So, what do I have to prove I have to prove that in the first case this event has probability 0. So, that the complementary even finitely often we have probability 1. See what is the complement of this will become this intersectional become a union then the union become an intersection and then you will have a complement right which means there exist n n such that each of the further a i's fail to occur which is what finitely often means right. So, you can just use demorgan and flip this around you want. So, if the first Borel can tell lemma. So, you have. So, you want to look at this right you want to look at probability of essentially this right you want to show this is equal to 0 right and what is the if I see something like this what is the first thing I do I invoke continuity of probabilities. Now, this B n's I think this B n's will turn out to be nested increasing or nested decreasing B n's are. So, B n is the event that at least one of a n a n plus 1 etcetera occurs. So, B n plus 1 is so nested decreasing right. So, this B n's are like Russian dolls they are nested decreasing everybody with me just think about it if at least one of a n a n plus 1 a n plus 2 etcetera occurs right that is a bigger set than at least one of a n plus 1 a n plus 2 etcetera occurring right correct. So, I can use continuity of probabilities to write this as probability of agreed this is continuity of probabilities and using the fact that B n's are nested decreasing fine. So, that is equal to limit n going to infinity probability of my B n is after all union i equals n through infinity a i I have just copied what B n is. Now, I can use union bound on that I have a union of events. So, probability of a union is less than or equal to some of the individual probabilities after all I have to get to a sum of probabilities at some point right. So, this is less than or equal to limit n tending to infinity sum over i equals n through infinity probability of a i. So, this is union bound this is continuity of probabilities everybody with me. So, now the proof is almost over reason is I know that the summation this is a converging summation right this converges to something finite. And you know that this guy is the tail sum of a convergent series and as n goes to infinity the tail sum has to go to 0 right you know that from your sequences in series right. So, this has to be equal to 0 right as n goes to infinity this goes to 0. So, I have prove what have I proven I have proven that with probability 1. So, with probability 0 the there is probability 0 that infinitely many a n's will occur which means with probability 1 only finitely many a n's will occur right which means beyond the point a n's will all a n's will fail to occur beyond the certain n naught none of the a n's will occur good. So, this is a very simple proof. So, this is hardly 3 steps with me any questions. So, now we have to prove second Borel Cantelli lemma. Second Borel Cantelli lemma says that it is a partial converse why am I saying it is a partial converse you need independence right it does not say that no matter how this summation is infinite I will always have infinitely many that is not what it is saying right you need independence. You have it is further stipulating independence right the second Borel Cantelli lemma it is a little bit of it requires some algebraic jugglery here. So, I will prove a lemma to prove a lemma. So, the second B C lemma I want to prove another lemma to prove it there we go suppose 0 less than or equal to p i less than or equal to 1 is such that sum over p i equals infinity then product i equals 1 through infinity 1 minus p i equal to 0. So, this statement has nothing to do with probability as such it is just a you are saying that if something if series sums to infinity right series of numbers you have this p i which are between 0 and 1 you can think of them as probabilities, but it is not necessarily the case right it sums to infinity. So, then you have this product is 0. So, the proof of this is again just algebra. So, we have the following. So, we have log. So, it follows from this fact log 1 minus x is always less than or equal to minus x for x in 0 1 right this is something you can prove this is just an identity this is identically true right for all x in 0 1. So, you apply. So, you take logarithm of this. So, let me do that. So, you have log product i equals 1 through infinity 1 minus p i equal to log of limit n tending to infinity product i equals 1 through n 1 minus p i. So, this guy see I want to prove this guy is 0 right. So, I want to I am just taking log because it is a product a log of a product just becomes a summation right. So, this will become. So, this I am writing as a limit. So, now I have less than or equal to log of pi i equals 1 through k 1 minus p i for all k right. So, this can only be bigger right I am only going. So, this is for n bigger and bigger. So, I am multiplying numbers that are less than 1 right. So, if I stop at some point right I will only be bigger right and log is a monotonic function right. So, this is again simple algebra just nothing very deep here. Now, what do I do I have log of all this product which is equal to I can write it as log of a product is sum of the logarithms right. I will have sum over i equals 1 through k log 1 minus p i which is less than or equal to sum over i equals 1 through k minus p i right. This is this relationship is because of this guy right. So, first or note that you just say note that here. So, from there you get this and this is also true for all k. So, taking k to infinity. So, if I take k to infinity what happens to this term yes I know that sum over p i is infinity. So, this summation will go to minus infinity which just means the product will go to 0. So, we have log of all this product 1 minus p i this way will go to minus infinity implying the result right. So, that is just a lemma to prove the lemma and this lemma has nothing to do with probability just algebra right. So, now I want to really prove this Borel Cantelli lemma 2 actually let me keep this part of the board I will prove number 2 here. So, I have to prove. So, in order to prove Borel Cantelli lemma 2 I have to prove probability of a n infinitely often is what 1 right or I can prove that the complement has 0 probability that is also fine right. So, what I will look at is in fact the probability of the complement of this guy. So, I have 1 minus the probability of a n infinitely often is equal to the probability of union n equals 1 through infinity B n complement agreed. I am looking at the probability of the complement a n finitely often essentially and that I am using de Morgan to de Morgan on this this. So, this I will again use union bound sum over n equals 1 through infinity probability of B n complement. Now, I want to prove that this is equal to what I want to prove this is equal to 0. So, in order for this to be 0 I need the only way this can be 0 is each of this B n complements should have 0 probability right. Otherwise it is not happen. So, I want to prove that each of this is 0 and therefore, you are up bounded by 0 which means it is 0 right. So, here I go. So, you have we will show probability of B n complement equal to 0 for all n greater than or equal to 1 indeed fix n and m greater than or equal to n. Then you have probability that intersection i equals n through m a i complement is equal to. So, this is. So, I am taking an intersection of a complements. So, why am I doing that I want to prove that B n complement has 0 probability and B n complement will have intersection a i complement. But, it will it will go from n to infinity, but eventually I will send m to infinity use continuity of probabilities. So, if you look at this a i's are independent events you can prove that if a i's are independent events. Then a i complement are also independent events you have to prove it you do it as homework. So, this you will get this what will happen to this this will product out right this will have pi i equals n through m probability of a complement which is 1 minus probability of a i good. Now, if I send m to infinity what happens. So, if I send m to infinity. So, look at this very carefully if I send m to infinity limit m tending to infinity I will use continuity of probabilities to put the infinity here which will give me probability of B n complement. And then I will get i equals n through infinity 1 minus p of a i. So, what I am saying is thus probability of B n complement is equal to limit m tends to infinity probability of intersection i equals n through n through m a i complement. So, this is by what relation this is because of continuity of probabilities right because B n complement is the intersection n equals i equals n through infinity a i complement. I am writing it as a limit using continuity of probabilities and by this relation I will get this is equal to a product. So, I will put the limit here and right. So, that will become i equals n through infinity I may be skipping one step here 1 minus p of a i. You know this statement of the lemma is that almost surely a n occur infinity of almost surely means. So, this almost surely means with probability 1 w p 1 or a dot s is something that used interchangeably it means that the probability of that event is 1. So, far with me. So, now this I know what this is why because I have see I am going from some n to infinity, but I know that if I some p of a i from n to infinity I will get infinity. So, by that simple algebraic lemma this will be 0 for all n greater than or equal to 1 by the lemma by that earlier algebraic lemma because this p of a i some to infinity right. So, this product will be 0 by the lemma. So, which means for every n greater than or equal to 1 probability of B n complement is 0 right which means that guy will be 0 right. So, from here you can conclude that that is 0 because each of this is 0. So, the summation is 0. So, probability of a n occurring infinitely often is 1 everybody with me. So, I will give an example in a little bit, but first I want to make sure that you understand the proof everybody with me. So, just look over it 1 more time it is not a difficult proof all steps are I mean you used repeatedly use union bound continuity of probability these are the things that you keep using all the time in this you kind of proofs. So, I will. So, in what remains of this lecture I will give an example to illustrate the first B C lemma second B C lemma. And I will also illustrate that if you throw away independence completely throw away independence from the second lemma and you still have this it does not necessarily mean that infinitely many a n will occur with probability 1. So, it is not a complete converse right first I will do the example. So, do some example. So, the simplest example is. So, you are looking at whether a sum of certain probabilities goes to infinity or is finite right. So, you have to look for some convergence series or non convergence some something that is going to infinity some divergent. So, example see I am going to toss. So, it is like this infinite coin toss model except my probability measure is not going to be what we already studied. So, I am going to toss coin infinitely many times and my sigma algebra is your familiar sigma algebra whatever we did except now the probability measure is not going to be my half the fair coin measure I am going to put something more complicated on it. Let p let a i be the event that the ith toss is heads as usual right we studied this look at this before. And let p be a probability measure on omega f such that probability of a n equal to 1 over n for all n greater than or equal to 1. So, I am taking some probability measure which has the property that the probability of the nth toss turning up head is 1 by n right. So, I will. So, this 1 by n. So, this has the property that summation is infinite right. So, I will later make. So, sum over 1 by n is infinite if I make this 1 by n square it is summation is finite you know right. So, if I want to give the example of the first Borel Cantelli lemma I should probably just give 1 by n square to begin with right. If I keep 1 by n it will be an example for the second lemma. So, let me keep 1 by n square. So, I am not specifying see I have not told you what the measure is on the entire sigma algebra you take whatever measure you want that has this property. I have not see I only specified measures for a n right I have not told you what the measure. So, you have to do the full story right like we did it for the fair coin toss model, but I am saying let p be any probability measure which has this property. You can prove that you can formalize this no problem, but I am all I am saying this. So, let us think about this intuitively I am saying that I have this infinite coin tosses the probability of the first toss being heads is 1 probability of second toss being heads is 1 fourth say third toss being heads is 1 9th and so on. So, it is becoming heads are becoming very very unlikely as you go along right the probability of the 100 toss being heads will only be 100 10000 right is very very unlikely as you go along. So, you have so this is since sum over p of a n is finite actually if you sum n equals 1 through infinity of p of a n you will get pi square over 6 right it does not matter what value it is right it is finite correct. So, sum over n equal to infinity 1 over n square is finite b c lemma b c lemma 1 implies implies that almost surely only finitely many heads will. So, all I am saying this that if your heads are becoming more and more unlikely like 1 by n square. So, in the nth toss the probability of finding heads is only 1 by n square I am saying that there exist an n naught almost surely beyond which no heads will occur you will get only tails beyond n n naught you will get only tails and with probability 1 there exist an n naught such that beyond n naught there is exist only tails that is what this is saying. So, heads will stop occurring all together with probability 1 correct is that clear. So, statement says almost surely only finitely many heads will occur which means there exist with probability 1 there exist an n naught beyond which there will be no heads with probability 1 there will be no heads whatsoever beyond a certain n naught and such an n naught exist with probability 1 yes. So, for any positive yeah. So, for any n. So, for any given n the probability of heads is 1 by n square, but what I am saying is the what the Boral cantile lemma is saying is that there exist some n beyond which you will never see a head with probability 1 right the set of all. So, the set of all and the set of all coin tosses which corresponds to having infinitely many heads will have probability may will have measure 0 the set of all coin tosses in the 0 1 power infinity which correspond to infinitely many heads will have probability 0 that is what this is saying. So, now if you want to tweak it around and make it into an example for b c lemma 2 what you have to do. So, suppose yeah so you have to of course, make this coin tosses independent you have to make a appropriate model in which. So, next suppose p of a n equal to 1 by n and that a n's are independent. So, you have independent coin tosses now. So, in earlier case you do not even need independence you did not need independence at all right now you are saying that these coin tosses are independent and my probabilities are now 1 over n. So, in my 17th toss the probability of getting heads will be 1 by 17. No there exist now with probability 1 there exist an n n naught beyond which none of no heads will occur. That is not what I am saying I am looking at a n infinitely often right I am saying probability of a n infinitely often is 0 you are not looking at probability of a n naught plus 1. So, which is why I am saying this Borel country lemma is very easy to state, but difficult to digest right. So, think about it. So, in this case let me just finish this example. So, you understand the example. So, you have your tossing coins independently and the probability of finding heads in the nth tosses 1 over n. So, here also that heads are becoming more and more unlikely right. So, in the 100th toss the probability of heads is 1 over 100 and probability of tails is 99 over 100. So, it is becoming overwhelmingly likely to see more heads, but according to second BC lemma see since. So, what I am saying is since sum over 1 over n is infinity and the events are independent right BC lemma 2 implies that almost surely infinitely many heads will occur. So, what you are saying is. So, if you will just look at this as n. So, let us say this is n equal to 1 and then you are looking for out. So, if you are looking for out if you are looking at some 100 the probability of finding heads is 1 over 100. If you go to 1000 it is 1 over 1000, but what BC lemma 2 is saying is no matter how far out you go you are guaranteed to find with probability 1 you are guaranteed to find at least 1 head turning up no matter how far out you go. So, although so you may imagine that n is a million the probability of heading getting head is 1 in a million 1 in a million, but I am saying that if you look at million plus 1 blah blah blah the rest of the tail events there is the tail event there will be 1 head popping of somewhere with probability 1 no matter how far out you go you can go to million or billion it is becoming more and more unlikely, but if you look at the entire tail there will be at least 1 head popping off. So, this is a highly non trivial result I want to this is not something that you can easily predict with some elementary probability calculation right. So, if you have 1 by n you get infinitely many heads occurring although it is becoming more and more unlikely, but if you make it 1 over n power 1 plus delta then the series will converge then the heads will stop occurring all together right. So, this 1 over n is this transition point right. So, the probability of heads is not 1 over n, but 1 over n plus n power 1.001 then heads will stop occurring almost surely because the series will converge right. So, I hope this example is clear think about it takes some time to digest the final point I wanted to make is that you cannot completely get away from the independence in the B.C. Lemma 2 for example, if you take you fix any event e whose probability is strictly between 0 and 1 strictly between 0 and 1 and you take all your a n's to be equal to that e right all your a n's are equal to that event e then sum over probability of a n will be definitely infinite right because you are summing something. So, that will be infinite, but still the probability of infinitely many a n's occurring will be simply the probability that e occurs because all the a n's are equal to e right. So, you cannot just throw away this independence, but it is also true that you do not strictly need independence right with independence the B.C. Lemma 2 works, but you can make a slight relaxation on the independence also which is why there are many Borel Cantelli lemmas, but you will only worry about 1 and 2 1 does not require any structure 2 for 2 we have independence you cannot throw away thank you will stop.