 Welcome back. In the previous lecture, we defined the Lebesgue measure on 0 1, the uniform probability measure on 0 1, which is an uncountable sample space. We also defined on the real line the Lebesgue measure, which formalizes the notion of length. So, these two measures are, so the Lebesgue measure on 0 1 is a probability measure and the Lebesgue measure on R is not a probability measure. It is just it is an infinite measure and both these are defined on the corresponding Borel sigma algebras on 0 1 or the real line, which are generated by open intervals. Today, we will consider another uncountable sample space. This model is called the infinite coin toss model. In this case, the sample space is the uncountable set 0 1 power infinity. So, each elementary outcome corresponds to an infinite coin toss. So, each little omega, so if you take a little omega in omega, it will be of the form omega 1 omega 2. So, these are bits. So, each elementary outcome is a infinite binary string and we know that by Cantor's argument, we know that this is a uncountable sample space. So, essentially what I am doing is calling head is 0 and tail is 1. So, this is an uncountable sample space and we want to do, we want to assign probability measure on this uncountable sample space. And you know that, because the sample space is uncountable. If you consider all possible subsets of this omega 2 power omega is 2 large sigma algebra to assign meaningful probability measures on. So, we have to again settle for some smaller sigma algebra, only consisting of those subsets of omega, which are in some sense interest of interest to us. And then, we will try and then what we will do is, once we define an appropriate sigma algebra on omega, we will assign a probability measure, what is what we like to be a uniform probability measure on this omega. Ideally, what we want is that all no particular string being preferred over any other particular string. That is the kind of we want intuitively, we want an uniform measure on this space. So, that is what we are building towards. So, this is also a very canonical probability uncountable probability space. And in some sense, you can prove that, what I am going to say is, has very close correspondence to the Borel measure on Borel sigma algebra and Lebesgue measure on 0 1. If you write it as binary strings. So, it is fairly close to what we have done, but because it is in the language of coin tosses, the flavor is a little different. So, far you have each outcome is a infinite binary string. So, in this, now the question arises, we have to put a sigma algebra on this. And as I said, if you put 2 power omega, it is too large, it is too complicated. So, what I want to do is, first start off with some collection of subsets, which allows me to describe the outcome of, let us say the first n tosses. Start with the finite set of tosses. So, let me say that, F n. So, this is going to be a collection of subsets of omega, whose occurrence can be decided by looking at the first n tosses. So, I have not really called it an event. Note, I only called it a subset of omega. I have not yet built my sigma algebra on this. So, I have deliberately avoided calling it the collection of events, because by event, I mean something very precise. I mean F measurable sets. So, this F n consists of, so F n consists of collections of subsets of omega. Omega itself is all these binary strings, infinite binary strings. And all possible, you take a certain collection of subsets of omega, whose occurrence or non occurrence can be decided based only on the first n first n coin tosses of any outcome. So, by this I mean, so if I want to ask, let us say an example, let me see. So, I want, let us say I am interested in whether or not the, so for example, I am going to look at at least 2 heads in the first n tosses. This is something I am interested in, let us say. I want to know, if there are at least 2 heads in the first, so I have an elementary outcome, which has an infinite bit string. I am looking at the first 10 bits and asking if there are at least 2 heads in it. So, this is a, so if you call this event some E or something, then E is an element of F 10, because I can decide whether E occurs or not by looking at the first 10 tosses. So, this is what I am looking, this is what F n contains. It contains all those subsets of omega, whose occurrence or non occurrence can be decided for certain by looking at only the first n tosses. So, this is a little bit of an English definition, I will give a little more of a formal definition now. So, one thing that is immediately clear is that, something that can be decided in 10 tosses can also be decided in 11 tosses. So, if some E is an element of F n, it is necessary in an element of F n plus 1, F n plus 2 and so on. So, if you can call, whether an event occurred or not in 10 tosses, you can do so in 20 tosses, no big deal. So, these are all nested. So, I will write that down. So, this is easy to show claim, F n is contained in F m for all n less than or equal to m, this is very easy. So, this is, so I have defined F n in a certain colloquial kind of a way. So, the correct way to define F n mathematically is to say that an E a subset, let us say E belongs to F n. If the first n bits follow a certain conditions, let us say in this case the condition was 2 heads, at least 2 heads out of 10 tosses. So, I am essentially looking at the first n bits and I am mandating that to be a certain particular subset of the 2 power n possibilities in the first n tosses. So, more formally, I think you will understand what I am going to say now. So, if you want to write this more formally, we will say that A belongs to F n. If there exists some A, I will write A superscript n that is a subset of 0 1 power n such that A equals the set of all omega for which omega 1 omega 2 dot dot omega n. If you look at the first n bits here, this is in this A n. So, is this clear? So, I am say this A index by I am superscripting by n. So, this is a subset of 0 1 power n. So, 0 n power n gives you all possibilities for the first n bits, but I am mandating that the first n bits of my set of interest should be a fixed subset of 0 1 power n. So, it should have a certain property. For example, in this particular example, the first the A n consists of those n bit patterns in which the at least there are 2 heads in the first n. So, you should look at 10 n equal to 10 and say that you look at in A superscript n will include all those patterns in which there are at least 2 0's in the first n. So, essentially this captures the property of the first n bits that I want to decide whether it occurred or not. Actually this is all this is very simple step I am I am coaching it in certain notation to be formal this I suppose this is clear. So, A I say. So, this is the definition of. So, I am saying that A belongs to f n if there exists some the first n bits have a certain property. You can look at this A n as a subset of 0 1 power n that satisfies the property I want for the first n bits. So, remember this A is an element of omega. See A is not an element of omega A is a subset of omega for which the coin tosses satisfy the certain property for the first n bits. So, for example, so if let me take another example. So, if I want to decide whether my third toss is a head I want to decide if my third toss is a head. So, it is an element of f 3 or f 4 f 5 everything let us say f 3. So, we say that f 3 so it is an element of f 3. So, we will say A 3 will consist of the first 3 bits in which the third bit alone is a head the other 2 can be anything there will be. So, how many possibilities 4 possibilities the third bit is fixed as head the first 2 can be anything. So, this A superscript 3 will consist of those 4 strings of triple bit 3 bits. So, is this clear. So, it is very simple stuff I actually couched it in a certain notation, but I hope this will be clear. So, what we really want so I want to eventually say I want to eventually assign some uniform probability measure. So, ideally I would like to say that as a particular n bit pattern should have a probability 1 by 2 power n that is what I want to say. So, I want to say that each bit being 0 or 1 should be probability half. So, many string of a particular string of n bit should have probability 1 by 2 power n that is what I want to say, but on the other hand I have to put a proper sigma algebra on it and do it formally that is what we are building towards. Now, what you can show another claim f n is a sigma algebra you can show this you try this as a home work. So, f n the set of all events whose occurrence or non occurrence can be decided in at most n tosses is a sigma algebra you can prove it try this as a home work. Now, given that f n is already a sigma algebra one might be tempted to say that my measurable space should be omega comma f n. So, if we take a big n let us say I take n equal to some million. So, I take some very big n and I say omega comma f n is going to be my sigma algebra it is a valid measurable space. So, I can put some probability measure on it to say that any n bit pattern has probability 1 by 2 power n the problem with doing that. So, if you take. So, if we take let us say omega comma f n as a measurable space. So, elements or sorry subsets elements of f m for m greater than or equal to n cannot be assigned probability measures. So, what happens? So, even if you take n equal to million or if I want to answer the question what is whether my million plus 1th toss is a head or not I will immediately cannot answer it. So, no matter what n you take if you take f n as your sigma algebra I will immediately ask you question about how about my n plus 1th bit or is my n plus 1th bit head or you know some question about something beyond n. So, then you will not be able to answer it because I am only taking events in elements of f n as my measurable sets. So, this is a little bit of a problem this is a sigma algebra there is no problem, but it is not a nice enough sigma algebra it is not a rich enough sigma algebra to help us characterize even beyond n nth coin toss occurrences beyond nth coin toss. So, is this clear? So, we have to do something else. So, we have to look beyond this finite horizons. So, we do the following we define. So, let us define f 0 equal to union i equals 1 through infinity f i you know I mean maybe I should actually write as I keep saying no maybe I should just write that is better notation I keep saying i belongs to n f i the same thing. Now, what are you doing you are uniting all the all these sigma algebras. So, f n consist of those subsets. So, f n is the collection of those subsets of omega whose occurrences can be decided in n coin tosses. So, when will this. So, this is another collection of this is also a collection of subsets of omega what kind of elements will it have will contain all those subsets of omega whose occurrences can be decide whose occurrence or non occurrence can be decided in no see I have union over all i countable number see it consist of all those. So, this is something you have to understand f naught consist of all those subsets of omega whose occurrence can be decided in some finite number of tosses. So, why do I say that. So, it looks less though it should you know I deliberately avoid putting infinity in order to not confuse you I equal to 1 to infinity I erase it and put i belongs to n. So, that you do not think of it as f infinity or some such thing is not f infinity f n is those those subsets which can be decided in n coin tosses. So, if. So, let us think of it this way if a subset is contained in f naught it must be contained in 1 of the f i's for some i right. So, it must be decided in some finite i number of tosses. So, it is not something it can be decided in some infinite tosses no right. So, f naught consist. So, you should probably write this down f naught consist of. So, consist of all subsets of omega which can be decided in some finite number of tosses. So, this is the important in some finite number of tosses yes. We take a bit of like intersection like this, but that time it is the same interpretation right it is the. So, if you intersect over a collection index it is the set of all elements contained in all the sets or if it is a union it should be contained in at least 1 of the indices. So, I think you are confusing things that are not related. So, let us just clarify this first. So, I want to make sure that this is clear in everybody's mind. So, if. So, f naught is some collection of subsets of omega right we want to know what kind of subsets are in this f naught. I am saying that if some subset is in f naught let us say if some subset a is in f naught then it must exist in at least 1 of the a i's, but you know that see these f i's are also nested increasing. So, if it is. So, it must be in 1 of the f i's let us say if it is let us say it is in f j. Then it will be in f j f j plus 1 f j plus 2 and so on that is because of the nested structure. So, it consists of all those subsets of omega whose occurrence can be decided in some finite j number of tosses, but the j can be a million or a billion it does not matter right. The j can the j can be as large as you want, but it should be able to call that event in some finite number of tosses it can be very large right. As an example of something if I ask you if the millionth bits the 2 millionth bit or all heads that is a subset in f naught because I can call it looking at the first 2 million bits correct. Can somebody think of an example of something that is not contained in f naught actually very easy all heads all heads right this whether my string is all heads it is all zeros cannot be called looking at f naught correct. Because f naught consists of those events which can be called in some finite number of tosses, but whether it is all heads I have to I can never stop looking at it beyond a point right I mean I have to look at everything right. So, an example of something that is not contained in f naught is whether my sequences all the all head sequence or if I want to answer if all my odd tosses are tails first toss third toss if there are all tails that is what I want to know that is also not contained in f naught correct is that clear. So, good now I want to know what is. So, this f naught I hope everybody understands f naught right. So, now I want to know what kind of an object this f naught is see these f i's are all sigma algebras, but we have to move away from it because it only gives a finite characterization of the first n bits f n gives only a characterization of the n bits. So, we have to move away from it now I will come up with some object f naught which consist of those subsets which can be decided in a finite number of tosses any finite number of tosses. Now, what kind of an object is f naught is the question ok. See if this were an intersection we know that intersection of sigma algebras is always a sigma algebras, but union of sigma algebras need not be a sigma algebras right. So, in fact it turns out that f naught is not a sigma algebras, but it is a algebras you can prove that it is an algebras it is closed under finite unions. So, by which what I mean is that if I have one subset whose occurrence can be called in let us say i tosses and I have another subset whose occurrence can be called in j tosses with let us say j bigger than i then the union of the 2 can be called in the bigger of the 2 j tosses again right. So, it is you can say that it is closed under finite unions, but you can also show that it is not closed under countable union countable infinite unions ok. So, here I go I will put that down. So, we can show that f naught is an algebra and then we can show that f naught is not a sigma algebra. So, this I will leave as a home work this is just chasing up all the definitions right chasing up the definition of an algebra can somebody tell why f naught is not a sigma algebra I just give an example actually right odd let us say odd tosses right. So, let me say for example that let. So, I am just going to prove 2 I am going to prove that it is not a sigma algebra proof of 2 in order to prove that it is not a sigma algebra I can show I can produce an example where it is not closed under countable unions it is closed under finite unions you will show ok. Let a i be the event that the i th toss is a head i th toss is a 0 then and let e equal to the. So, be the event I should not really say event right you know why it is not an event right I have not yet defined the sigma algebra I should say be the subset of omega such that. So, for example a 2 will consist of all those coin tosses all those infinite bit strings in which the second string is a head right similarly a 10 will consist of all those strings for which the 10th is a head right. So, I want to know I want to consider the following e equal to subset of omega such that let us say all let us say all even all even index tosses are heads fine. So, a i so you will know that a i belongs to what f i right a is clearly an element of f i because I can call it in i tosses f i implies a i is a element of f naught for all i correct agree. So, a is an element of f i therefore, a must be an element of f naught because if your a is in even one of the f i is then it will be in f naught right. So, this much is clear. So, after all a i is whether the i is toss is a head right can be decided in finitely many coin tosses, but I am claiming that my e. So, I can define my e as a countable union of the a i is is not it or .caddle intersection probably is not it true or no. So, if I write e I can write e note that e equal to intersection a i i is equal to 2 4 6 dot dot dot right I am intersecting all the even index a i. So, a i is the corresponds to the ith bit being heads I want all even index tosses to be heads right. So, which means e must be the intersection of all is a i's where the index runs over only the even numbers if you want all your tosses to be heads you will say i equal to 1 through i belongs to n right for example. So, is this clear right I am just mandating that all my even tosses are heads. So, this is all elements of f naught right, but e is not an element of f naught. So, a i belongs to f naught e is not an element of f naught implies f naught is not a sigma. So, I have argued here is for each i a i is an f naught, because whether the ith tosses a head I can decide in finite number of tosses correct which is what f naught is, but e says all my even index tosses must be heads and that is clearly not an element of f naught. Because f naught only consist of those subsets which are decidable in finitely many tosses and I cannot decide e in finitely many tosses, but e is a countable intersection of elements of f naught correct. So, you see the proof. So, here is yet another example of an algebra which is not a sigma algebra. So, just think about this a little bit here I mean I guess it is a little bit easier on the interval and so on right. So, this is a little I am doing this specifically to make you think about this kind of sample spaces as well. So, my f naught is an algebra, but it is not a sigma algebra ok, but there is something nice about an algebra as well right. Yesterday we stated a theorem that says if I tell you the measure on an algebra some pseudo measure on an algebra, then I can extend it under certain circumstances. So, if I tell you the measure measure I would like to put on those subsets which can be decided in finitely many tosses, then may be I can invoke caratheodory and then transfer the measure on to extend the measure on to a sigma algebra. Now, what is the sigma algebra now? Sigma of f naught sigma we do not know what sigma algebra it is some sigma algebra right. So, we can what we will try and do is put a p naught some pseudo measure on f naught then use caratheodory's theorem and extend the measure on to the sigma algebra generated by f naught. Remember f naught is not a sigma algebra. So, you have to make it a sigma algebra by taking countable unions and so on. So, here is the program all right. So, I do the following f naught is not a sigma algebra. So, this is done lemma is done. So, I want to define p naught from f naught to 0 1 satisfying p naught of omega equal to 1 and countable additivity on f naught right. That is what caratheodory wants us to do, but what we should initially do is we have to set our p naught to be right on the algebra in the first place. Just like we did it on the intervals we put p naught's as the length of the interval and then decided we will extend it. Now, we have to say that if you give me an element of f naught which can be decided let us say in n tosses I will basically make all these n 2 power n possibilities equally likely that is what I am going to do to correspond to a uniform measure right. So, we want to define p naught to correspond to fair coin tosses. So, we do as follows let us say I give you a in f naught. So, if a is in f naught. So, I want to decide what the probability of some a in f naught is. So, if a is in f naught then it must be in one of the f i's right. Then a is in f i for some i in natural numbers. See you know that if a is in f i it must also be in f i plus 1 and so on right. So, if you want you can put that i for which you can put the smallest i if you want right, but we will see that it does not matter you can take any i because f i's are nested increasing right. So, if you find an i you can also find other i's. So, never mind. So, a belongs to f i some i. So, if a is in f i then what can you say about the first i bits it must be of a certain form right. Then a must be written of the a must have a representation a remember that guy a superscript i must belong to oh wait a second sorry omega. So, I must really write omega 1 omega 2 omega i is in a superscript i is not it this is my characterization of f i. So, I am saying that the first i bit should have a certain pattern well it could be any one of the subsets of 2 power n possibilities right. This a i some subset of 0 1 power n 0 1 power i actually right define I am going to now define the measure define p naught of a is equal to cardinality of a superscript i over 2 power i you see what I am doing. So, this a i will correspond to some pattern among the first i bits in the example I gave i was 10 and I was looking at all those possibilities where there were at least 2 heads remember right. So, in that case my i will be 10 right. So, I want to assign the probability of well p naught which is the pseudo measure as the cardinality of that set. So, what is the cardinality of that set now of the first 10 tosses you want at least 2 heads is what I said. So, you will have 10 choose 2 plus 10 choose 3 plus that will be the cardinality of that set divide by 2 power 2 power 10 in this case. So, that is what I am doing. So, I am again couching it in some notation, but it is actually very simple. So, if for example, to take a very simple example I am looking at the set of all tosses where the third toss is the head. Then I will have it will be an element of f 3 correct. So, I will have 1 by 2 power 3 in the denominator will be 2 power 3 and here the set of a i's will be there are 4 possibilities right you are fixing the third bit, but first you can be anything. So, there are 4 possibilities 4 over 8 half. So, I am basically saying that the probability of some bit being whatever heads or tails is half and I am also saying that if you give me any specific n bit pattern. So, I may be I tell you that my first 10 tosses should be a very specific string. Then there will only be once such thing here and it will be 1 by 2 power 10. So, this is what you would normally call a fair coin toss except now there is an infinite many coin tosses. So, this is perfectly under I mean intuitive it is just couched in some notation. So, is this clear now there is 1 little issue here. So, remember if some a is an f i it is also an element of f i plus 1 and so on. So, I may for example, choose some i which is not the smallest type. Then you may ask me over is this is this measure going to change it will not change come to think of it. Let us say I take the third coin toss being heads let us say I call it f 4 now it is an f 3, but it is also an f 4 clearly. Now, I am looking at 4 strings in which the third bit alone is heads. So, there are 8 possibilities now and the denominator is 16. So, it is again the same as you can verify that in fact, this is this definition of a probability of p naught of a the pseudo measure is independent of whether you actually manage to choose the smallest type or not. So, you let us say you start define saying this is my smallest type, but it does not really matter you can even choose some bigger i this is invariant under choosing bigger i for example. So, this is perfectly well defined uniquely defined. So, this is what I want now I have to verify that p naught of omega what happens to know p naught of omega I have to verify it is 1 is that true. So, we look at p naught of omega then what will be my first i bits will be all 2 power i possibilities. So, it will be 2 power i over 2 power i. So, my omega will consist of all possible binary strings infinite binary strings. Therefore, my first i bits can be all possible 2 power i strings. So, p naught of so clearly p naught of omega is 1. Now, there is in order to invoke karatheodori we need just one more thing. So, you have an algebra f naught you know that p naught of omega is 1 you just need countability over is f naught right now that is non trivial, but it is true. So, now you can invoke karatheodori and say this p naught can be extended into a unique measure on sigma of f naught. So, you can verify p naught of omega equal to 1 this is easy this is 1 p naught is countably additive on f naught this is non trivial it is not trivial, but true therefore, karatheodori holds. So, by karatheodori p naught can be uniquely extended to a measure on what omega comma sigma of f naught extended to probability measure p on omega comma sigma of f naught. And that is my probability measure that is all there is to it and that probability measure will agree perfectly with p naught on this f naught sets right. So, if you ask me under this measure p what is the probability, but my 13th toss is at head it will be half right because p naught says it is half. So, p will agree with it except now this p is also defined on sets like e see remember e was never on f naught e is the event that there all event tosses are heads let us say this e was not in f naught. So, my p naught will not assign any value to it, but my p will be defined on y is now y is p defined on e it is a countable intersection of f naught sets. So, this must be e is not in the algebra, but it is in the sigma algebra. So, p naught is not defined for e, but p is defined for e. So, now what do you think this p of e is half p of e will be 0 try showing that p of e is 0. So, p of e is the p of a countable intersection right you have to use continuity of probabilities. So, do I have running out of time. So, you can write. So, in this case I will just put a little box and give you a hint for this p of e equal to p of intersection i is equal to 2 4 6 dot dot dot a i right. So, this is like a you can write it as i equal to i belongs to 2 n or whatever right. So, you can essentially what you can do may be you can put i equal to 2 i a 2 i may be and put i equal to 1 infinity. So, same thing now what you do. So, this by continuity of probability is limit n tending to infinity probability intersection i equals 1 through n a 2 i right. This is by continuity of probabilities correct there is no nestate decreasing business here, but this is true by continuity of probabilities. Now, what is that 1 by 2 power n right limit of that is. So, this is this guy is you are looking at there are certain n bits have to be heads the probability of that is you know is 1 by 2 power n right. So, this is 1 by 2 power n. So, this will be 0. So, the probability of all even bits being heads is 0. So, that is an example. So, the p naught is not defined for e, but p is defined for e and it is equal to 0 ok. Let us stop here.