 We were discussing independence of events. So, we said. So, we fixed a probability space and we said that A and B are independent. So, two events A and B are independent under this probability measure if probability of A intersection B equal to probability of A times probability of B under some other measure they may not be independent. So, it is a whether two events are independent or not depends on what probability measure you are talking about. But, I would not keep saying that they are independent under measure P because when it is understood what measure it is. It is not clear we have to specify under what measure they are independent. Are there any questions on this? We discussed this in reasonable detail last class. So, now I want to move on and define it for this is just for two events. So, I will the program is that I will define it for now n events. Then I will define it for an arbitrary collection of events possibly infinite collection probably even uncountable collection of events. Then we will move on to defining independence of sigma algebras that is something you will not be familiar with independence of events you would have seen at various levels. So, this is one definition two. So, a 1 a 2 dot dot dot a n are independent if for all i naught if i naught subset of dot dot n. We have probability of intersection i in i naught a i is equal to product i in i naught probability of a i. Did I write it correctly? Yeah for all actually for all non empty i n i subset of i naught i naught subset of 1 2 n. So, what this means is that independence of let say n events or 3 events you have to not only verify that the product of the three way intersection products out. You have to verify that every possible subset intersection also products out into the individual probabilities. So, if you have a 1 a 2 a 3 for example, it is not enough to verify that probability of intersection a 1 a 2 a 3 is product of the individual probability. So, you have to verify it all two at a time and then all three at a time similarly for any events you have to verify that all possible subsets have this property. So, if you have any events how many equations do you have to satisfy how many equations do you have to check 2 power n all non empty some of the 1 of them is non empty it is 1 of them is empty. So, I should say for all non empty i if for all non empty non empty i naught all that. So, now I have to there are 2 power n possible subsets of 1 through n, but 1 of them is empty and the single consultants lead you to trivial equations. So, you have to actually verify 2 power n minus n minus 1 equations to be true each 1 of them has to hold true in order to assert that a 1 through a n are independent even 1 of them fails to hold they are not independent. You can for example, you can create examples of 3 events which are all parallel independent a 1 a 2 will be independent a 2 a 3 will be independent a 3 a 1 will be independent, but they may not all be independent. You can also create examples where probability of a 1 intersection a 2 intersection a 3 products out, but 1 of the couple do not product out all these are possible, but you have to have all 2 power n minus n minus 1 equations checking this verifying this in order to assert that n events are independent. Any questions I think this is something you already know right this is something you would have seen I think at least for 3 events you would have seen it any questions. Now, how do we talk about independence of an arbitrary collection of events see today's class is going to be all definition definition definition you just have to understand all these definitions properly. So, if you have not let us say not n events you have some arbitrary collection of events a i and that index at a i the index it may not even be some countable index it may be some uncountable index it. So, if you have let a i i belongs to i be an arbitrary collection of events they are all if measurable events and this index it may not even be 1 2 3 dot it may be index by real numbers or some such a uncountable set the collection of events itself may be an uncountable collection in that case how do you say that these a s are independent. So, now see independence of so this is probably an so this is may be an infinite collection of events. So, independence is only verified for finite subsets it is only you verify for every possible finite sub collection of these events you verify this right these events are said to be independent if for all non empty and finite i naught sub set of i we have same thing. So, what do I have I am given some arbitrary collection of events index by i right for every finite sub collection. So, a i i belong to i naught will be a finite sub collection of event for every such finite non empty sub collection I need to verify this right definition there is no y to it right this is definition clear. So, the number of conditions you have to verify is also possibly infinite infinite right. So, you may not be able to do it 1 by 1. So, you may have to prove a general statement saying for every any sub collection i naught I have this relation holding. So, my a i s are independent as an example. So, if you recall. So, let us say example recall the infinite coin toss model and let a i be the event that the i th toss is heads. So, what does a i consist of remember the infinite coin toss model right. So, it the sample space is 0 1 power infinity and you put an appropriate sigma algebra on it which we discussed and we also put a uniform probability measure on it which corresponds to fair tosses right. We said probably we said the probability of any particular toss being head is half remember right. So, a i consist of all those infinite strings in which the i th toss is heads right. So, what is probability of a i probability of a is half right because you had the other bits. So, a i is an element of f i what we discussed right. So, and there are 2 power i minus 1 possibilities for the rest of the bits then the denominator is 2 power i right. So, probability of a i occurring is half now what we can show is that a i and a j are independent for i not equal to j. You take the event that the i th toss is a head and the even the j th toss is a head under the probability measure that we assigned we can show that a i and a j are independent events you know how do we do that. Correct. So, if you have let us say j is bigger than i for the sake of argument then both a i and a j will be f j measurable events correct the bigger of the 2 f j measurable events. And what is the probability of a i intersection a j you want the i th bit and the j th bit to be heads right. And the rest of the guys you can have anything 2 power j minus 2 possibilities and denominator is 2 power j. So, 1 by 4 is the probability of a i intersection a j. Similarly, that is equal to probability of a i times probability of a j why because probability of a i is half then no we are proving independence therefore, a and a j are independent is that clear. So, any 2 so the event of the j th coin toss i th coin toss coming up heads is there 2 independent events in fact what you can show is. So, indeed indeed a i i greater than or equal to 1 are the independent events what do I mean now a i is the event that the i th toss is a heads I am saying that this whole collection of events independent this is the what kind of a collection of events is this no no no no indeed no see I am this is a collection of events right. So, this is what this is like this it is a countable it is a infinite collection of it is a countable infinite collection of events and I am saying they are all independent now. So, what do I have to verify I have to verify definition 3 right. So, for every finite sub collection I have to verify that this relation holds. So, what will I do. So, I have to verify infinitely many relations right. So, I cannot do that. So, I will fix any finite sub collection you want how many elements do you want i naught let us say you want k elements for any k I will prove a general statement saying for any k take i 1 i through i i k there are arbitrary indices in this i they may not even be consecutive or anything right. Now, you do the same trick right there all f i k measurable right you are only fixing those particular bits right you can compute the probabilities right the same trick. So, you would have proved a general statement about any finite sub collection of these events will satisfy this and therefore, by definition they are all independent point of this is that clear I just complete this argument I think I it is a very good exercise to do. So, you have to fix an arbitrary k and an arbitrary i 1 i through i k and then prove the statement. So, this you will show homework are there any questions there is no mathematical induction involved you are just proving it for an arbitrary k and arbitrary i 1 i 2 i k it does not work it is not true if you prove for 2 it is not true for it is not true for 3 any questions on this should try doing this this is a very instructive any questions. So, you know now you can talk about independence of an arbitrary collection of events if I give you any collection of events possibly even an uncountable collection what you will do is you will collect you will look at all finite sub collections and look for the products the joint whatever the intersection producting out right. Now, independence of sigma algebras. So, now you are only talking about independence of events and collections of events now you will talk about independence of 2 sigma algebras and then eventually independence of a collection of sigma algebras. Now, see now you are talking about the probability measure p right you are all this independence is being discussed with respect to this probability measure p right and this p is defined on which one f right. So, if you want to speak about independence of 2 sigma algebras let us say f 1 and f 2 on omega you have to you can only look at sub sigma algebras of f because p is only defined on f right. So, we will only discuss it is not even meaningful to discuss p may not be measured on may be not be defined on some other sigma algebra right. So, we will say let f 1 and f 2 be sub sigma algebras. So, what does that mean no I am looking at the sub sigma algebras of f I am asking what that means. So, by sub sigma algebra I just mean see f is some collection of sub sets of omega f 1 is a sub sigma algebra means f 1 is contained in f. So, every sub set contain in f 1 is also an element of f similarly every sub set contain in f 2 is also an element of f, but they are both sigma algebras. So, if you want. So, can you give me some example of a sigma algebra and a sigma sub sigma algebra yes. So, that is a very trivial example right you can take a phi omega to b 1 sigma algebra which is a sub sigma algebra of phi omega a a complement right that is a very simple example if you want to be little more complicated you can take let us say omega equal to 0 1 let us say 2 power omega is still a sigma algebra 2 power omega is a sigma algebra, but boral sigma algebra is a sub sigma algebra of 2 power omega right actually in your home work you are encountered in the first home work I think right there was the collection of all sub sets of 0 1 which are countable or have countable complements remember we call it f 3 or something in your home work that is a sub sigma algebra of boral sigma algebra you think about it only have countable sets and sets with countable complements it will be a sub sigma algebra of boral sigma algebra. So, these are just examples good. So, you are you are with this you have 2 sub sigma algebras of f and you want to speak about the independence of these 2 sigma algebras remember p is defined on f. So, p is defined on elements of f 1 and f 2 as well right f 1 and f 2 are said to be independent if for any a 1 in f 1 and any a 2 in f 2 a 1 and a 2 are independent. So, I have 2 sub sigma algebras f 1 and f 2 they are collections of sub sets of omega you pick any event you want from f 1 and any other event you want from f 2 right it is completely up to which one you pick. So, no matter which one you pick those 2 events must be independent events if it is true for every such a 1 picked from f 2 and every such a 2 picked from f 1 and f 2 respectively then f 1 and f 2 are said to be independent sigma algebras. See if you happen to pick null or omega will trivially hold if you happen to pick omega from here and something else from f 2 it will trivially hold. So, for phi and omega there is no problem, but rest of the events the independent should hold is this clear yes. This is exactly what I said. So, f is a p is defined on f. So, no matter which one you pick from f 1 and which one you pick from f 2 p is defined on see f 1 and f 2 are after all sub sigma algebras. So, measure defined on f is also defined on f 1 and f 2. So, all of this is independent with respect to that measure p defined on f which is why I said see if these were not sub sigma algebras if they were 2 some other sigma algebras which are not necessarily contained in f then you may not be able to define p on it which is why I said sub sigma algebras understand. So, if I take f s say I cannot take f s let say borel on 0 1 and take f 1 as something bigger because Lebesgue measure for example may not be defined on some bigger sigma algebras that does not work. So, you have to have sub sigma algebras yes. So, there all sub sigma algebras is f. So, if just a intersection I take a sample space of f 1 and just any other element in f 2 there will be any kind of joint. No, that is not true. So, I do not understand your question. So, f is see f is a sigma algebras defined on omega and f 1 and f 2 are sub sigma algebras. Yes, because they are sub sigma algebras of f. So, if you take sample space of f 1 as 1 event and then it is intersected with. No, the sample space is omega. The sample space is always omega. They are not on different sample spaces. In fact, they are on the same see the sigma algebras f and the measure p are fixed and I am just looking at sub sigma algebras f 1 and f 2 of f. And I am intersecting with other elements of f 2 there will be any way some Yeah, but they will trivially satisfy independence. So, let us say a 1 you pick as omega. So, in particular f 1 is a sigma algebras. So, it must contain omega and a 2 you pick as whatever you want. Then you will get probability of omega intersection a 2 which is just a 2 equal to probability of a 2. So, that is very trivially satisfied and similarly for null set. So, these are not problem creators at all. So, if at all this fails to hold it will fail to hold only on some other non trivial subsets. So, independence of 2 sigma algebras means all these events here are independent of all these guys here. Anything you pick here and anything you pick here are independent. Finally, I will just put more one more definition. Let f i i belongs to i b and arbitrary collection of sub sigma algebras of. So, the next logical step is to move to see event from 2 events to some collection of events. Similarly, we are going from 2 sigma algebras to some arbitrary collection of sigma algebras. This i may be finite infinite even uncountable probably. They are all sub sigma algebras of f. What do you expect now? What do you think the definition should be? You pick any a i from f i any whichever way. So, from each of this a i speak in each of this f i you pick any a i you want. Then you will end up with the collection of events a i indexed by capital i. Then I will apply this definition that is all that is a very simple definition. These f i's are said to be independent sub sigma algebras f z. Said to be independent f for any choice of a i in f i i in i we have a i i belongs to i are independent events. So, you are given some arbitrary collection of sub sigma algebras of f. From that collection each of the sigma algebras will contain f e omega and a bunch of other subsets. Now, from each of the sigma algebras you pick any a i you want. No matter which way you pick this a i if a i i belongs to i are independent events. Then this collection of sigma algebras is said to be independent sigma algebras. And what is the meaning of this I have to go to this. So, I have to select this could be a infinite collection or even an uncountable collection. So, I go back to that definition and say for every finite i naught I have this whole. So, you have this hierarchy of definitions are there any questions. So, this is very it is very important to digest these definitions should really you should really internalize them. Because this will come up when we do random variables and so on. We will define independence of random variables in terms of these notions. No, no, no see you are given a bunch of sub sigma algebras which are indexed by i. So, this is one sigma algebras this is one sigma algebras so on. From each of them you pick any event you want. So, no matter which way you pick it that collection you end up with which is also index by i. See after all this a i is picked from f i right and no matter which way you pick this a i. So, which is why I have said for any choice of a i in f i i belongs to i this collection is an independent independent collection of events. And independence collection this is what I mean by a i is an independent any other questions. So, let me just give an example. So, I just want to make one general remark which may not be so relevant to you the see the fact that you are given some let us say you are given some arbitrary collection of sigma algebras. It may even be an uncountable collection of sigma algebras the fact that you can pick one. So, I am what am I doing I am picking one a i from each of these f i's. But if this is uncountable collection can I even do it. So, you this is called axiom of choice you assume that you can pick one element from any arbitrary collection arbitrary set. So, all of this measure theory probability and all assumes this axiom of choice to hold. So, because you will ask me right or if I what if I give you some uncountable collection of sigma algebras how do I pick one from each of them you will never exhaust it. So, there is no notion of time or picking or any such thing it just assume that you can do it right this is consequence of this axiom of choice. So, again so to give you an example of this. So, I have may be 10 more minutes let us see. So, a i so as usual a i is the event that the i th the coin toss model. So, now again a i is the event. So, omega i equal to 0 1 power infinity a i is the event that i th toss is heads is 0. Now, let me see let me define. So, let c o equal to the collection of all. So, this I am define a collection of events c o as those a i is for which i is odd and c e as the a i is for which i is event So, I what I mean is i is 1 3 5 and so on here i is 2 4 6 and so on the claim is. So, claim let f o equal to sigma of c o and c e is equal to sigma of oh sorry f e equal to sigma of c e yes f e equal to sigma of c e. So, f o consist of all those events which only have to do with the outcome of the odd tosses and f e is a sigma algebra which consist of the outcome of which can be decided only by the event tosses. You can show that f naught and f e are sub sigma algebras of what we called f it is requires a proof and you can show that f naught and f e are in fact independent sigma algebras. So, this is an example to show you can. So, can show f naught f. So, these are sub sigma algebras of f and under the measure p f naught and f e are independent. So, in order to do that what you do you fix any event b which is a. So, any event pulled out of here and any other event pulled out of here must be independent. So, an event. So, the core of the argument is the following any event pulled out of here will only depend on the outcome of the odd tosses and any event pulled out of here will only depend on the outcome of the event tosses. But, you know that you know this fact that the across these indices the tosses are independent. So, you have to put these two things together make it precise you can show that f naught and f e are independent sigma algebras. So, this is something again you should try I mean I am saying this in all seriousness it is not all that trivial it is the definition is easy to understand, but executing it will take some work. So, I will I think I should stop here see I will just mention one fact see I will not actually I will not. So, I will say this on another occasion I will just I will just state at this point that if you want to verify the independence of two sigma algebras. See sigma algebras are fairly complicated let us. So, let us agree on that they have all sorts of accountable unions in them. So, if you take Borel sigma algebras consists of several complicated sets. So, if you have to verify the independence of two sigma algebras you have to you have to go about pulling out all sorts of complex sets from here and here and show that independence holds whether the measure products out on all these complex sets in the sigma algebras. However, there is a saving grace in order to show that two sigma algebras independent you can it is enough to show that independence holds on certain generating classes alone I will just roughly put it at that level. So, if you have so if you have a collection of sets which are closed under finite intersections only finite intersections no compliment no countable union anything. So, a collection of sets closed under finite intersections if you verify independence under those for those collection of events alone it is enough to say that the sigma algebras generated by these collections are independent. I will state this properly as a theorem when we move on to random variables, but at this point I want to say that you do not have to worry about really checking independence for all sorts of complicated sets Borel sets and all these complicated subsets that exist here. It is enough to verify on a certain generating class this generating class which are closed under finite intersections they are called pi systems. So, that is a like sigma algebra there is a word for it pi systems. So, independence under pi system is enough to show independence of the sigma algebras I will state this later just make a mental note now that it is not as difficult as it seems you do not have to pull out all sorts of complicated events from the sigma algebras and verify I will stop here.