 Welcome back, we were discussing events right. So, we defined events as subsets of omega which are of interest right. So, this is a very loose and informal I mean nobody would think that this is a mathematical definition. So, it is just some plain English kind of a definition for now right. So, we have been trying to formalize this what is it that we actually mean by events right. So, we have this sample space omega which contains all the possible elementary outcomes of a random experiment and we are looking at subsets of this sample space. We said that all events are in fact subsets of omega, but not all subsets of omega are necessarily considered events right. So, this is not a point you may fully appreciate at this point, but we are building towards understanding this right. So, one structure we imposed for events yesterday was that if A was an interesting event A complement should be an interesting and if similarly if A and B are events if they are interesting then A union B should be interesting right. And so, this lead to the structure called algebra right. So, an algebra F naught is nothing, but a collection of subsets of omega which are closed under complementation and finite unions or also under finite intersections because of De Morgan's laws right. So, we argued yesterday that the structure of an algebra sometimes falls a little bit short of being able to describe events of day to day interest right. So, we gave an example in terms of this tossing a coin until you get a head you know and we looked at the event that is even number of tosses right. So, that was not easy to describe in terms of the algebra itself. So, it turns out that to do an interesting probability theory to build a rich theory you need a little more than the structure of an algebra alright. So, that is so all of probability theory works with slightly stronger structure known as a sigma algebra right. So, sigma algebra is nothing, but a collection of subsets again of omega which are not just closed under finite unions, but closed under countably infinite unions countably infinite unions not necessarily closed under arbitrarily infinite unions, but countably infinite unions. So, that is what brings us to the concept of a sigma algebra. So, let me write down the definition of a sigma algebra a collection f of subsets of omega is called a sigma algebra if null set is in f 2 if a is in f then a complement is in f finally. So, these 2 are the same as in an algebra alright the last axiom of the sigma algebra is different from the axiom of an algebra if a 1 a 2 dot is a countable collection of subsets in f then union a i in f. So, that is the definition of a sigma algebra. So, again so this is scripted f right as I said we denote collections of subsets collections of sets using scripted letters. So, we use f scripted f for a sigma algebra and these 2 are the same. So, if the null set should be in f always if a is in f then a complement should be in f and only difference with an algebra is that if you are given any countable collection of subsets in omega the union of those subsets countable union of those subsets must be in f. So, there is one little point about say in a notational sense. So, this union right. So, if you look at the write up I had in the set theory bit. So, this is really interpreted as union i belongs to n. So, they had a little remark in the write up we had uploaded on set theory. So, this is simply. So, this is not like you are you are not unioning a 1 then a 2 then a 3 and so on. This should be interpreted as the set of all elements contained in at least one of the a i's. This should not be interpreted as some limit of some finite union or any such thing there is no such notion. So, this is better interpreted like this it is union over all a i's I belong to n which means this is the collection of all little omega's contained in at least one of the a i's right. So, that is just an aside. So, this these three constitute the axioms of a sigma algebra. So, it turns out that this structure is enough right you need. So, as I mentioned that the structure of an algebra falls a little bit short of what we actually need to build a very interesting theory of probability. It turns out as soon as you impose closure under countable countably infinite unions it is enough to develop a very rich theory of probability. So, all of probability theory works with a sample space under sigma algebra of subsets defined on the sample space. So, you can also show by the way. So, exercise you can also show that if a i. So, this is all subsets in f then you can show that intersection is also in f using De Morgan's laws again this clear. So, you apply this and that together you will get the fact that the sigma algebra is closed under countable intersections as well. Yes. That is ok. So, one thing I want to make very clear is that see f naught both the algebra and the sigma algebra right they I am not saying that for example, that an algebra should only contain finitely many subsets that is not what I am saying it can have any number of subsets, but it should be closed under finite unions. Similarly, a sigma algebra can contain any number of subsets it can even be an uncountable infinity of subsets, but it should be closed under countable unions right. It can have even uncountable number of subsets in it. So, this f may have an uncountable infinity of subsets of omega in it right, but it should be closed under countable unions that is all we are imposing. I am not saying for example, that the cardinality of f is countable or any such thing no right it should be closed under countable unions. Is this clear everybody. So, this is very easy to show this is a very simple exercise. So, one thing you can show also is that a sigma algebra. So, this is one exercise number 2 a sigma algebra is also an algebra. So, what we are saying is that a sigma algebra is a strictly stronger structure than a well yeah we have not said it is a strictly stronger structure. I am saying now that every sigma algebra is a algebra which means that if you have closure under countable infinite unions you will have closure under finite unions why you can always take beyond. So, let us say you can take a i's beyond a n you can take all the a i's as null sets right and you will have closure under finite unions right. Just take after a n plus 1 onwards you take as null sets right then you will get closure under finite unions right. So, this is this is very easy to show it is a very simple exercise you will show it. What turns out is it is not very it is not entirely trivial to show that there are algebras which are not sigma algebras. So, that is an exam we will see an example in the first homework, but the converse is not true the converse to this is not true not every algebras necessarily a sigma algebra every sigma algebra is necessary in a algebra. So, the converse is not true and. So, in order to show that not every algebras sigma algebra you just need to produce an example right you will see one such example in the homework slightly non trivial, but you will see in the homework if you do the homework you will see. So, is this everybody with me any questions. So, subsets. So, another terminology subsets in f are called f measurable sets it is just another terminology. So, omega is a sample space and you are collecting subsets of omega and you make a sigma algebra f and depending on what the sigma algebra is elements of that f are called f measurable sets. And there are subsets of omega which may not be in f right those are not called f measurable sets those are just some subsets of omega right. So, there are some examples. So, if you have some trivial examples of sigma algebra we can give all right some non trivial examples we will see later. So, the most trivial sigma algebra in some in any sample space omega is that right. So, it is a collection of subsets of omega only containing the null set and the sample space itself this is a sigma algebra right it is a very very trivial example right it is of no use in most situations, but it is a sigma algebra right this is once as example. So, I am just trying to say that you can build many sigma algebra for a given sample space all right and which sigma algebra you want to work with again depends on what you are interested in. So, not only is it your responsibility to build a sample space based on the outcomes of the random experiment it is also your responsibility to decide what you are interested in and what subsets of omega you want to you want to include in your f all right. So, another such example is. So, you may for example, just say phi some event a some subset a complement and omega right this is also sigma algebra actually there also algebra because they are sigma algebra right. So, here what are you saying here there is other than phi and omega which is always there in the sigma algebra there is one more event one more subset a which is of interest to me. So, I am including a, but if I have to include a I have to include a complement all right. So, this is another sigma algebra this corresponds to only one interesting event other than the null set a right and it is complement at the other end of the spectrum is 2 power omega what is 2 power omega that of all possible subsets of the sample space right include all possible subsets of omega right. So, omega is natural numbers for example, you will include all possible subsets of natural numbers. So, if omega is n f will be in this case f will be 2 power n right. So, you can see already that 2 power n is an uncountable collection of subsets right, but we are only imposing closure under countably infinite unions not necessarily uncountable union. So, it is just closed under you are imposing only closure under countably infinite unions and there is everything in between right. So, this is completely trivial this sigma algebra just has one event you want 2 events a and b you will have a b a union b a intersection b and all their complements right and then omega right that is that is how you would do it right and at the other end of the spectrum of the very end of. So, this is the biggest sigma algebra that you can define on omega right and it consists of all possible subsets of omega the sample space right. So, given so which one you choose depends on what you are interested in again. So, it is not true an algebra is not a sigma algebra we will give an example. See in these cases they are all they are both algebra and sigma algebra these very simple cases, but they you can construct examples where an algebra is not a sigma algebra it is possible we will see in the homework these are trivial examples. So, now that brings us to the. So, this brings us to the question. So, if I choose f is equal to 2 power omega right I can say that all subsets are interesting to me right. So, in some sense this may be the if I can choose f is equal to 2 power omega include all subsets of omega as being interesting as being f measurable right. Then I can talk about all possible subsets as being interesting and eventually assign probabilities to them right. So, that is what I am heading towards the only issue is this this is again an issue you will not appreciate now when omega when the sample space is finite or countably infinite. Then you can actually afford to take f is equal to 2 power omega include all subsets in this sigma algebra and still do a still assign probabilities to them. So, for countable sample spaces both finite and countably infinite sample spaces you can actually assign probabilities to all subsets of the sample space. It is possible to do that I will we will get to the all this right. I am just giving you a preview now, but if omega is uncountable right if omega is uncountable like 0 1 interval or real line or something like that. Then the power set of that uncountable set is too larger collection to assign probabilities to. This is again something you will not appreciate now it is not possible to always assign probabilities to all possible subsets of let us say the real numbers uncountable sets. Therefore, this problem arises you cannot always take f equal to 2 power omega especially when omega is uncountable you would have to settle for a sigma algebra which is smaller. This is something you will you will appreciate little more later. Otherwise, we can see if this problem never arises we will never we will just keep f s all subsets of omega right. I want to keep ideally I do not want to throw anything out right, but all this missionary becomes necessary only because it is not possible to do a interesting probability theory a consistent probability theory for certain uncountable sample spaces. So, now another definition so omega f is called a measurable space again another terminology. So, the beginning will introduce a bunch of terminologies. So, what is the measurable space it is some set omega endowed with a sigma algebra of subsets right the pair omega comma f is called a measurable space. This f can be any sigma algebra it does not have to be any of this particular one. So, you know it does not have to be any particular sigma algebra on omega as long as you endow omega with some sigma algebra f this omega f is called a measurable space. So, the one thing that you should know about probability theory is that it is actually just a special case of measure theory and probabilities are simply special cases of measures which is why some people talk of probability measure. Now, have you heard that terminology you do not just say probability, but say probability measure right. So, it is a special case of a mathematical concept called a measure which we will define very soon. So, let us define a measure this is definition. So, measure is a function mu from f to 0 infinity included such that mu of phi is always 0 and number 2 if a 1 a 2 dot is collection of this joint f measurable sets. Then the measure of the union countable union of a i is equal to that is the definition of a measure. So, it is a mapping from. So, you are given some measurable space omega comma f and. So, this measure is a function from f to 0 infinity included. So, it can actually take values. So, it can take any real value or it can take the value plus infinity. So, it is not just a real valued function it is an extended real valued function. So, it takes values from 0 infinity included and. So, what is this mean you are assigning measures to what subsets not all subsets of omega I want to make this very clear only f measurable subsets. So, measures are assigned not to all subsets of omega, but to those subsets which you have decided to include in your sigma algebra which is a judgment call you already made let us say depending on what you think is interesting to you. You created some sigma algebra f and you say omega f is my measurable space why is it called a measurable space I am going to put measures on this space. It is something you can put what is the measurable space after all it is something you can put a measure on that is why it is called a measurable space. So, very this actually very fairly simple definition. So, it is it takes as input f measurable sets and produces a real number or plus infinity positive real number or plus infinity and does not take negative values that is all it takes all values from 0 to plus infinity. We must necessarily have that the measure of the null set is 0 now. So, this null set remember this null set is an f say which is why we impose this condition know remember this. So, the null set is always f measurable and that is specific f measurable set phi has measure 0 always it is one of the axioms of measure. The second axiom of measure is known as the countable additivity axiom. So, this says that if I have a bunch of a countable collection of this joint f measurable sets then the measure of the countable union is equal to the sum of the individual measures. See these a i's are after all f measurable. So, mu of a i's will define right some positive number and also since a i's are f measurable the countable union is f measurable right. So, mu is also defined for the countable union right. So, and the axiom imposes that if you take disjoint a i's. So, if I have some big sample space omega and I have disjoint events disjoint f measurable sets disjoint means they do not have each of these a i's have null intersection correct. So, a i intersection a j is null for all i j that is what disjoint means right. So, if I take the measure of the union of a i's right. So, I am only drawing a finite number of them obviously, but there are actually a countable infinite number of these a i's they are all disjoint pair wise right. And if I take the measure of all these guys the union of all that guys is equal to the addition of the measures of each of them. It is actually very intuitive come to think of it think of measure as something that says how much is contained in there or something like that right. So, if they are disjoint I want to be able to add the measures right. So, there are these two axioms yes no I am talking about the generic measure a measure is a function from your. So, it is a function that maps f measurable sets to 0 infinity and you impose two conditions right one is that the null set should have 0 measure the other is that if I have disjoint a countable. So, the countable union of disjoint f measurable sets the measures can be added up right this axiom if you note down is called the countable additivity axiom this is a very this is actually the function this is the most important one right this is a fairly trivial one this is the most important property of a measure. Now, the triple you just let us say the triple omega f mu right. So, now you have defined some measure on this measurable space. So, you started with omega and then you endowed it with a sigma algebra f. So, this this pair you called a measurable space in anticipation that you are going to put a measure on this space. Now, that you put a measure on this space this is called a measure space now it is it has a measure a measure space. So, what is a measure space it is a triple consisting of a set a sigma algebra on that of subsets and then a measure defined on the f measurable sets right. So, this property should be satisfied for that good. So, far we have actually defined a bunch of things it is actually conceptually fairly simple just that the thing is very abstract at this point right you probably do not have very concrete examples I am deliberately keeping it abstract because we can then once we develop the property develop a proper theory we can give number of examples and make several special cases all right. So, far logically everything is all right. So, now remember that see this phi is in the sigma algebra right always and therefore, omega is also also in the sigma algebra right. So, the whole space is always if measurable omega is always if measurable. So, omega must have a measure associated with it because I cannot leave out any f measurable set correct. So, mu of omega is well defined correct. So, if mu of omega. So, what value can mu of omega possibly take no I mean I have not said anything about it right. So, mu is just a so, mu maps f measurable sets to 0 infinity infinity included correct. So, it could be that this is finite right. So, this is again something very intuitive right if you if you so, happen if it so, happens that your mu assigns only a finite value to the entire space then it is called a finite measure and similarly, if this is equal to infinity plus infinity then mu is called a it is called an infinite measure all right. So, mu of the entire sample space is plus infinity then you say the measure is an infinite measure if it is something finite you say it is a finite measure right. Finally, the case that is of most interest to us is if mu of omega equal to 1 right can take any real value it can be finite or infinite if it is finite you say it is a finite measure a finite measure in particular if omega is equal to 1 then it is called a probability measure. So, in that sense a probability measure is a very special case of a measure it is a finite measure in particular with which assigns 1 as the measure to the entire sample space. So, that is really all there is to it in so, in this sense probability theory is a special case of this measure theory. So, but so, this is not a course on measure theory right this is a course on probability theory. So, we will not go on and on about what generic measure spaces we will study probability measures in greater detail. So, in particular when mu is a probability measure when mu of omega is equal to 1 we will no longer call it mu we will call it p and we will say omega f p and we will not call it a measure space we will call it a probability space. So, we because we are studying probability I want to write it down. So, that there is no further confusion. So, probability measure. So, I have already defined what a probability measure is, but I want to do it again right just because this is so important. A probability probability measure it is denoted by p with two lines standard notation p on omega f is a function p mapping f measurable sets to what is it mapped to 0 1 no satisfying is a function that satisfies p of null is equal to 0 p of omega equal to 1 and then if a 1 a 2 dot dot are disjoint f measurable sets then probability of union i equals 1 to infinity a i is equal to sum over i equals 1 through infinity probability of a i. So, this is just a repeat of what you have seen. So, probability measure is nothing but a special case of a measure with this additional property. So, the only difference between a measure and a probability measure is that probability has this property an ordinary measure need not have this property it measure can be anything positive it can be a positive real number or plus infinity. So, this one thing I like to point out again. So, this is an important matter it could cause confusion. So, this I said right. So, if you have this union i equals 1 to infinity or a better notation in my opinion is union i belongs to n a i. So, this is the set of all omegas contained in at least one of the a i. So, this union is not defined in terms of some finite union going bigger and bigger or anything like that. However, this summation this is an infinite summation infinite summation is actually the limit of a finite summation. So, this is like limit n tending to infinity summation i equal to 1 to n probability of a i. Now, if you write down an infinite summation the question that comes to your mind is does it converge right. I will happily written down this summation this summation etcetera these are infinite summations right. You have to mean to make in order for this to make sense you have to argue that this is in fact well defined right. Why is this well defined yeah no see when you write a summation like this the one thing you do not want is see you do not mind if it goes off to infinity then you call this whole summation equal to infinity right. So, you do not want this summation to be something undefined right. If you like if you write down i equal to 1 to infinity minus 1 power i or something like that for example, it is something that keeps jumping right. So, that summation is not defined at all, but such a situation is not arise here why not. So, each yes yes yes so mu of a i is always non negative. So, if you write down. So, if you really interpret this summation is nothing, but limit n going to infinity summation i equals 1 through n mu of a i correct this is the definition of the infinite sum. So, what I am saying is that if you look at that as a sequence in n call it some x n or something it is a monotonically increasing monotonically non decreasing sequence in n. So, monotonically non decreasing sequence has to either converge or go off to infinity which also is in some sense converge right. What it does not do is oscillate or do any such thing. So, if mu is not positive you cannot have this being well defined right. Similarly, for probability measures again this is a well defined summation except what you are constrained here is that after all what is on the lift should also be at best 1 right. You cannot the map values can never exceed 1 right. So, you have to have a situation where the summation is not only well defined it has to be at most 1 cannot be bigger than 1 right. Good and finally, omega f p is called a probability space. So, we have come 3 steps right. So, what is the probability space you start off with a random experiment and collect all it is possible elementary outcomes and make your sample space. And then you collect subsets of the sample space which you think are interesting which you are interested in assigning probabilities to alright. And you end omega with a sigma algebra of subsets of omega alright. And finally, so then omega f will be a measurable space then you finally, assign probability measures which satisfies these rules these axioms. So, these are called axioms of probability then these 2 are called axioms of measure these 3 are axioms of probability. So, you end up with a probability measure right. And this triple omega f p is called a probability space this is where everything begins when. So, this is the beginning of what we are going to build up on. So, this one thing I want to make clear. So, this does not tell you. So, none of this what we have to what we have developed here tells you how to generate these f or how to assign probabilities alright. It does not tell you how to do this is up to you right. But, if you were to do it it just tells you the rules you have to satisfy within these rules. You can do whatever you want you can create any f you want satisfying those rules you can create any p you want satisfying these rules right. But, how you do this how exactly you do this is up to you depends on your intention right or depends on what you want to model or what you want to capture right. So, it just gives you a frame work it does not give you answers immediately right. If you supply some numbers to the theory it will give you some other answers that is all. So, it is your responsibility to create f create p etcetera. So, questions. So, as I said it is a map from f to 0 1 right. So, it takes as inputs f measurable sets right. So, in the specific case of a probabilities. So, if you have. So, now, we are after all the story we are ready to give a precise definition of what an event is right. If omega f p right if you are looking at a probability space f measurable sets are called events that is all there is to it. So, you go and put a sigma algebra on omega depending on what is interesting to you right. Whatever you include in your f is an event what you do not include in f is not an event that is simple as that. So, let me just write that down as well. So, if omega f p is a probability space. So, f measurable sets are called events. So, now, we have a very precise understanding of what events are. So, far we have been giving some wishy-washy definition that over their subsets of sample space which are of interest to you right. Now, this is the correct definition right. So, in a general measure space. So, in the general measurable space you have omega f mu right some measure and the elements of f are called f measurable subsets of omega. And the specific case of a probability space you do not say f measurable set you say it is an event right. So, there is no difference between f measurable sets and events as far as probability spaces are concerned. Is that clear to everybody? Sigma this guy omega that is not true. So, we are saying probabilities to those sets that are in f f is a sigma algebra on omega right. You are saying probabilities to only those subsets of omega which are in f. So, that f will always contain phi and omega or it may contain other it will contain other subsets of presumably, but it will not necessarily contain all subsets of omega right unless your f itself is 2 power omega. See f is a collection of sets it is not a subset of omega it is a collection of subsets of omega right. So, unless f itself f is 2 power omega you will not be assigning probabilities to all subsets of omega right. What I was saying a little earlier was that when omega is countable finite or countable infinite you can actually afford to take f s 2 power omega and you can assign it is actually possible to assign probabilities to all subsets of the sample space right when omega is countable. But when omega is uncountable it turns out that if you take f equal to 2 power omega assigning probabilities consistently to all subsets of something like real line is not always possible. This is not something you will understand now I will this is an impossibility theorem which is a non-trivial theorem. So, if you are working with an uncountable sample space omega which is like say like real number or 0 1 or something like that. Then you have to define a sigma algebra on omega which is smaller than 2 power omega. So, how you do that is the interesting part. So, that we will get to later. So, far any question any other questions. So, if omega is countable let us say omega is n for example, we know that 2 power omega is 2 power n. So, f will have uncountably many subsets, but that is I am saying if omega itself is uncountable. Then it is 2 power omega is even bigger in some sense. So, if omega is 0 1 for example, or r 2 power r is a collection of all subsets of r that is a very huge collection of subsets. And it is too difficult it is in many cases it is impossible to assign interesting measures on it. Yes, if omega is countably infinite also you are fine either omega is finite no problem omega countably infinite no problem. It does not matter that f has uncountably many subsets omega if it is uncountable. If the sample space is uncountable you get into trouble. So, it is you have to be much more careful. So, probability theory is actually very easy when omega is countable if it is either finite or countably infinite probability theory is very easy, because you can take f as 2 power omega. And assign probabilities to all subsets of sample space which is what you are actually used to. But if omega is uncountable you get into trouble. That is something you may not have studied in elementary courses. So, I just want to make sure I am done with everything I want to say here. Yes, we have defined probability space event is. So, an F measurable set is called an event. So, are there any other questions? All of measure theory is a very important branch of mathematics in itself. And the only thing that is not included is this mu of omega can be anything positive or plus infinity. Probability theory is a special case of measure theory, but it is also a very interesting special case. It is a very important special case, but measure theory in its own right is a very important part of mathematics. So, even things like length, area, volume, these are all examples of measure. So, the concept of a measure generalizes things like length, area, volume, etcetera into arbitrary measure spaces. I mean satisfying certain properties. So, it is a very important area of mathematics. Is there anything else? So, we will stop the lecture here. So, I want to find out