 Welcome back. So, today we will begin the study of probability theory from the basic axiomatic perspective. So, we will start building on the basic probability axioms and work towards building defining probability spaces, probability measures and so on. So, in this mathematical theory of probability. So, it is an axiomatic theory which starts of assuming that there are two entities which are not defined. So, these two so the two entities which are not defined in the beginning are the following. So, the first is the concept of a random experiment. So, this is we do not define what it is. So, just like so we have to be and we have to understand it like just in English. It is an experiment whose out curve is random. So, which brings me to the second undefined concept which is outcome of the random experiment. So, these two concepts are not defined. So, you cannot make a. So, just like in geometry you do not define a point or number theory you do not try to define what a number is. So, you have to start of somewhere you can only define certain things in terms of certain other things you understand you cannot define everything. So, probability begins with two entities which you do not question or try to define. So, there is a random experiments random experiment. So, it is an experiment whose outcome is random just as the English says and it has an outcome. Every time you perform this random experiment there is an outcome. So, that is it. You cannot say anything more about these terminologies that is all there is to it. Now, everything in probability that is defined is in different terms of these entities and we build up from here on. So, the first most important concept most basic concept that is defined is the concept of a samples space. The sample space of a random experiment. So, it is often denoted by the capital Greek letter omega big omega. So, it is defined as the set of all possible outcomes of the random experiment. So, if you see this is a definition of a sample space. So, I have defined a sample space in terms of these two things I have not defined. So, I have defined it in terms of two things which I have not defined. So, that is the way it always progresses. So, you think you understand these two things and you move on and define other things in terms of things you do not divide in the first place. That is how it works. So, this is a sample space. It is just a set. The set containing all possible outcomes of a random experiment. So, whichever outcome is possible everything should be included. You should not leave behind any possible outcome. Now, so you can. So, for example, you can always say. So, if you should give an example. So, let us say toss a coin once. So, here you want to know what the sample space is. So, now the thing I want to emphasize here is what a sample space is corresponding to a random experiment itself depends on what you are interested in. So, it is not. So, if it so happens that I am tossing a coin that is my random experiment. And if I want to know what the face it lands on then the sample space will have two elements in it heads and tails. Assuming that it does not land vertically it will have only two outcomes two heads and tails. However, it is not true that the random experiment of tossing a coin once always have to have this sample space. It depends on what you are interested in. Somebody else may be interested in finding out how many times the coin tumbles in the air. In that case your sample space is not head tail. In that case your sample space will be the number of time all possible number of times it tumbles. So, which is probably positive integers. Student is the definition then the all possible interested outcome. So, which is why the outcome itself depends on what the experiment is interested in. So, I might toss a coin, but you may be interested in counting the number of tumbles and somebody else may be interested in the face it lands in. So, it is not. So, what is an outcome itself depends on what you are interested in which is why the sample space depends on what you are interested in. So, this is something I want to emphasize. So, this is the experiment. So, tossing a coin once. So, if we are interested in the face that shows up the face that shows then your omega will be simply head then tails. It will only have 2 elements only 2 possible outcomes, but for the same random experiment. So, if you are interested in the number of tumbles say. So, in this case it is reasonable to say that your omega will be some subset of natural numbers or the natural numbers themselves. If any number of tumbles are possible then you would want to choose omega equal to n. So, it is not. So, even for a given random experiment it is not as though what is an outcome or what is the set of all possible outcomes is very much a function of what you are interested in looking at. So, it depends on what you are interested in studying or modeling. So, this is something I want to this is a conceptual point. So, I just want to make this very clear. So, you may also you may be interested in the velocity with which the coin strikes the ground. It seems a bit strange may be interested in knowing that. So, in this case probably you should choose R plus or some subset of R plus some positive real numbers. So, you understand what I am trying to say here. So, the sample space capital omega depends on not only on the random experiment, but what you are interested in measuring or knowing. And as you can see just here we have seen that the omega could be finite samples set of all possible outcomes could be finite or infinite actually here it is countably infinite here it is uncountably infinite. So, there are all these possibilities exist you could have an omega which is finite you could have an omega which is countably infinite you could have an omega which is uncountable all this is possible depending on what you are interested in looking at. And so an elementary outcome. So, an outcome of the random experiment. So, let us say you have fixed what you are interested in let us say you have interested in number of tumbles or whatever. Then so an outcome is sometimes known as an elementary outcome. So, this is also a term you know the outcome and elementary outcome are one and the same. So, an outcome or elementary outcome is denoted by little omega this is not a W this is a the Greek letter little omega and this is an element of your sample space omega and big omega. So, now here is. So, this little omega the choosing of the little omega is what is random this is what this is the thing that you have no control over in probability theory this is in fact the source of randomness. So, you can. So, you have a sample space you built a sample space of the set of all possible outcomes depending on what you are interested in. And then this little omega is chosen by some you can think of it as a genie or some goddess of chance who picks it right. You have no control over it and every time you run a random experiment the goddess of chance picks a little omega from this big omega. And that you do not question you have no way of you have no way of controlling that if you can think of it as the weather tomorrow which is like a natural phenomenon or something else right something that you have no control over right or full knowledge of or model ability to model or something like that. So, this is where the randomness is. So, the selection of the. So, you have big sample space this is a set of all possible outcomes and which particular omega realizes when you run the experiment once is in fact the source of randomness you can think of it as being picked by a genie. And it every time you run the experiment it will be a different little omega picked from the sample space. And this is the source of randomness in probability theory one this is the thing that you do not have control this is any questions on this. So, it is an elementary. So, little so an elementary outcome which is why it is called an elementary outcome. So, outcome is a particular point in omega. So, if you can so in this particular case if your omega is this h and t there are only two possible elementary outcomes. So, little omega must be either h or t it could be different at different runs of the random experiment, but it is a single term it is just a one element of capital omega. So, if you are interested in. So, it is really depends on what you are interested in right if you want to measure both you will have an omega big omega correspondingly right no that is fine. So, it is see you have to decide what you are interested in right the you have to see you have to build a sample space after all. So, there is some random experiment going on either you are running it or nature is running it, but what you what you consider the sample space is obviously your function of what you are interested in determining. You may be interested in determining the velocity or number of tumbles or whatever it is that you want to this thing you put that in the sample space, but it should contain all possible elementary outcomes should not leave out any possible outcome that is all right. So, you have to determine that. So, in everything I am going to in. So, in building a probability space we have to it is R responsibility to determine what the sample space is and so on right we will see that it is R responsibility to determine what the probability should be also. So, for another example let us say toss a coin n times and you are interested in determining the phases that show. So, now you are tossing a coin n times the same coin you are talking n times and you are interested in the phases that show. So, now you can either you can look at this experiment as n different repeats of this experiment or you can just say this whole n trials is one random experiment that is also valid way of looking at it right. You toss a coin n times or toss all n coins all at once right and say that string of n outcomes that I see n phases that I see that is really what I am interested in that is elementary outcome I am interested in. So, in that case. So, your omega will be all possible heads and tails all 2 power n possibilities of heads and tails. So, it should be h t power n right. So, each elementary outcome each little omega in the sample space will be like h t t h h h t t some n string of heads and tails and this each of these elementary strings of h t t h is considered an elementary outcome. If you just say that all n tosses corresponds to the random experiment right that is also valid way of looking at it. So, this is what kind of a sample space is this it is a finite sample space that only 2 power n elements right n is fixed n may be 10 or 20 or whatever, but there will be only 2 power n possible elements in the sample space corresponding to the 2 power n possible strings of heads and tails right. Now, you can also now this is let us say n toss a coin infinitely many times and then again you are interested in the all the faces that show. So, now you may get you may well may wonder how do I toss a coin infinitely many times will ever finish and so on right and then will ever finished according may outcomes and so on, but these thing you do not worry about in probability theory again you will just say that you are tossing some coin infinitely many times. So, your omega the set of all possible outcomes is what the set of all possible strings of heads and tails infinite strings of heads and tails right. So, every trial of the random experiment. So, I want emphasize here that one trial of the random experiment is not tossing a coin once right one trial is a particular infinite string of heads and tails. So, in this particular case your. So, this is the infinite strings of heads and tails and your omega for example, may be the string H H T H T H something right some see some string of infinite string of heads and tails this is an elementary outcome I want it is not like that is an elementary outcome. So, now your random experiment itself is tossing it infinitely many times and you do not ask questions like will ever finish will ever be able to count right you do not ask that right that is the experiment and that is a sample space. And what kind of sample space is that is an uncountable sample space because it is like you can say H is 0 T is 1. So, it is like 0 1 power infinity. So, now you are all very familiar with Cantor's argument. So, this guy is an uncountable sample space. So, there many examples you can give right you can take a you can take a line you are throwing dots at the line and the let us say is the 0 1 interval you are throwing dots at the 0 1 interval. And the elementary outcome will be some real number between 0 and 1 and the sample space will be in fact there is interval itself another uncountable sample space right. So, these are some examples. So, in that case our sample space is a set of real numbers. Where is the positive real numbers? It is an uncountable set. No, I am saying it is a continuum of values continuous values we are counting like. Yes ok. So, if you are interested in a particular velocity then the probability that we assume will be 0 always because we are. I have not said that right. So, that is some yeah. So, that is you are getting ahead of me by several lectures right. I am only talking about the sample space now right. What are the set of all possible outcomes? I have not said anything about probability so far have I right. So, sample space everybody with sample space. So, now. So, in probability right. So, we are often interested not in whether a particular elementary outcome has occurred or not. We are often interested in whether a subset of a sample space has occurred or not. So, we may be interested let us say if we toss a coin n times we may not actually say what is the specific string that came up. We may say O is the number of heads even right. So, those are things that are of actually modeling interest so to speak right. So, you may be interested in whether this is your sample space you may not really care the whether this elementary outcome showed up or some other elementary outcome showed up, but you may say O that subset show up right. So, in the case of. So, in the case of this number of temples you can say O that is the number of temples greater than 5 right. So, it is not an elementary outcome in the sense right. So, these kind of subsets of sample space which are of interest to the modeller interest to the person doing this business of mod probability modeling. So, this subsets are called events. So, this is a I just gave a very informal definition because the formal definition requires more build up. So, may be I wonder if I should just write this definition down. So, let us say informally subsets of a subset this informally let me just say subsets of omega which are of interest events. So, I may be so for example, and the so toss a coin thrice and you are interested in the phases. So, your sample space is h t power 3 you may be interested in. So, the event of interest may be this at least 2 heads. So, in this case you are not really asking. So, I toss the coin 3 times I am not really asking if whether a particular string showed up I am asking if at least 2 head showed up right. So, this corresponds to more than one elementary outcome right. So, in particular so what are the outcomes that correspond to this. So, you have say first of all you can have all heads right or you can have t h h right h t h h h t there anything else that is it right. So, these are the elementary outcome when you toss a coin thrice the elementary outcome is a string of 3 string of heads and tails. And the event of interest so you are asking me the question over did at least 2 head showed up that corresponds to these strings which is actually a subset of that of the sample space. So, for all I mean so for all colloquial purposes an event is a subset of the sample space except that is not the entire story that is something you will get to later. For now you can just consider an event as a subset of the sample space, but I put something in the codes here there of interest to us right. So, we will see later that all events are in fact subsets of the sample space events are necessarily subsets of the sample space, but just put a caveat in your mind that not all subsets of the sample space are necessarily considered events. Only those subsets which are of interest which we are interested in assigning probabilities to in particular are called events. So, all events are subsets of omega right. So, you can call this event a for example right this is this event is a subset of omega right. So, all events are subsets of omega, but not all subsets of omega are necessarily considered events and this is not a point that will you will completely appreciate now. So, if you are little bit confused we will get to it right. So, this is something just bear in mind not all subsets of omega are considered events everybody fine. So now, as I said so let us continue on this slightly informal intuitive line. So, events are subsets of sample space which are of interest to me. So, I want to build up a structure of these subsets of omega. So, I want to say that say let say a which is a subset of omega is an event it is of interest to me. So, it is an event. So, I want to say that not a is also of interest to me right when an occurrence of something is of interest to me it is very reasonable to say that it is non occurrence is also interesting to me it is after all one determines the other right if a is of interest to me I should say that a compliment is also of interest to me. So, I want to be able to say that if a is an event an a compliment is an event. So, remember a compliment is a subset of omega right, but since I am only saying that events are subsets which are of interest I want to impose a certain constraint that if a is of interest a compliment should be of interest to me right it is makes perfect sense right. So, that is one thing I want right in this intuitive notion of interesting subsets of sample space. Another structure I want to impose is that if there are let say two events a and b the both subsets of omega of course, a and b I want to know whether a or b a union b is of interest right. So, if a and b are of interest I want to say that at least one of them occur is of interest to me right these are things that I want to impose right. Finally, I want to also say that omega itself which is a subset of itself the sample space that is sure it occur right. So, if any time I run a random experiment the omega itself will occur right after all the omega contains all possible outcomes of the random experiment. So, you may say that omega itself should be an interesting event right the sample space itself as a as considered a subset of itself must be an interesting event right. So, this is the structure we are building up towards this these three things right that omega itself should be an interesting when a is interesting a compliment should be interesting and when a and b are interesting then a union b should be interesting. So, that is what I am building up towards by the way there is just one little definition I forgot. So, maybe I should put that here definition an event is said to occur if omega is in a. So, that is a fairly simple definition I am just saying that. So, I am considering some subset of the sample space I say that let us say that is an event and I say that that event occurs if my elementary outcome lands in that set right. So, this goddess of chance picks a little omega right that little omega may be in this set may be somewhere else right if that little omega. So, happens to be in this a then I say that the event a has occurred right. So, in this particular case if the elementary outcome is any of these guys then I say that the event corresponding to this guy you know at least two heads has occurred that is just a question of terminology it is just a name. So, far so going back to what I was just saying I want to impose certain structure to these events right I want to say that a is of interest a complement is of interest if a and b are of interest there are events then a union b should be of interest and omega itself should be of interest. So, this structure leads to what is known as an algebra this concept is mathematically known as an algebra. So, let us put definition. So, as usual omega is a sample space a collection f naught of subsets of omega is called an algebra following three properties should be satisfied. So, the null set should be in the algebra if a in f naught then a complement is in f naught and three if a. So, a is a set and b is a set in f naught then a union b is in. So, this is scripted f. So, there is one question of notation I want to clarify right away. So, when I am talking about sets. So, when I am talking about sets I will use capital English letters and when we are talking about not necessarily English letters actually even omega this is a capital letter this omega is in the Greek letter after all and when you are talking about elements of a set you will use small letters small omega or small x you know small letters. So, we should write a statement like that should this is an element of a set right and similarly. So, when I write some something scripted right it is a collection of sets right. So, you this is a set right and you collect many of the sets a collection of sets is indicated with a scripted letter. So, this is a convention we will follow we will stick to as much as possible. So, I will have x belongs to a, but when I write when I write the let us say some subset sign these two must be of the same type right this they must be both sets in this case. Here they are this is an element that is a set right and for example, I can write a is an element of f naught f naught is a collection of sets. So, here is an element here is a collection of sets. So, here it is this is an element this is a set this is a set this is a collection of sets. So, that is like a hierarchy right and similarly I may write some f 1 is a subset of f 1 is contained in f naught right f 1 may be some collection f naught may be some other collection this is contained in that means all the sets which are in f 1 will be contained in which are which are also elements of f naught right. So, this is like a hierarchy this is something you have to get used to. So, capital letters sets small letters elements and scripted letters are collections of sets and of course, you can also talk about collections of collections of sets right and so on right this is whole hierarchy. So, in that spirit so I have denoted f naught as a collection of subsets of big omega right. So, big omega is sample space and it has many subsets right possibly finite possibly infinite possibly even uncountable right. So, this has many subsets and that is a collection of subsets of omega is said to be an algebra if these three constraints are satisfied. So, you should have that the null set should always be an element of the algebra all right and when a is an element of the algebra a complement is always an element of the algebra. Some people write omega here right it does not if you have this then 5 complement of that is omega right. So, you can either write omega or phi here. Here you are saying that if a is an element of f naught and b is an element of f naught then a union b is an element of f naught this is like this is the structure that I was talking about. So, any collection so what is an algebra after all. So, an algebra is a collection of subsets of my sample space which has certain properties it must be closed under complementation and if I take two elements it must be closed under union and after all then the null set should be contained in it. So, from these from this structure you can easily prove that if you have n elements of the algebra a 1 a 2 a n let us say they are all in f naught then a 1 the union of all these guys will be in f naught right. So, this may be should do it as a homework. So, exercise let a 1 a 2 a n see these are all subsets of omega let us say all of these are elements of the algebra you can show that union a i is in f naught and intersection. So, n is a fixed number here any fixed finite number and you are considering n subsets of the sample space which are all elements of the algebra using these are called axioms of algebra using these axioms of the algebra you can prove that all these finite unions and finite intersections are in fact elements of the algebra can someone indicate to me how one would do this let us say union first. So, for 3 elements you can say well if you have a b and c a 1 a 2 a 3 let us say you know that a 1 union a 2 is element of f naught. So, if you union that with a 3 which is also an element of f naught you should get another element of f naught. So, we want to just prove it for any n you will use induction you can use induction very trivial. So, this you can prove easily how do you prove that you can use de Morgan's law after all this intersection this can be written as union i equals 1 through n a i compliment the whole compliment right this is by de Morgan's law right. So, after all a compliment or events whenever a a r even a compliment or events and union of all that is sorry I am sorry. So, a i what did I say a i is all elements of the algebra a i then a compliments are elements of the algebra. So, union is an element of the algebra and a compliment same right. So, this is an element of. So, this is very clearly an element of f naught. So, you get done right. So, this is the structure you this is the structure you want to impose on these events right and that leads to this mathematical structure called algebra algebra of subsets everybody ok. So, if you want to say it in words an algebra of subsets simply the collection of subsets which contains null set and it should be closed under finite union finite intersections compliments finite unions. And therefore, under finite intersections it comes out as a consequence of de Morgan's law right it is a collection of subsets closed under complimentation and finite unions not unions finite unions I want to make this very very clear finite unions. And therefore, also finite intersections because of de Morgan's law ok. So, this is the structure you want to impose for these events ideally right. So, events we have still not defined what it is right we only said it is some interesting subset of the sample space right. So, is this enough right. So, let us take an example I have only made this statement that an algebra is I only defined what an algebra is right. An algebra I have said is a collection of subsets which is closed under complimentation and finite unions right this n is finite remember right it can be anything but it has to be finite right. And finite intersections by de Morgan's law that is all I have said. . Which one yeah. So, you can actually jolly well define it like this also right in this case some people define algebra as oh if a 1 a 2 a n is an f naught then the union of a 1 to a n must be in f naught right there both if you define it for 2 elements you get the n element version or you can just define it as the n element union right for all n right it must be closed under finite unions. And this is something you do not put in the axiom because it automatically follows from union and this compliment axiom right you apply both together with de Morgan's law you get this you never put in the axiom because you can prove it right. No it is just a collection of subsets of yeah this f naught can have infinitely many no problem right it can have infinitely many subsets of omega right. But all I am saying is it is an algebra it is called an algebra if it is closed under complimentation and finite unions and finite intersections. No de Morgan's law is valid always for all unions and intersections it is always valid. But I am saying that an algebra is only closed under finite unions ok I am defining an algebra as a structure among subsets which has the property that a null set is contained in it compliments it is closed under complimentation and it is closed under finite unions that finite is very very important ok that is what a algebra is. So it turns out so it is a so you so it seems like intuitively this is enough right because after all you are closed under any finite unions. So it turns out though that in order to make in see in order to actually study events of interest in day to day this modeling of probability actually this closure under finite unions is actually a little bit short of what you actually need ok in order to make this proper interesting probability theory this closure under finite unions is a little bit short of what you need ok you need a little more structure than an algebra ok. So that is what there are some examples you can use to motivate it. So let us say that you are tossing so let us say you are tossing coins and you will toss until you see the first head ok if you have a situation like that let us see what the outcomes are. So this is your experiment ok so you may have the first thing may be a head you are done you will stop right the first time you see a head you will stop or may be you will have a head and then a tail and then you will stop right and or you may have two tails and a head then you may stop and so on right you may have t t h dot dot dot right. But you will toss until you see the first head ok that is your experiment. So this is the set of all possible outcomes right the string of faces that show. See you may be interested let us say so you may be interested in some event like this let us say the event is the total number of tosses even. So this is the question I want to answer and this is what I want to be considered an interesting event let us say this is what I am interested in studying. Now we are so let us say this is my sample space let me say that let say I build my let say I take each of these elementary outcomes and I will let say I try to build an algebra from it. So let if I take all these elementary outcomes so I will take this is an elementary outcome so I have to include it is compliment right then I have to include then I have to take that and include it is compliment and so on and then I will also have to go on and include all these finite unions and I have to include that union that that union that compliment and so on right. So that is how I build my algebra right correct. So according to my F naught now the problem is this guy this event let us say this this may let me call this A right this this A is actually consists of the following elements what does it have T H T T T H and so on I hope this is right. So what is the argument I want to make now so this is all these guys that I am interested in. So this is the subset of the sample space I am interested in. However if I try to just build so if I take elementary outcomes here little so each of these elementary outcomes include them in the F naught and then I have to include their compliments and then I have to include all these finite unions. But if you do that you will notice something like this it will never be contained in your algebra because this as this what kind of this is a countably infinite set right this is also a countably infinite subset of this set right. So this set you can never build by just taking these elementary outcomes taking their compliments and taking finite unions. You can actually verify that this A is now you can never build this A by just taking these elementary outcomes and taking compliments and finite unions after all this is an infinite countably infinite union of elementary outcomes right this is see this is that union that union that but it is a countably infinite union right. So a perfectly reasonable sounding event this is something I may be very much interested in in modeling right what is the what is the event is the event of total number of heads being even just occur right that is a perfectly reasonable event one would think and one would want to make it an interesting event. But the structure we have just imposed namely the structure of an algebra is does not seem to be enough to build this from the elementary outcomes just think about it right. So which is why I said that this structure of an algebra it falls a little bit short of what is actually needed to make probability theory interesting right. So that leads us to a slightly stronger notion a slightly stronger structure among subsets known as sigma algebra that is something you will cover next class. So we have defined algebra today. So we will we motivated why we need not just finite unions but we are going to say that I need closer under complimentation and countably infinite unions and that structure is called a sigma algebra and that is what gives you a very rich probability theory that is what gives you the ability to model even fairly elementary events like this right. So fairly simple events like this which algebra is not able to capture right. So probability theory operates with a sigma algebra of subsets that we will see next class. Thanks.