 So, today we will study discrete probability spaces. So, these are the easiest and nicest kind of probability spaces to work with. They are very simple and I think most of you would have already know what I am going to say. These are the simplest and nicest kind of probability spaces to work with. This corresponds to the case where omega the sample space is countable either finite or countably infinite. So, here this corresponds to omega is countable and when omega is countable we can always take f is equal to 2 power omega. So, we can afford to take the set of all possible subsets of sample space as the sigma algebra and in fact we will be able to assign probabilities to all subsets of the sample space. So, in the case of discrete probability all subsets of omega are necessarily are in fact events. I said earlier that in general not all subsets of the sample space are considered events only the once in the sigma algebra f are events. In our case we are taking f is equal to 2 power omega. So, all subsets of omega are events and we will assign probabilities to all subsets of omega. So, the way you assign probabilities. So, now I have to tell you how I assign probabilities. So, in the case of a countable sample space we assign probabilities to each subset of omega via probabilities assigned to singletons. So, if you have. So, in this case your omega will look like a later look like omega 1 omega 2 dot dot dot omega n finite sample space or it will look like omega 1 omega 2 dot dot dot countably infinite. So, in both these cases when both these cases any subset of omega you assign probability by in fact assigning probabilities to singletons. I will write that down. The probability of each a which is a subset of omega is defined in terms of probabilities p of omega i whatever you know that may be just called p of omega of the singletons subsets of the singletons subsets. And probability of a is simply given by sum over omega belonging to a probability of omega. So, this is this should also be satisfied. So, probability of the assignment should satisfy the assignment should satisfy p of omega sum over all omega n omega equal to 1. So, you have a countable sample space and you would always take f equal to 2 power omega you can afford to do that and assign probabilities to all subsets of omega. So, given any subset a of omega you can assign probabilities and the way you do that is to assign probabilities to all singletons. So, omega will look like that. So, you assign probabilities to each of the singletons in other words you are assigning probabilities to each of these elementary outcomes. See I told you earlier the probabilities are always assigned to they never assigned to elements of omega they are always assigned to elements of f right. But, in discrete spaces it looks as though you are assigning probabilities to each one of these elementary outcomes that is not technically correct. In fact, you are assigning probabilities to elements of f only, but it is the singleton element omega it is not you are not assigning probability to omega 1 you are assigning probability to the singleton set containing omega 1 that is the correct interpretation right which is why some this is why some students are sometimes confused you are assign probabilities to elements of omega elements of f always assign probabilities to elements of f right. In this specific case of discrete probability spaces it looks as though you are assigning probabilities to elements of omega it is true and not true. So, it is when you say p of omega you actually mean the singleton omega alright. So, you are still assigning probabilities to elements of f. So, sometimes you abuse notation by. So, technically you should write a curly brace here to denote the fact that it is a set singleton set right. But then you start abusing notation and forget this curly braces. So, even when I write p of omega without the curly braces I do mean that it is a singleton set it is not the element of omega that I am talking about it is the element of f I am talking about. And given any subset a of given any subset a of the sample space the probability of a is determined by the summation omega belonging to a and just add up those probabilities. Because a can be written as a union of the singletons and it is a countable union. So, you have to have the singleton of disjoint. So, you have to have countable activity satisfied. So, p of a is equal to the sum of the probabilities of the elements in that a right. So, if you have something like this if you a simply consists of omega 2 and omega 3 probability of a will be probability of omega 2 plus probability of omega 3 that is it again by probability of omega 2 I mean the singleton omega 2 right it is not the element omega 2 is this clear. Finally, I must have that the probabilities that I assign must sum to 1 because after all this is nothing, but again countable relativity right. Because if you take union of only singletons right. So, actually this is nothing, but probability of union see the sample space itself is a union of singletons countable union of singletons the after all this omega is countable right. So, these are disjoint. So, it should should become that sum right. So, this is equal to 1 we know right because this is nothing, but the sample space itself right. So, this should be satisfied right. So, that is all there is to it. So, in 10 minutes you can finish saying what is a discrete sample space you just take over f is equal to 2 power omega and you assign probabilities to each subset of omega by in fact assigning probabilities to each singleton. And if you are looking at some non singleton subset a the probability of a is determined by adding up all the probabilities of each of the singletons in that a correct. You should only take care that the sum of all these probabilities that you assign to singletons must be 1 right. And any such probability space omega comma 2 power omega comma p which satisfy all this is a discrete probability space it is a valid discrete probability space. Any questions it is not necessary see any question is it is it necessary that f is equal to 2 power omega right. See the thing is whenever possible you would like to keep all your subset of omega right why do you want to throw away something right. The reason that you cannot the reason that you cannot always take f equal to 2 power omega is because when omega is uncountable you will see that case. Then f equal to 2 power omega turns out to be 2 bigger sigma algebra to assign probabilities to all each and every subset right that becomes actually it is mathematically impossible for in certain circumstances. But in discrete probability cases this problem never arises which is why when you study elementary probability courses you never make this big deal right you always assign you think that all subset of omega in fact events right. So, that is this easy case this is the case you already know right you can assign probabilities to all subset of omega right which means you can afford to take f equal to 2 power omega and assign probabilities to all subsets of omega which is in some sense the best thing you can hope for right you do not want to throw out any subset as uninteresting right because you can keep it and still do a consistent probability space on it correct when omega is uncountable this is not possible. So, we have to settle for a sigma algebra which is not 2 power omega we have to settle for something smaller, but still keep interesting subsets in it right. So, that is something we will get to in a little bit. So, far we are just saying that when omega is countable you can afford to take f as 2 power omega and assign probabilities to all subsets this clear. So, again this does not tell you how to assign probabilities right it should only satisfy this assign probabilities to single tens such that this is satisfied that is all there is right and any such assignment is in fact a valid discrete probability space, but what assignment makes sense for your experiment depends on again what you are interested in modeling and capturing right. So, if you are for example, so if you go back to our. So, if you go back to our coin tossing and you are interested in the phase showing up heads or tails this is a finite sample space right there are only 2 outcomes and use you take f equal to 2 power omega. So, f equal to 2 power omega means what will it contain it will contain null omega itself the single 10 h single 10 t right that you can write out. So, you will assign probabilities to each of the single tens h and t and you have to make it such that. So, you want you want assign probability of single 10 h and probability of single 10 t such that they sum to 1 right any such assignment is a valid assignment. So, you can take. So, probability of h right. So, that is like my probability of little omega here right. So, this you can take it to be some p where and probability of tail must be 1 minus p this is p can be anything between 0 and 1 any such assignment is a valid probability space. Now, probability it does not tell you what this little p should be right and nothing of what we have done says what this little p should be that should be governed by your practical experience or whatever you know. So, if you suspect that your coin that your tossing is a fair coin and it shows up roughly half the time is heads you should put p equal to little p equal to half. If you suspect that your coin is slightly biased towards being a head you should put p is something greater than half right. So, that is something governed by what you think what you are interested in and what you think the coin is likely to do you know. So, this is not. So, probability theory does not tell you what the probability of a coin turning up head is it does not. So, that is something you have to supply to the theory, but once you supply that to the theory it will the theory helps you calculate other probabilities of other complicated events right it will. So, it will be able to tell you if you toss a coin 1000 times what is the probability of getting 600 heads. Those things you can calculate once you make the assignment, but you have to make the assignment in the first place is that clear. So, it is your responsibility to determine this little p if you think it is a fair coin you will put p equal to half. So, this is of you put a measure right you put a measure on this discrete sample space and a more I mean a little more complicated example will be to take an n phase die 1 2 3 n again f is equal to 2 power omega and you can put probability of I equal to p I any p I want such that sum over p I is 1 right that is like an n phase die and if it is a fair n phase die you will take p I equal to 1 over n right. If it is a n phase die and if you think it is fair you will put 1 over n if it is not fair you will put something else would you have to sum to 1 right. So, let me give you some more. So, let us look at another example what do I have here let us look at a countably infinite sample space omega equal to n. So, that looks like that right again I will take blindly I will take f is equal to power omega right whenever omega is countable I will take f equal to 2 power omega. So, now I have to assign probabilities to all subsets of natural numbers remember there is an uncountable infinity of this f is has uncountably infinite number of subsets. But, does not matter I am saying that omega should be countable that is all f can be uncountable does not matter right. So, in this case I want assign some probabilities. So, what will I do I will assign probabilities to singletons right what are the singletons here. So, the natural numbers themselves right. So, to each natural number k yeah assign p k for each k equals 1 2 dot such that sum over k equal to 1 right again I am assigning probabilities to the singletons which are elements of omega right singletons which are singletons subsets of omega can you think of any such assignment. So, this is all you need to satisfy right. So, one such assignment is this right you can put probability of k equals 1 over 2 power k for k equals 1 2 dot. So, probability of 1 will be half probability of 2 will be 1 by 4 and. So, 1 right you can verify that this sums to 1 right this is a geometric series that sums to 1. This is an example of a valid probability measure on the natural numbers or more generally you can or more generally you can write probability of k another such assignment can write p power k 1 minus p that is that I think k or k minus 1 normally actually we write k minus 1 here let us say 1 minus p k minus 1 p right k equals 1 2 dot that is it also sums to 1 right. Let us say examples right these are not this is this does not have to be the only way you can put a measure I am just giving an example of a valid probability measure. So, these are valid measures on natural numbers this measure is called a geometric measure on natural numbers yet another example you take omega equal to whole numbers and again your f is whole numbers are countable right. So, then you take omega equal to f is equal to 2 power omega and you put the following measure. So, you have to now assign probabilities to all singleton subsets of omega which means you have to assign probabilities to all whole numbers right. So, one such measure this is also a very well known measure probability of k is equal to e power minus lambda lambda power k by factorial k this is for k is equal to 0 1 2 dot dot dot and lambda is something that is a fixed positive parameter. So, here again you can verify that if you sum this from k equal to 0 to infinity you will get the summation for e power lambda and you will get 1 right 0 factorial is 1 taken to be 1. So, this is another this is a valid measure on the whole numbers right this is just an example you can do it you can do it whichever way you want just an example this is a famous measure this is called Poisson measure on whole numbers. So, I am just putting measures on these sets these sample spaces omega comma 2 power omega is my measurable space here and I am putting some measure on it right it is all. So, and if you want to compute the probabilities of let us say under this measure if you want to compute the probabilities of all let us say prime numbers what will you do. So, if you want to determine. So, you want to determine the probability of prime let us say you will add sum over all prime numbers this guy right and it is what it is right that is it you want probability of all odd numbers you can add over k equal to 1 3 5 and so on actually if you add over all odd numbers you can actually simplify it right this. So, you can actually simplify get an answer closed formats are there any questions. So, this is all very nice right this is all very nice and simple you know all of this I think right I am just couching it in the language of the sigma algebras and whatever that we have developed. So, the only thing to bear in mind is that although it seems like we are assigning probabilities to each elementary outcome we are in fact assigning probabilities to f measurable sets, but we are doing it through the root of singletons right and you can do this in a countable sample space you cannot do this in an uncountable sample space completely collapses. No it is no see you have assigned. So, if you are looking at the problem in this case or generally in this case of probability of primes right this primes let me call it primes right the subset of whole numbers which are primes. So, you will simply sum over k all that and even you will not get a closed form answer, but it is what it is it is defined no see the sum this summation is see this is a monotonically increasing sequence and it is bounded above by 1. So, it has to converge that number may not have a closed form answer see this closed form business is overrated right I mean y is e power minus lambda closed form and something is not closed form right it is just r 1 perception on what we think are elementary functions right that is no big deal right. If you submit you get some answer you can always do it in a computer to whatever precision you want and this is mathematically this is a convergence series. So, this is a monotonically increasing sequence right each term is non negative. So, the summation is increasing and it is bounded above. So, it has to converge to something right that it will converge to 1 or see in this case it cannot converge to 1, because you are leaving out elements of positive probability this is a sum over this is a sum and this sum as you add more and more terms it is monotonically increasing and it is bounded above by 1. So, it has to converge to something right it has to converge to something in this case it has to converge to something strictly less than 1, because I mean there are non prime numbers which have positive probability right it is some real number that is all you care does not matter that you do you cannot simplify it right. If you are summing over odd numbers you can actually simplify it if it gives you any comfort, but it is conceptually the same thing fine. So, it is all very easy. So, when you go to uncountable sample spaces life is not very easy right that is why you need all the machinery of measure theory. Otherwise if uncountable sample spaces were as simple probability theory will be very easy right there is nothing more to it all this technical machinery of measures and sigma algebra will be not very necessary right. So, you move on to uncountable sample spaces there are no questions on discrete sample spaces. So, let us take an example of let us say omega equal to the interval 0 1. So, you want to deal with this sample space. So, you have this interval and let us say you are interested let us take a very specific example let us say you are throwing darts on this line throwing a dart on this line it will it will land somewhere in this line. And you want to model mathematically the intuitive concept of the dart landing uniformly on this 0 1 interval. So, you want to have the sense that the dart is equally likely to hit anywhere between 0 and 1 that is the intuitive thing we want to build up towards. So, in some sense I want to be able to say that if I have some set here which I am interested in probability of the dart landing on that set. And I move that set around here the probability of landing on that set should also be the same right. So, in this this is the experiment we want to model let us say. So, essentially in more formal terms we want to assign a uniform probability measure on this 0 1 interval which is an uncountable we know that 0 1 is uncountable right the interval 0 1 is uncountable. Now, in this case it. So, happens that if you take 2 power 0 1 2 power omega which is the set of all possible subsets of the 0 1 interval it is a very huge collection of sets. It is actually you can show that 2 power omega has a strictly bigger cardinality than even the continuum the real numbers. In fact you can the cantor showed the theorem that all power sets have the power set of any set has a strictly bigger cardinality than the set it is. So, 2 power omega will have a strictly bigger cardinality than real numbers or 0 1. So, that is 2 bigger sigma algebra 2 assign probabilities to in fact there is an impossibility result that I will state, but that gives that brings us to the problem that you cannot assign probabilities to all subsets of the 0 1 interval right. So, 1 see the elementary approach of simply assigning probabilities to singletons will definitely not work because I mean if you were to assign each probability. So, let us say there is an omega here between 0 and 1 if you want to assign probability to that little omega well it cannot be anything positive. Because if you were to put a positive probability on that singleton little omega you would want to put the same probability on another singleton. Because you want the notion of a uniform probability measure, but then you will quickly find that there is an uncountable infinity of these omega's and if any of them has positive probability the probability of the interval will blow up right will go to infinite that is meaningless. So, the only thing you can do is the singleton should have 0 probability right that is the only way to make it work, but if you put 0 probability on a singletons that does not tell you the probability of an uncountable set right. Suppose I put probability of 0 on all the singletons and I ask you what is the probability of the interval half 1 this has uncountably many singletons in it right, but you cannot add probabilities over uncountable unions you can only add them over countable unions right. So, even if I assign 0 probability to each of the singletons I am not able to figure out what the probability of let us say an interval half 1 is right. Because I have not add the probabilities of these I mean I cannot add the probabilities of these uncountably infinite sets unions right. So, this is a problem. So, the way out of this is to stop worrying about singletons the idea is to directly assign probabilities to sets we consider interesting subsets of omega we consider interesting. So, abandon singletons all together for an uncountable sample spaces we will only say that for example intuitively we would like to say that the probability of my dart landing in this interval half 1 should be what 1 by 2 I want it right that is that is what I want. So, similarly if I have some interval a b or something I want the probability of the dart landing there should be b minus a right that is what I want right. But of course, I have to define the proper sigma algebra and so on and assign the measures properly. So, this requires some work this stumped a lot of mathematicians for a few centuries it was only sorted out in early 1900s by Borel Ebay you know right. So, this is all this business is only 100 years old people have been studying probability for several centuries, but this proper theory consistent mathematical theory particularly for uncountable sample spaces is only 100 years old. So, we want this. So, we want a measure right. So, we want the measure of the whole thing is 1 of course, but then we want measures of let us say intervals like that a b to b b minus a and. So, even if I move it around I want it to be equal to the length right. So, this is what we want. So, I want to put a measure mu or probability measure p such that mu of the interval a b equal to mu of interval a b equal to mu of a b equal to mu of a b closed all of this equal to I want it to be proportional to b minus a right. So, this what I am saying here I am just saying that the dark landing in open a b or close j b or half open half close is all equal to they are all equal right. Because I want to I do not want to say that a dark landing at some point that should be probability 0 right as I just argue I want this property and I want the property of translational invariance. So, if I have a which is a subset of 0 1. So, if I give you a subset of 0 1 right I want my measure mu. So, I am trying to define a uniform measure mu satisfying certain properties one of them is translational invariance. I want mu of a equal to mu of a plus x where a plus x is the set of all little a plus x such that a belongs to a and a plus x is not equal to 1 union a plus x minus 1 that a belongs to a and a plus x greater than 1. So, this is complicated notation for something very simple I will tell you what it is. So, this a o plus this o plus x is simply the translation of the set by a some fixed number x. So, if I give you a set some set and I move it by x that is what this is saying except if you move a set by a certain x it can go outside 1 in that case you wrap it back that is what this business is doing. So, if your a plus x is less than or equal to 1 you are happy if not you wrap it back right you subtract 1 and bring it back here. So, this is the translational invariance property. So, I want my uniform measure to satisfy these two properties this is the I want a measure which satisfies these properties right uniform measure to satisfy these properties. So, I am demanding that my intervals have a measure right a b intervals of the form a b have a measure and that open or closed should not matter the measure should be the same and I also demand that for any subset a not necessarily intervals it must have the translational invariance property. If I demand these two things which is perfectly reasonable for a uniform measure it turns out you cannot do this on 2 power omega there is an impossibility theorem. So, there is an impossibility result. So, this is something I will only state the proof of it is beyond the scope of our class I will not hold you responsible for it I will only state what it says it simply states that there exist no such measure on 2 power omega there does not exist a definition of. So, I should say there does not exist a measure it is not exist a measure mu a defined on 2 power omega i all sets of all subsets of 0 satisfying 1 and 2. So, this is a this is a theorem that is proved for example, Rosenthal has a proof of this the book by Rosenthal has a proof of this, but the proof is by contradiction you assume that there exist a measure on 2 power omega and construct and show that 1 of the axioms of the measures will not hold. So, it is a proof by contradiction, but it is it is not very important for us. So, what is important for us is to simply know that you cannot this very perfectly intuitive properties that you like uniform measure to satisfy cannot be satisfied on 2 power omega you cannot possibly assign measures to all subsets of 0 1 satisfying the axioms of measure and satisfying these 2 properties. So, in plain English it is impossible to assign a uniform probability measure on 0 1 satisfying some very simple properties like this a very intuitive properties. So, there are other impossibility results as well there are other impossibility results that say that this is only talking about a uniform measure. So, there are certain large class of continuous measures which can never be defined on uncountable on 2 power omega. So, all this just means that we have to what should be what we have to do now we have to compromise on a sigma algebra. So, we could very happily take 2 power omega as our sigma algebra for discrete probability spaces, but when omega is uncountable 2 power omega as a luxury we cannot afford. So, we have to work with a smaller sigma algebra, but still we have to figure out a way of retaining subsets that are of interest to us. So, I will get to the details next class, but what essentially these early people who develop the foundations of measure theory what they did is that I want to keep interesting subsets I cannot keep all subsets. So, I have to keep interesting subsets. So, if you take a real line or 0 1 interval or something like that people decided that the interesting subsets are the intervals. So, I want to keep a smaller sigma algebra than 2 power omega. So, I want to create a sigma algebra that is smaller than 2 power omega, but contains all the intervals of the form a b and. So, you decide to include your intervals in your sigma algebra. So, now you include intervals you also have to include .complements countable union of intervals and so on. So, that is what lead to this concept of a Borel sigma algebra and the sigma algebra the elements of the sigma algebra are called Borel sets that is the word you might have heard it is a sounds familiar to some. So, that is so Borel sets are interesting subsets of 0 1 interval for example. So, the Borel intervals are all Borel sets and all countable unions and countable intersections and complementations of intervals also are Borel sets. So, that is what you will build up towards next class we will deal with this more formally. So, are there any questions at this point. So, I will stop with the impossibility result for today. So, we will prove that single 10 sets are Borel sets. So, I have not gotten to it yet that is a result we will prove any other question thanks.