 So, we start with review of whatever we have discussed in the last class. So, we started with application of probability in engineering, technology, non mathematical sciences like biology, physics, etcetera, etcetera. I try to give you some example that probability is not just a branch of mathematics, but it has a lot of realistic applications that we have to deal with in real life. And not only science and technology, it goes beyond that. Probability application in finance, it has application in management, it has application in almost any kind of subject and processes that you can think about. So, those are real life examples that we discussed in the beginning of the last class. Then we discussed about various kinds of uncertainty and we said that uncertainty is the primary thing why you are having a class like this. Uncertainty is kind of the primary force that leads us to study the theory of probability and how to apply the theory of probability to make proper decisions under uncertainty. So, that phrase decision under uncertainty is very, very important. You will always be given conditions like this is what is available with us. Given that information, given that data, what do you infer? So, again statistical inference, probabilistic inference, those things come into the picture. Then we discussed two different notions of probability. One is the subjectivist, where it is more like a belief of a person, what is the likelihood of some event and the other one is the frequentist, which you are going to follow mostly in this course, which goes by the method of counting based on occurrence of events. You have some hard data and you use that data, you do some counting, do some mathematical calculations and come out with the likelihood numbers. I also said that for both kinds of probabilities, the mathematics, the theorems and other things remain the same. So, whichever notion you apply, you can use the mathematics that you learn over here. Then we discussed about sample spaces, sample event, set theory and Venn diagram, which was a review for you anyway and axioms of probability. Those three axioms, I will again look back at those. So, this was the first axiom that probability of an event E lies between number 0 to number 1 and that makes sense to you. We never say the likelihood of an event is more than 100 percent or less than 0. We never say that it has a negative likelihood. That is something I cannot even imagine how it is going to be. So, this definitely makes sense. The second one, again connected to the first one is that the probability of occurrence of anything in the whole sample space. S is the sample space and we know that the sample space has all the possible outcomes. So, any outcome lying in that sample space, what is the probability of that? That is 1, 100 percent. So, that was the second and the third one, where we said that the union of various events E i from 1 to n, where these events are mutually exclusive. The E i, E j being a null set that intersection being a null set says that these two events do not have any common occurrences, no common outcomes for these two. So, if all E i and E j are like that, then the probability of the union of all these events are nothing but the summation of the probability of all these events. Those are the three axioms and they are of course, related to each other. And then we also briefly went through these probability properties. And the first one says the probability of occurrence of the complementary event E c is 1 minus the probability of occurrence of E. That also makes sense, because 1 is nothing but the probability of occurrence of the sample space S. So, E is nothing but S minus E c. So, P E c you can write as 1 minus P E. As an example, I said this one, if the outcome of an event is the sex of an unborn child, then we can write that this E the event that it is a boy and the complementary event that it is a girl. These two constitute the whole set. So, they are also mutually exclusive. So, for these two we can easily see that the event that the newborn child is a girl child is nothing but 1 minus the probability or the likelihood that the newborn child is a boy child. And you also define something like odds of an event. This is defined in that relative sense in the ratio of the probability of occurrence of an event E to the probability of occurrence of the complementary event. So, if let us say that the event E the likelihood of the event E is 0.45 and the likelihood of the event E c automatically for this one becomes 0.55, then the odds of the event E is 0.45 over 0.55. That was another extension of the probability assumes where you say that the union of two events, the probability of the union of two events is nothing but the summation of the probability of individual events minus the probability of the intersection event. Do you see that clearly? Of course, from said theory you know that that union f is E plus f minus E intersection f. But from a probabilistic sense if you extend that to the notions of outcomes of an event does it make sense to you? Can I have a yes or no? Well you know you can just respond saying yes or no. So, let us go through this example. So, we take the outcome of the event which is the experiment is what is the age of a student in this class. We cite two events over here event E says that the outcome is less than 16 year and 11 months. Event E is above 16 year and 11 months and the other event f is below 18 years. So, are these two mutually exclusive sets? No, they have something in common. So, of course, that P E f the intersection probability of that intersection is not 0 over here. So, E f is all the outcomes which range in between the 16 year, 11 months to 18 years. Now, you can see I hope you can see very easily that the E union f event what will it be? What will be the value of P E union f? Do you have any idea what will be that number P union f for this class? Will it be 1? Because that constitute that should constitute everything. May be P probability of the event a would be let us say 0.4, probability of event f will be 0.75, then probability of the intersection E f will be 0.5. You can do the math and you will get the probability of the union of E and f will be equal to 1. To simplify all these things the best way is to draw a wind diagram. So, let me try doing that. Let us say this is 16 year, 11 months and this is the 18 years. So, this thing is nothing but event E. And this is event f. Now, can you see that sentence that probability of E union f is probability of E plus probability of f minus probability of E f or E intersection f. So, let us get back. Now, we go to the definition of two types of sample spaces. Sample spaces are of two kinds discrete and continuous. Discrete sample space I try to explain that through examples. Let the explain be the outcome of rolling a die. What are the outcomes? If it is a six sided die, then the outcomes are 1, 2, 3, 4, 5, 6. You cannot have any other number. You will only have those six numbers as the outcome and these are discrete numbers which are the possible outcomes of the event. So, we call it a discrete sample space. Take the second one the number of students in first year classes over here. So, I see 102 is one class. Take any other class mathematics class or a physics class. Let us say for this class you have a population of 700 students. For the other class you have population of 650 students for the other 683 students. So, anyway these can have only whole numbers, positive whole numbers as an outcome. So, again that set will be a discrete sample space. Is the concept clear to you? It cannot have any number between 0 to infinity. No, if it could have any number between 0 to infinity then it could have included let us say square root of 2. But of course, the number of student in a class can be square root of 2. It can only be 1, 2, 3 and so on. The third example grade of a student in IC 102. Again these are discrete sample space can start from A, A, B, B, B and so on. But you cannot have see as opposed to expressing your marks as percentage. There you could have said that I have got 80.123 percent or I have got 65 percent. That would be an example of a continuous sample space. But when you are thinking in terms of grades then it is definitely a discrete sample space. So, now you also know about what the continuous sample spaces look like. And here you have two examples. So, room temperature let us say of this room, this hall at any given time in degree centigrade. So, it could be any number, any positive or negative number from minus infinity to plus infinity theoretically. So, that is the continuous sample space or the average age of IITB faculty members. Let us say the average age is at one given time it comes out to be 38 years. At another given time it could be 40 years, 1 month, 2 days and hours, minutes, seconds you can go on. So, again that is a continuous sample space because it could have any number from 0 to whatever be the lifespan of a human. So, these are two different kinds of sample spaces. We are going to use this concept later on when you discuss about different kinds of probabilistic or random distributions, log normal, log normal. I hope you are familiar with the names. So, we will see how do we deal with this different sample spaces. Then we go to the concept of equally likely outcomes. There are certain sample spaces for example, here is site one where we say that this is a sample space of equally likely outcomes. So, S is composed of these events E 1, E 2 to E n and if we know that probability of E 1 is same as the probability of E 2 is same as probability of E i to probability of E n. Then we say that S is a sample space of equally likely outcomes and we can easily compute the probability of each of this event E i as P equal to 1 over n, where n is the total number of events considered. Now, I want to I want you to see in detail of what I said over here. This theorem does it say anything about the nature of those events E 1, E 2 to E n. Other than being equally likely can you say anything else about those events E 1 to E n? Yes, can you be a little louder mutually exclusive and collectively exhaustive? Can you say that for definitely over here? I want you to think out it a little bit. So, example of equally likely outcomes sample space with those is rolling of a fair or unbiased type. The term unbiased will come very often in probabilistic perspective. We also sometimes say it is random. By random we mean that there is no definite direction, there is no definite preference of one over the other. So, that is what an unbiased experiment would be. The second example is a picking of a ball out of a box with 5 red and 5 blue balls. Again these are an experiment you do not know what is there inside the box. You pick out a box, check its color and what is the event? If it is a red or a blue one is equally likely because of the number of red balls and number of blue balls that are there in that box. The basic thing that I want to emphasize over here is that for sample spaces with equally likely outcomes the primary thing that we need to do is to count the total number of outcomes that is N. If we know N, we know that the probability of each outcome is nothing but 1 over N. This sample spaces this may whatever we discussed so far may sound little bit theoretical one, but in real life also we many times see sample spaces with equally likely outcomes very common. So, can you give some example of sample spaces with equally likely outcomes? Give some real life examples not picking a colorful ball out of a box or rolling a die tossing a coin something else something else. I know those are typical examples that we always discuss in a course of probability or mathematics go beyond that. See I have tried to cite some examples in last days class trying to give you some flavor of engineering applications science application biology applications of probabilistic theories probabilistic concepts. So, can you cite something in that sense? Nothing that you can think is of a totally random in nature any thing in that direction. Use the Moodle forum for having these discussions yes cricket matches very good what is the event betting equally likely outcomes are is this side wins or the other side wins, but you will see betting odds are never 50 50. So, you can say usually it is equally likely outcome right that was a good case, but it is not necessarily a equally likely one. Let us leave that there let us go to the concept of counting because you said that for sample spaces with equally likely outcome this is what we do we just count the total number and inverse of that is the probability of each individual event. The basic principle of counting says that if r number of experiments are to be performed and if the first experiment has n 1 possible outcomes and for these if the second experiment has n 2 possible outcomes and the third one similarly has n 3 and so on then the total number of outcomes is nothing, but the multiplication of all these n 1 to n r does it make sense to you I think they should be clear it is like a matrix thing r dimensional matrix. So, the first event has n 1 outcomes. So, you think of n 1 rows let us say the second event has n 2 outcomes. So, now think of for each one of that row think of a column so, that would be n 2 number of columns. So, you have a matrix of dimensions n 1 times n 2. Now, if you have a third outcome then you will have a system with dimension n 1 times n 2 times n 3 and so on on a r dimensional space you will have the product of n 1 to n r. So, for counting we try to use the concept of the basic principle of counting and we use a lot of permutation and combination which I am sure you are very very familiar with. So, I will skip going through details of permutation and combination theories you know a lot more than probably I know and we will emphasize a little bit about the consideration of replacement and ordering. Ordering means arranging or ranking of events. Many a times you do this you know specific ranking of a number of events like when you grading the first thing you do is to rank the numbers and many a times we have to deal with problems of in how many different ways we can rank in how many different ways we can arrange a system. We will go through some examples and a lot of examples are there in the text book. So, I would request you to go through those and the next tutorial problems also discuss with similar concepts. Whenever we do sampling we go through things like if the sampling is with replacement and with ordering it could be with replacement without ordering and all combinations of these two kinds. What do I mean by replacement? Let us take the example of picking a ball out of a box. So, the previous example five red boxes, five red balls and five blue balls and let us say you are supposed to pick three. Now, the probabilities of any event outcomes will be different if you pick the first ball and do not return it back and then pick the second do not return it back and then pick the third. That would have one probability as opposed to if you pick the first one return it back again pick another one return it back and pick another one return it back. So, this is with replacement and without replacement sampling. Similarly, you can have ordering or not ordering well you have gone through permutation. So, you would know exactly what we mean by ordering or arranging. Such examples are there a lot of these things you can find in any mathematics books. So, we will skip that part we will not go through the details we will go through simple examples. So, here we discuss about a party where a person Mr. Jones say invites his friends for dinner and four are professors and we do not distinguish between professors and cheer lawyers. The question is what is the likelihood that the first two guests are one professor and one lawyer and we do not bother about the order who comes first and who comes second. We just say that of the first two one has to be a professor and the other one has to be a lawyer. So, this is how we do the calculation. First we find out the what is the total number of outcomes we just count the total number of outcomes. So, for the first guest we have seven outcomes four plus three for the second guest because we are not replacing we have six outcomes. So, the for the first two guests the total number of outcomes as per the basic principle of counting is seven times six which is 42. Then you go to the specific cases which constitute of having one professor and one lawyer. The first case A which says the person coming first is a professor and the person second is a lawyer. What is the possibility? How many different ways it can happen like that? Number four times number three four for the professor and three for the lawyer. So, in 12 different ways you can have that and if we change the order that means if the first person who arrives is a lawyer and the second person who arrives is a professor then the total possible outcome is also three times four which is 12. So, the total possible outcome of the event that we have one professor and one lawyer is the previous one that is 12 plus this one that is 12. So, this what we have 12 plus 12 divided by the total number of outcomes which is 42. This gives you the probability that of the first two guests one is a professor and one is a lawyer. So, this is very simple simple counting techniques and this was counting without replacement and without ordering for which generic formula is n choose r. Can you guys see the formula over here at the bottom? Yes or no? I take the second example where the same person wants to arrange some books in terms of which category they fall in. So, let us say the same person has this five books or novels, three story collections and two books of others plays essays whatever and he wants to arrange them category wise and we try to find out how many different kinds of arrangements possible. By categories arrangement I mean that novels will be together, short story collections will be together and other books will be together adjacent to each other. So, we find the number of outcomes and we pick typical order let us say at the very left he puts the novels in the middle he puts the short story collections and at the very right he puts the other books. So, the order n s o will have how many different arrangements? Five factorial, three factorial and two factorial, five is the number of novels, three is the number of short story collections, two is the number of other books. So, this is one way of arranging books. The other way of arranging books would be the second one where he puts the short stories at the left in the middle he puts the other books and the right novels. Then you will have the same number of arrangements because it is again three factorial times two factorial times five factorial and so you find out in how many different ways you can put the set of novels, the set of short story collections and the set of other books. So, n s o s o n s n o s o n and so on. You can have three factorial as the type of those orders n s o s n o o s n etcetera. There are three and you can permute them in three factorial different ways. So, the total number of arrangements possible would be three factorial times each. What was the likely outcome number of outcomes for each one? It was high factorial times three factorial times two factorial. So, that times the number of ways you can order them which is factorial three. So, this is the total number of arrangements possible if you arrange them category wise and if you want to find out the probability of having the novels at the very left which is the orders n s o and n o s only two orders possible which is factorial two. Then the total number of this possibility is two times that five factorial three factorial two factorial divided by the total number of outcomes which is three factorial times five factorial three factorial two factorial gives you the probability of having the novels at the very left. Instead of going through all the detail you can also say that there are three types of books I want to have one type at the very left. So, we can arrange them in two divided by factorial three instead of going to the second part which is this five factorial three factorial two factorial right. We are just thinking in terms of the order of book n s o n o s and so on two out of six. Now, you go to the third example and this you will find in any book that you pick from a set of n items a random sample of size k needs to be selected. Now you can see when you say a random sample of size k we basically consider that it is a sample space of equally likely outcomes. Now the question is what is the probability that a specific item let us say a will be a part of those k that you have sampled. And the answer is this does it make sense to you the samples that you are picking those k there has to be a specific one a in that k. That means the possibility is you are given the option of first picking one and then what are the options of having the others how many others out of n minus 1 you are choosing k minus 1. That is the number of possibilities number of outcomes of having one among the k and what is the possible number of ways that you can choose k out of n that is n choose k. So, the total number is nothing but n minus 1 choose k minus 1 divided by n choose k which comes to be k out of n. Then you discuss the concept of conditional probability are you familiar with this concept you have gone through so I will try to go through it quickly. It is written usually in this format and we call it P e given f which is nothing but the probability of occurrence of an event e given that the event f has occurred. So, there is the known information there is the condition that event f has already occurred. So, you are trying to find out what is the probability of occurrence of event e in that situation. And f has occurred implies that the sample space has been reduced to f. I will draw a Venn diagram to explain these things simply and in that f it is only the intersection e f which gives the possibility of having e also occurring given that f has occurred. So, I will shift this is what we are trying to understand through the Venn diagram this is your sample space this is e and that is f. Now, whenever we say given f that means f has occurred. So, now your sample space reduces to f whatever outcomes are there has to come from that f. So, f is the new sample space and in that sample space if e has to occur which is the area representing that this intersection. So, this will be nothing but e f divided by f does it make sense to you and I hope most of you have gone through a similar concept. So, this is how we express the conditional probability. So, to find out the probability that e given f that means f has occurred what is the likelihood of e occurring is to find out that we need to know these two individual probabilities probabilities of occurrence of e and of occurrence e f the intersection. Now, we come to an example dealing with this and we discuss the case of a traffic signal let us say this is intersection of two roads and the one car reaches the intersection it has three options one of going straight which we denote by event s one of turning left event l and the other is to turn right. Let us say these are the three options and these are again what kind of options mutually exclusive and collectively exhaustive right. So, you say that these are three mutually exclusive events and they also constitute the sample set that means they are collectively exhaustive. And let us say we know the probabilities and it is given that the probability of going straight is double of the probability of going turning towards left and the probability of turning left is double of probability of turning right these are known information based on that we have to calculate some conditional probability. So, based on the first information based on these two we can find out individual probabilities see PR turning right has the least probability PL is double of that. So, if you put everything in terms of PR. So, PL is two PR and PS is four PR and these three probabilities summed up will be equal to one because that is the whole sample set. So, four PR plus two PR plus PR will be equal to one based on that we calculate the individual probabilities as PS is four PR by seven PR PL is two PR by seven PR PR is of course, PR by seven PR is that this calculations very simple. Now, the question is what is the possibility that a car turns left given that it is turning. So, you know that it is turning it is not going straight what is the possibility that it goes in the leftward direction. So, this is the probability we have to find out PL given L union R, L union R is of course, either goes towards left or goes towards right can they happen simultaneously no. So, you know that L union R is nothing, but L plus R. So, yeah so all this calculation I do not if you can read that, but we can simply say that this given L union R this is based on simple counting. Can you see how it can be PL over PL plus PR because when you say that the car is turning it reduce the sample space to that set L union R and we know that L union R is nothing, but PL plus PR. So, that is the total number of possible outcomes and we are finding the probability that it goes toward left. So, based on counting we can find out the solution if you can look at this if you can read this you can see that you can also find out just using simple set theory formulas without going through the counting process. And we get the same answer of 2 by 3 now we come to the concept of statistical independence which is very much related to conditional probability. So, we say that if for two events E and F we have that P of probability of E given F is probability of E and probability of F given E is of probability of F. In other words if the occurrence of E does not depend on the occurrence of F and vice versa then we say that these two are statistically independent events. I will go back. So, we say that the probability of occurrence of E which is P E is same as the probability of occurrence of E given F that F has occurred probability of E does not say anything like that. So, this means if F has occurred or not you have this probability and if F has occurred you have the same probability right. So, the occurrence of E does not depend on the occurrence or non occurrence of F and if the other way is also true then we say that those two are statistically independent events. Now, the question is does it also mean that they are mutually exclusive events no right. So, we say that what we say that mutually exclusive events are much more stricter concept that statistically independent events. If those events are mutually exclusive what would you have it would say that if F has occurred what is the likelihood of P of E occurring 0. See I will explain it again with a Venn diagram. So, for two events which are mutually exclusive then what is the probability of occurrence of E given that F has occurred. F has occurred means your sample space has reduced to this shaded area only F your outcome has to lie in that area. Now, if the outcome lies in that area F then there is no likelihood that it would be in this area E. So, for mutually exclusive once you will have this equal to 0. So, when we say mutually exclusive does it mean that they are statistically independent and the next question is the other way around when you say statistically independent do you mean they are mutually exclusive is there any relation can mutually exclusive events be statistically independent no good. And then we extend again the concept of conditional probability through this theorem of total probability. And to get there we will first go through the multiplication rule which is nothing but the extension of the previous formula that we have seen which was for E given F extension of this. So, you just take this denominator on the left hand side and this what you have the intersection of two events the likelihood of that is probability of E given F times probability of F or probability of F given E times the probability of E that is the multiplication rule. From here we go to the theorem of total probability which says that if you have events E 1 to E n which are mutually exclusive and collectively exhaustive does it mean that these are equally likely outcomes of a sample space yes no no ok. So, we have this mutually exclusive that means disjointed and collectively exhaustive sets of events E 1 to E n. Then for an event A in the same sample space S which is considered all these events E 1 to E n. We can write the probability of occurrence of A as it is an extension of the multiplication rule. So, you can see what it is probability of A given E 1 times probability of E 1 plus probability of A given E 2 times probability of E 2 and so on to probability of A given E n times probability of E n. Again I will use the diagrams. So, all these are very handy tool this is total probability theorem this is the sample space which is considered of let us say three events E 1, E 2 and E 3 ok. Now, we consider an event A in the same sample space. So, now these three E 1, E 2 and E 3 definitely belong to the mutually exclusive and collectively exhaustive kind of events for this sample space. So, what is the probability of occurrence of A these what we are trying to find out using the total probability theorem. I can see that this is nothing, but P A E 1 plus P A E 2 plus E. So, this is A E 1 this one is A E 2 and this one is A E 3. Now, if those are the intersections then based on the multiplication rule which we considered over here. We can write the total probability theorem that the total probability of an event in a sample space of several mutually exclusive and collectively exhaustive events are the probability of that event given any of those event times probability of any of those E i's and the summation of that. In real life also you see many application of these kinds of events. I will go through one simple example which says we are trying to find out the likelihood of a dam overflowing. This likelihood we denote by O depends on the flood in the upstream river or the rainfall that occurs in the upstream river. The river situations can be described in three different ways and only these three disjointed events which is number one flooding denoted by F and we know the probability of flooding that is 0.3. If it remains normal the probability of remaining normal level it is 0.5 and if it is remain below normal or low flood level we say it is 0.2. See it is basically the event of flooding might be a continuous sample set, but we are breaking it up into three different regions, three different events of collectively exhaustive and mutually exclusive set. If we know this and if we also know the conditional probabilities that if there is flooding in the upstream river is the likelihood of overflow of the dam which is 90 percent. If it is at the normal level at the upstream river then it is 50 percent of having flood at the dam and if the water level is low at the upstream river then the likelihood overflow is 0.1. If we know these information then we can find out the total probability that it will be overflow at the dam and using the total probability theorem we find it out. So it will be nothing but the probability of O given flooding times probability of flooding which is 0.9 times 0.3 probability of overflowing given normal flood level upstream times probability of normal flood level 0.5 times 0.5 and similarly for the low flood level. So this total information tells you the total probability that you will see an overflow at the dam given various conditions at the upstream river. So this is just a summation rule and of course you can understand that instead of considering this discrete 3 events e 1, e 2, e 3 or f n and l if we consider the events to be continuous in a continuous sample space instead of summation we will do an integration of the conditional probability times the individual probability of those events. I think that concludes today's talk. So this is what we covered we started with the review of probability properties sample space of equal likely outcomes we discussed and the counting principle and discussed examples with counting conditional probability examples of conditional probability we extended that to the multiplication rule theorem of total probability and finally the example on total probability theorem. Thanks.