 As Salaamu Alaikum. Welcome to lecture number 20 of the course on statistics and probability. You will recall that in the last lecture I discussed with you the relative frequency definition of probability and then we went on to the axiomatic definition which is the most mathematical definition of probability. After that we discussed two laws of probability. The first one was the law of complementation according to which probability of A bar is equal to 1 minus the probability of A and after that I discussed with you the general addition theorem of probability. Today we will begin with some examples to illustrate the general addition theorem. As you remember according to this theorem the probability of A union B is equal to the probability of A plus the probability of B minus the probability of A intersection B. In other words the probability that at least one of the two events A and B occurs is equal to the probability of the first plus the probability of the second minus the probability that both of them occur. Let us apply this to a simple example. As you now see on the screen suppose that one card is selected at random from a deck of 52 playing cards and suppose that we are interested in finding the probability that the card is a club or a face card or book. So we have a jack and a king and a queen for spades, for hearts, for diamonds and for clubs. Leaza 3 plus 3 plus 3 plus 3 gives us 12 face cards. In this example in order to compute the probability that this one card that I have selected it is either a club or a face card or both. The first thing to note is that these two events, a face card and a club, these are not mutually exclusive. These are not events like this that if one of them is doing it then the other cannot do it. After all if your leaf is a bird then you will note that both of these events are appearing in it. Since it is a king it is also a face card. Now since they are not mutually exclusive therefore we are going to apply the general addition theorem which is valid for such cases and according to this theorem if I may repeat it once again the probability of A union B is equal to the probability of A plus the probability of B minus the probability of A intersection B. So it is obvious that in order to apply this theorem the first thing to do is to denote one of the two events of my interest by A and the other by B. If in this example we call a face card or club B then our problem will be solved. Of course we can talk about the opposite of it. We can say that club represents the event A and face card is represented by the event B. So that is totally up to you. Now if we have taken this decision that A represents club and B represents face card, now we have to compute their probabilities. You know that according to the classical definition it is M over N favorable over the total. So the probability of A probability of clubs is 13 over 52. Similarly as I told you a little while ago in the entire deck there are 12 face cards. So the probability of face card that is probability of B is 12 over 52. And the third thing we have to compute is the probability of the joint event A intersection B. That means this event is also a club card and a face card. So as I told you earlier if the king or the queen of the club or the jack of the club if you consider these three cards then your joint event is that the card is a face card and that card is also a club card. Because these three cards are probability of this particular event is 3 over 52. Now we will substitute all these quantities in our formula and as you now see on the screen we will obtain probability of A union B is equal to 13 over 52 plus 12 over 52 minus 3 over 52. That is 22 over 52. In other words the probability that if I draw one card out of a deck of cards it will either be a club or a face card or both this probability is equal to 22 over 52 which is less than 50 percent. Now let us consider the special case of this general addition theorem. Suppose that the two events that we are talking about they are mutually exclusive. That means if one of them is doing it then the other is not doing it. In this case you will appreciate and agree with me that the probability of the joint event A intersection B is 0. Because when that joint event does not happen then obviously it is an impossible event and if it is an impossible event then the probability of an impossible event is 0. What happens in this case is that in our formula of the general addition theorem the third term becomes 0 and our formula reduces to a very simple form as you now see on the screen. Probability of A union B is equal to probability of A plus the probability of B. And students is this not exactly the same statement that you read as the third axiom of the axiomatic definition of probability. Let us now apply this to a simple example. As you now see on the screen suppose that we are tossing a pair of dice and we are interested in the event that we get a total of five or a total of eleven. In this situation students you are seeing that these two events are mutually exclusive. Why? It cannot happen at the same time. Either it will be a total of five or it may be a total of eleven. So in this situation we will apply the special case of the addition theorem. Probability of A union B is equal to probability of A plus the probability of B. If we denote total of five as A then we have to think about which outcomes are of this experiment that favour this particular event. Look, thirty-six possible outcomes are one, four, two, three, three, two and four, one. These are four outcomes which are favouring this event that the total of the two is five. Or if we divide this number four by thirty-six then four by thirty-six or one over nine this is the probability that I will get a total of five. Now let us talk about eleven. Eleven can be achieved in two ways. Either you have a five-six or you have a six-five. And when we divide this number two by thirty-six we get the probability of getting a total of eleven as two by thirty-six. If we represent this event total of eleven as B then probability of B is two by thirty-six or one by eighteen. But we were interested in getting a total of five or a total of eleven. So according to the theorem the probability of A union B the probability of five union eleven the probability that I get a five or an eleven is equal to four by thirty-six plus two by thirty-six and that is six by thirty-six or sixteen point six seven percent. This addition theorem for the case of mutually exclusive events can be extended to any number of mutually exclusive events. As you now see on the screen if A one A two so on up to A k are k mutually exclusive events then the probability that one of them occurs that is either A one occurs or A two occurs or so on A k occurs is given by the sum of the probabilities of the separate events that is probability of A one plus the probability of A two plus so on up to probability of A k. You may be thinking that the examples we have discussed so far are very very simple examples. Then why is it that the probability is considered to be the most difficult subject. As it is more difficult there is no subject or no topic. Actually it is true that sometimes the probability problems can be quite tricky. But as I said earlier that all you need is first of all a methodological approach. If you try to approach it systematically then you will be able to progress a lot and it is possible that you will be able to reach the solution very easily. I would like to now discuss with you an interesting example to illustrate this point that what might appear difficult in the beginning may not be that difficult. As you now see on the screen suppose that three horses A B and C are in a race A is twice as likely to win as B and B is twice as likely to win as C. What is the probability that in a particular race either A wins or B wins? Students you have noted that this seems to be a very interesting problem. First of all A is twice as likely to win as B and B is twice as likely to win as C. It is then obvious that the events that A wins or B wins or C wins these events are not equally likely. And classical definition that you have read earlier that is valid only if the various outcomes are equally likely. So how will we approach this problem? As you now see on the screen what we can do is to denote the probability that C wins by small p and if we do that then the probability that B wins is equal to 2 p because B is twice as likely to win as C. Similarly the probability that A wins will be equal to 2 times the probability of B winning and that is equal to 2 times 2 p in other words 4 p. In this problem we are assuming that no two of the horses A B and C can win the race together. That is the race cannot end in a draw and if we make this assumption then of course the events A B and C are mutually exclusive. Either A will win or B will win or C will win but they cannot win together. Since A B and C are mutually exclusive and collectively exhaustive therefore the sum of their probabilities must be equal to 1. And applying this condition we obtain p plus 2 p plus 4 p equal to 1 in other words 7 p is equal to 1 or p is equal to 1 by 7. But if p represents the probability that C wins so we have the probability of C winning equal to 1 by 7. The probability of B winning equal to 2 times 1 by 7 and that is 2 by 7. And the probability that A wins comes out to be 4 times 1 by 7 that is 4 by 7. Individual probabilities we have computed in this way 1 by 7, 2 by 7 and 4 by 7. And you have noted that the first condition that A is twice as likely to win as B and B is twice as likely to win as C is being covered in this. 4 by 7 is double of 2 by 7 and 2 by 7 is double of 1 by 7. But students our question was that what is the probability that either A wins or B wins. And since we have made this assumption that these events are mutually exclusive and no two horses can win together. So obviously we will apply the special case of the addition theorem. As you now see on the screen the probability of A is equal to the probability of A plus the probability of B that is 4 by 7 plus 2 by 7 and that is equal to 6 by 7. Alright, we have discussed the addition theorem in a lot of detail. The next theorem that we will be discussing is the multiplication theorem. But students before I can proceed to the multiplication theorem I will be discussing with you and I need to discuss with you the concept of conditional probability. This is what I mean by conditional probability. The set of all possible outcomes relating to our random experiment is that if we have some additional information pertaining to that experiment the sample space will change and it will reduce by eliminating certain elements as being impossible which were considered possible prior to our having received this additional information. Friend tells you that the die has landed on the floor and it is an even number. He has told you that it is an even number but the interest is that you get a 6 because if you get a 6 then you will be proceeding further in the game. Now having this information that the number that has occurred on the die is an even number this is that additional information that information that will reduce my sample space. If this kind of information were not available then obviously we would have said that there are 6 possible outcomes in our sample space. 1, 2, 3, 4, 5 and 6. But now when we have received this information that the number that has occurred on the die is an even number it is not possible to have a 1, 3 or a 5 with this information that it is an even number. So as you now see on the screen the sample points 1, 3 and 5 are eliminated as being impossible and our sample space is reduced to only 3 sample points 2, 4 and 6. With this reduced sample space students if I am interested in computing the probability that I get a 6 then the total number of outcomes is only 3 it is no longer 6 and the outcomes which are favoring what I want that is only 1 the 6 itself. So according to the classical definition of probability m over n in this case my probability I g that is going to be 1 over 3. If this information were not available then the probability of getting a 6 would have been 1 over 6. As you now see on the screen the probability that you get a 6 given that the number is an even number is equal to 1 over 3. If we denote the occurrence of a 6 by a and the occurrence of an even number by b then this probability will be denoted by p of a given b. Students you may be thinking that its name is conditional probability and what I have just explained was not conditional or conditional. But you see it is quite simple you interpret that the probability that a occurs given that b has already occurred is equal to so much so we can also say that the probability that a occurs under the condition that the event b has already occurred. So this is why we have this term conditional probability. The next thing that I have explained is that the basic concept is that your sample space reduces by eliminating certain elements which now become impossible in the availability of this information that we have. But sometimes students it is not very convenient to compute a conditional probability by first determining that reduced sample space in which we have to then compute our probability. So there is an alternative procedure to compute any conditional probability and without going into the details of the proof I will simply convey to you that formula. As you now see on the screen the probability of a given b is equal to the probability of a intersection b divided by the probability of b and this formula is subject to the condition that probability of b is greater than 0. Now in this formula we note that if we want to remove this type of conditional probability then the numerator and denominator of this formula are both unconditional or simple probabilities. They will be computed not in the reduced sample space but in the original sample space. The second point to note is that the denominator of probability of b should be greater than 0. Why? Because if it is equal to 0 then of course this expression will not be a finite quantity. As you know anything over 0 is equal to infinity or less than 0 so this is the formula. Now let us apply this formula to the same example that we considered a short while ago. A represents the event that the die shows a 6, B represents the event that the die is an even number. So the numerator probability of A intersection B means that we are talking about the joint event that it is a 6 and it is an even number. So there is only one outcome which is even number or 6. And that is 6 only. There is no other outcome which is even number B or 6. So numerator probability of A intersection B when we will do its computation by the simple formula m over n then we will get 1 over 6. So there is only one outcome which is favorable to the joint event A intersection B and total 6 outcomes when you throw the die. Now let us concentrate on the denominator. The denominator is probability of B and B denotes the event that the die shows an even number. Now it is obvious that there are total 6 outcomes in a die and 3 of them are even numbers 2, 4 and 6. So according to the classical definition once again m over n 3 by 6. So now when we divide the numerator 1 by 6 by the denominator 3 by 6 what do we get? 1 by 3 exactly the same result that we had a short while ago when we solved the same question through the concept of the reduced sample space. Let me now illustrate the application of this formula of conditional probability with the help of another example. A mantos is two fair dies what is the conditional probability that the sum of the two dies will be 7 given that number 1 the sum is greater than 6 and number 2 the two dies had the same outcome. Now students in order to solve this problem the first thing to note is that the sample space for this experiment consists of the following 36 equally lightly outcomes 1 1, 1 2, 1 3, 1 4 and so on up to 6 4, 6 5 and 6 6. Now let A denote the event that the sum of the two dies is 7 then obviously A will consist of the ordered pairs 1 6, 2 5, 3 4, 4 3, 5 2 and 6 1 or yes I'm ordered pairs is make you shamil on the issue a K in my sick is equal to you know that the sum is equal to 7. So the total number of ordered pairs for the set A is equal to 6 and therefore the probability that the sum is 7 in other words the probability of A is equal to 6 by 36. Similarly let B denote the event that the sum is greater than 6 then students we will have in the set B all those ordered pairs whose sum is greater than 6 . Therefore the probability of this particular event is equal to 21 by 36 also let us denote by C the event that the two dies have the same outcome. So the total number of such ordered pairs is 6 and hence the probability of this particular event is equal to 6 by 36. All right students. Now let us consider the probability of A given B and the probability of A given C and let us now consider them one by one. In order to find probability of A given B we first need to find probability of A intersection B because in the formula that we wish to apply you know that this expression occurs in the numerator. So for this purpose what we should do is to first of all determine the number of ordered pairs that belong to this particular event. The event A intersection B which represents the event that the sum of the two dies is 7 and that the sum is greater than 6. So students the very interesting point is that the six ordered pairs that represent the event A represent A intersection B as well. Why? Because we are saying that we are talking about A intersection B that is the sum is 7 and the sum is 6. All right because of this discussion we agree that the total number of ordered pairs in the set A intersection B is 6 and therefore the probability of A intersection B is equal to 6 by 36. Now applying the formula probability of A intersection B divided by probability of B we obtain 6 by 36 divided by 21 by 36 and that is equal to 2 by 7 students this is the conditional probability that we will have a sum of 7 given that the sum is greater than 6. The other part of the question was that what is the conditional probability that the sum is 7 given that the two dies showed the same number to students this case may first of all we will be wanting to find probability of A intersection C which has to occur in the numerator of our formula. Now the most interesting and important point is that A intersection C is equal to 5 and that is the null set and why is this? The reason is that none of the six ordered pairs 1 1 2 2 3 3 4 4 5 5 and 6 6 none of them is such whose sum is 7. Therefore A intersection C is a null set and hence the total number of ordered pairs in this set is 0 and this means that the probability of A intersection C is equal to 0 by 36 which is the same thing as 0. And now substituting the values 0 and 6 by 36 in the formula of probability of A given C students it is obvious that this particular conditional probability comes out to be 0. So this is the procedure by which we can compute conditional probabilities using the formula probability of the joint event over probability of the conditioning event. Students you should note the point that conditional probability satisfies all those basic axioms that you studied in the axiomatic definition of probability. As you now see on the screen probability of A given B lies between 0 and 1, probability of S given B is equal to 1 and probability of A 1 union A 2 given B is equal to probability of A 1 given B plus the probability of A 2 given B provided that the events A 1 and A 2 are mutually exclusive. Students let us apply this concept of conditional probability to a more realistic example. Suppose that at a certain elementary school in a western country the school record of the past 10 years shows that 75% of the students come from a two parent home and 20% of the students are low achievers and belong to two parent homes. What is the probability that such a randomly selected student will be a low achiever given that he or she has come from a two parent home? Now in the statement of this problem you have noted that we are saying that what is the probability that a student will be a low achiever given the information that he or she has come from a two parent home? We are talking about conditional probability. Now how am I going to solve this problem? It is fairly simple. Apply the same formula. All you have to do is to denote one of the events by A and the other one by B and if you want to apply this form of the formula probability of A given B is equal to so and so. Then it is obvious that you will call that event A whose probability you want to get because that is the same event which will be written before the slash. And you will call that event B whose information you have obtained because that is the same event which is written after the slash. So as you now see on the screen we will let A denote a low achiever and B a student coming from a two parent home. So applying the relative frequency definition of probability we have probability of B equal to 0.75 and probability of A intersection B equal to 0.20. Maybe you are thinking that in the last question we were talking about classical definition. Now suddenly here we have a relative frequency definition. But students maybe you remember that in the last lecture I have discussed this in detail that in real world situations more often than not we are going to be applying the relative frequency definition of probability. After all in this question what was the statement of the question? It was known that in that particular school 75% of the students belong to two parent homes and 20% of the students are such who are low achievers and belong to two parent homes. This is a proportion given to you and in the last lecture you will remember that if your sample size is large then the proportion we treat as the probability of that event. So the point is students that you need to link up all the various concepts with each other. Whenever you read a new concept then keep the last concepts with you and see how it goes and everything falls into place. Coming back to this example the probability that a student comes from a two parent home is 0.75. The probability that a student is a low achiever and comes from a two parent home is 0.20. According to the formula of conditional probability the probability that a student will be a low achiever given the information that he comes from a two parent home is equal to 0.20 divided by 0.75. In other words this probability is equal to 0.27. Alright let us now talk about the multiplication theorem of probability and students you will be happy to note that it is extremely easy to arrive at the formula of the multiplication low from the formula of conditional probability. As you now see on the screen the formula of conditional probability is probability of a given b is equal to probability of a intersection b divided by probability of b. And if we bring this term which is in the denominator to the other side we obtain the multiplication theorem. That is probability of a intersection b can be written as probability of b multiplied by probability of a given b. But if we interchange the role of a and b then the multiplication theorem can also be written as probability of a intersection b is equal to probability of a into probability of b given a provided that probability of a is greater than 0. Students it is called the general rule of multiplication and it can be stated as follows. The probability that two events a and b will both occur is equal to the probability that one of the events will occur multiplied by the conditional probability that the other event occurs given that the first event has already occurred. The probability of the joint event a intersection b is equal to the probability of a into the probability of b given a. Let us apply this to a simple example. A box contains 15 items 4 of which are defective and 11 are good. Two items are selected. What is the probability that the first is good and the second defective students. What is the probability that the first item selected is good and the second is defective. This indicates that we are talking about the joint event a intersection b where a represents the event that the first item selected is good and b represents the event that the second item selected is defective. Yani cracks of the matter is k and kajo loves a that will indicate to you that we are talking about the joint event and we are talking about the multiplication theorem of probability. So how do we compute this particular probability? Let us denote the first event that is that the first item that we select is good by a and the event that the second item that we select is defective by b. Next let us consider the total number of items that we have and the various kinds of items that we have. As you now see on the screen we have 4 defective items and 11 good items in the beginning of the experiment so that the total number of items is 15. Since the selection of the items is totally at random therefore all the items are equally likely to be chosen and hence according to the classical definition of probability the probability of the event a is equal to 11 over 15. The reason being that there are 15 ways of selecting one item out of 15 and there are 11 ways of selecting one good item out of 11. Yanni the total number of possible outcomes is 15 the total number of ways in which I can select one item out of 15 is 15. But the number of outcomes which are favorable to what I want that is a good item that number is 11. So according to the classical definition m over n it is very simple 11 over 15. Now this is the probability of a where a represents the event that the first item that I select is good. So once we have done this first part of the experiment what is the situation after that? Note that we have thrown out the first item so now only 14 other items are left in that box. And if the first item that we have thrown out that was a good one that means the 11 good items came out of them and we have 10 other good items left in the box. Yanni as you now see on the screen the situation that we have now is that the total number of items is 14 and the number of items that are good is 10. But the number of items that are faulty is 4. Therefore the probability of selecting a defective item after a good item has been selected is 4 over 14. This quantity 4 by 14 again it is very simple m over n which are defective. But students the point that I would like you to note is that this particular probability is not an unconditional but a conditional probability. After all this is the probability that the second item that I select is defective given that the first item that I selected was good. And what am I interested in computing overall in this problem? The probability that the first is good and the second is defective. So according to the multiplication theorem of probability probability of A intersection B is equal to probability of A into probability of B given A and that is equal to 11 over 15 into 4 by 14. And it comes out to be 0.16. Let us now consider an example which illustrates the application of the addition theorem in conjunction with the multiplication theorem of probability. Suppose that a card is drawn at random from a deck of ordinary playing cards. What is the probability that it is a diamond, a face card or a king? Now students in order to solve this question let us denote by A the event that the card drawn is a diamond. And let B denote the event that the card is a face card and let C be the event that the card is a king. Now what we need to find is probability of A union B union C. What is the reason for this? Students A union B union C means that either A occurs or B occurs or C occurs or any two of them occur simultaneously or all three of them occur simultaneously. Now the addition theorem in the case of three events is given as probability of A union B union C is equal to probability of A plus the probability of B plus the probability of C minus the probability of A intersection B minus the probability of B intersection C minus the probability of A intersection C plus the probability of A intersection B intersection C. Students, this lengthy equation is simply an extension of the general addition theorem that you studied in the case of two events A and B. Now, in order to apply this formula, we need to find probability of A, probability of B and all these other probabilities that we have in the right hand side of this equation. So, first of all, what is probability of A? It is equal to 13 over 52. The reason being that A represents the event that the card is a card of diamond and as you all know, there are 13 cards of diamond in a deck of cards and therefore, according to the simple formula M over N, the probability of getting a diamond is equal to 13 over 52. B represents the event that the card is a face card, Jack, Queen and King and you know K, Chidia, Hookam, Paan or Eint, in Charome we have a Jack and a Queen and a King. So, 3 plus 3 plus 3 plus 3 is equal to 12 and therefore, the probability of B is equal to 12 over 52. Now, C represents the event that the card is a King and we have in our deck of cards a King of spade, a King of diamond, a King of heart and a King of club. Therefore, we have 4 kings in all and hence the probability of C is equal to 4 over 52. Now, students in order to find the probability of A intersection B, we apply the formula of the multiplication theorem. The probability of A intersection B is equal to probability of A into probability of B given A and probability of A is equal to 13 over 52, but what about the probability of B given A students, it is equal to 3 by 13. Now, what is the reason for B given A, that we are going to obtain a face card given that it is a card of diamond. Now, since diamonds have 13 leaves and 3 of them are the face cards Jack, Queen and King. Therefore, in this reduced sample space of cards of diamond, the probability of getting a face card is given by 3 by 13. Now, multiplying 13 by 52 by 3 by 13, we obtain probability of A intersection B is equal to 3 by 52. In a similar fashion, the probability of B intersection C comes out to be 4 by 52. The point to note here is that, when we talk about probability of C given B, we note that we are talking about the occurrence of a king given that it is a face card, so students who have 12 face cards in their deck therefore, this particular conditional probability is equal to 4 by 12. And putting it in its appropriate place in the formula of P of B intersection C, the overall answer comes out to be 4 by 52. What about probability of A intersection C? It is equal to probability of A into probability of C given A and what do we mean by C given A? Students, it means getting a king given that it is a diamond. Therefore, the probability of C given A is equal to 1 over 13. Now substituting P of A equal to 13 by 52 and P of C given A is equal to 1 by 13 in the formula of the multiplication theorem, we obtain probability of A intersection C equal to 1 by 52. Last but not the least, we need to compute probability of A intersection B intersection C and students, you will be interested to note that this quantity is equal to the probability of A into the probability of B given A into the probability of C given A intersection B. This is the extension of the multiplication theorem for two events and this can be read as the probability that A occurs and B occurs and C occurs is equal to the probability that A occurs into the probability that B occurs given that A has already occurred into the probability that C occurs given that A and B have already occurred. So as you can see it makes reasonable sense. Now what are the quantities that we need to substitute in the right hand side of this equation? Probability of A is 13 by 52 as we have already noted. Probability of B given A is equal to 3 by 13 as already noted. Now let us look at the probability of C given A intersection B in the last expression. The formula for this is that we are saying that what is the probability that we will have a king given that the card is a card of diamond and is a face card. So students, you should note that the cards of diamonds have three face cards, jack, queen and king of diamond and in these three there is only one king. Therefore the probability of this particular event is equal to 1 by 3. Substituting this value in the right hand side of the equation along with the other values the overall result for probability of A intersection B intersection C comes out to be 1 by 52. And now we are in a position to apply the addition theorem that I convey to you a short while ago. probability of A union B union C is equal to 13 by 52 plus 12 by 52 plus 4 by 52 minus 3 by 52 minus 4 by 52 minus 1 by 52 plus 1 by 52 and students upon solving this expression our result comes out to be 0.42. Hence the probability that the card that is drawn is a diamond, a face card or a king or for that matter any two of these simultaneously or all three of them simultaneously this probability is equal to 42 percent. And students this is the way we apply the addition theorem and the multiplication theorem suitably in any problem according to the statement of the question. Students in today's lecture I discussed with you the addition theorem of probability and the multiplication theorem of probability. Many of the students confuse when to apply the addition theorem and when to apply the multiplication theorem actually if you remember one basic point then you will see that it will not be so difficult where you are talking about the probability that A occurs or B occurs think of the addition theorem of probability or the probability that A occurs and B occurs you think of the multiplication theorem of probability. Best wishes to you in your studies of probability theory and until next time Allah Hafiz.