 Hi, I'm Zor. Welcome to a new Zor education today. I would like to present you an interesting probabilistic problem And let me just spend a couple of words about the history of this problem I was Preparing some lecture on statistics in particular statistics and correlation between statistically defined Random variables experimental data, and I was trying to find an example when two Random variables are dependent on each other And yet their covariance or correlation coefficient Is equal to zero and I was trying to do it in a very simple case with with the random variables Taking only two different values and every time I came to strange Results that variable cannot actually have two different values. It should have only one value a constant basically being a constant and I was surprised tried it a couple of times and Had exactly the same result. So then I went back to The pure theory of probabilities and thought that maybe there is there is basically some theorem which can be proven this way and Indeed I was able actually to to prove this and I'm going to present it to you as a problem basically Now as usually I'm trying to encourage you to Solve problems just by yourself first. So I do recommend you to go to unizor.com to this particular lecture it's in probabilities and random variables problem number seven and There is a problem basically is explained And then there is a solution and don't pay attention to solution first try to solve it if you can't then then obviously go to solutions, etc All right, and then listen to my lecture and after that when you finish all that Just open the new sheet of paper and try to do everything just by yourself It's a very good exercise All right, so now let me just present you a pure problem Let's consider you have two random variables and I was trying to actually find as simple variables as possible So my random variables C takes values A and B and my variable Eta random variable Eta takes values C and D Now we were talking about their Dependence or independence, which means we have to really get involved in their mutual probabilities of taking certain values at the same time, right? Now I put it in a table so my C and D would be the values of Eta and My A and B would be the values of C and I have probabilities P, Q, U, V as Mutual probabilities, which means the probability of C This is C and this is and this is Eta the probability of C to take A and A to take C at the same time is P. That's their mutual probabilities And all other pairs like probabilities of C to take B and A to take D is V, right? Now, obviously P plus Q plus U plus V is equal to 1 because they completely encompass the whole Set of all possible elementary events and each elementary event is obviously simultaneous taking a value for C one of these two and for Eta Okay, so that's what we have Now what's given is that their covariance or correlation coefficient, which is the same thing Is equal to zero Okay, so covariance is equal to zero Now what I have to prove is basically that there are no non-trivial random variables Which are at the same time dependent on each other, but still have the covariance is equal to zero now If it's independent, we know that for independent random variables covariance always equals to zero Now since I was trying to find dependent ones Well in this particular case when only two values are available for each of these two random variables The answer actually is there is none. There are no A dependent non-trivial dependent way and I'll explain what not trivial means. There are no non-trivial dependent variables which take only two values each with such a covariance equal to zero now But I mean non-trivial Okay, the trivial case is when a is equal to B or C is equal to D and these are actually constants So it's not really like a random variable. So if we in the very beginning Consider the case when a is not equal to B and C is not equal to D Then the only possibility for this to be true is independence between C and and eta and that's what I'm going to assume. So I assume that a is not equal to B and C is not equal to D these two Variable random variables are non-trivial non-constant basically so they have more than one value with different probabilities Okay. All right, so I have to prove that they are Independent of each other now to prove independence. See, let me just Spend a couple of minutes about to talk about dependent or independent now you remember that independent events for instance event a and B they are independent if Conditional probability of a and the condition that B occurred is equal to unconditional Now what is this? this is actually probability of a and B at the same time occurring divided by probability of B This is basically what it is because if you remember if you have this is the The the sample space that where all the elementary events are represented. This is a and this is B What I'm saying is this is a this is B and this is AB and both occur at the same time, right? so a Well geometrically the area of a B Relates to the area of B is actually the same way as area of a relates to an entire space That's what basically independence mean all right, which Which means from this? that P odd a B equals pilot a times P of B Right if this in the conditional probability is this now if we're talking about random variables Which take certain discrete values then absolutely the same way if And Another variable takes one of its values So this simultaneous probability is equal to Product of their corresponding Probabilities where Xi is one of the values which you can take and why J is one of the values with it which a data thing takes Take takes so if this probability of simultaneous taking these two is equal to the product for any values Xi and yj then the Variables Xi and eta are independent and as a follow follow up from here It was very easy to prove the following that expectation of their product is equal to product of their expectation Now For those who have problems with all this I do recommend it to go back to the lectures about conditional probabilities and mathematical expectation in the course and Unisor and everything is explained in all the details, so I'll just use it as given and What it means actually in this particular case if you remember the covariance is defined as a difference between these two values So if covariance is equal to zero then this is an equality Which is given for our random variables, which I am going to write on the top Instead of this as given Okay, so we have that expectation of C times 8 is equal expectation of Xi times Expectation of eta that's given and from this particular equality. I have to derive Independence between Xi and eta so if they are independent then this is true but if this is true is Is independence the only case well apparently if my Random variable take only two values and they are different ways if that's the case Then yes, the answer is that from this follows the independence and that's what I'm going to prove And that's what I'm asking actually to prove first you by yourself All right, so let me just go into this What is Expectation of Xi times eta well Xi times eta can take values AC with a probability P or a D with a probability Q or BC with a probability U or BD with a probability V right so we know all the possible values for this product and We know all the probabilities, which means on the left. I will have a C P plus a D Q plus B C U plus B D V that's on the left Now and I know that this is equal to this product All right, what's the probability of C well C can take values a and B So what is the probability to take the value a? well the value a Can actually be broken down into two variations value a and a It is equal to see and then all this probability Value a with the value of a to D and this is this probability So a would Would occur with the probability of P plus Q because there are no other variations. So the probability of a is P plus Q or Random variable see you can take value B And the probability will be obviously you press U plus V That's my expectation of C What's the expectation of a to it can take value C or D? C will take in two cases with the probabilities P and U. So we have to add them up. So multiply by C times P plus Q and D plus D times Sorry, not P plus Q, but P plus you P plus you D is Q plus V So that's what we have as given basically and again from this I have to derive independence between C and 8 all right now here is what I Am going to do to basically speed up the whole process I'm not going to go through all the opening the parentheses Multiplication blah blah blah. I know the answer because I did it before and I want to save my time as a lecture however What I would like you to do whenever you will after listening to this lecture you will do it yourself and this I Do hope you will do it You will basically do all the calculations Just by yourself completely without any kind of shortcuts. So I will make a shortcut because I know the answer So what I did is the following You see if you will multiply this by this and then this by this and Then multiply this by this you will have something like let's say a P times C times P So P will be the square it will be four different terms like a P C and B in every term which we are adding together will be four Multipliers here. We have only three multipliers So the idea which came to my mind was why don't we multiply the left part by this Which is equal to 1 but it gives me extra peak P square or Q square or something like this so without actually Changing the equality if I multiply the left part by one It doesn't change anything But if I do that and again opened all the parentheses etc. Etc. I will have lots of things which are cancelling out And whatever is left can be grouped together and here is my answer. I Was actually quite delighted whenever it happened So from this If I multiply the left part by one which doesn't change it, which means still equality will be open all parentheses Cancel out whatever cancelable and then group together things. I will get this and This is quite remarkable actually. I mean I really loved it when I saw it was very interesting Well, first of all Obviously can be equal to 0 if a is equal to B and C is or C is equal to D But we have agreed in the very beginning that our random variables are not that trivial So A and B are different ways C and D are different where it's otherwise They're not really random variables, but constants. So from this I can only derive that P V minus Q U equals to zero right that's the only way and Now what I'm saying right now is that from this Follows independence and that's what I'm going to prove and this is another very interesting piece Well, actually I will do instead of this. I will do equality, right? That's the same thing. So basically from this We have derived that this is given well, not exactly given but Straight-forward derivation. All right So we can consider this as a true statement, but I would like to prove independence now Let's think about what is independence? Well, as I told you before Okay, you know what let me just write it on the top So I have room for everything else So I have P V equals Q U By the way for those who remember matrices and the determinant P V minus U Q is the determinant of this matrix But that's another story All right, so this is something which we have basically derived from whatever is given now Let's think about what is independence. All right now as I said before Independence means that the probability of C taking some value X i and Eight equal one of its values is equal to the product of their probabilities That's what it means to be independent for any Value X i from whatever C can take and from any value y j Whatever Eta can take so for any pair Now in our case Well, let's just go into our case now X i well X i is either A or B right so let's just make an example so probability of C is equal to a and Eight is equal to let's say C for instance should be equal to the probability of C is equal to a Times probability of a to taking the value of C right now. What is this? well probability of a C for taking a and a to taking C is P so this is P probability of C is Taking a is basically P plus Q and Probability of a to taking C it's P plus you so I have to prove this For a and C and then actually similar Results for all others all right Now what is this? So I have to prove that P is equal P square plus P q plus P u plus Q u right I have to prove that However, since I know this I can change this to this now I can Cancel by P and I will have one is equal P plus Q plus u plus V which is something which is already Proven thing so how can I prove this well? I start with this Multiply it by P. It's a reversible right I start with something which is true So this is true and this is true So I start with a true statement Multiply by P give this Change this using this to q u and the result would be my Equation, whatever I have to prove Which means this is true. This is true and this is true That's for a and c but similarly Instead of multiplying this by P I can multiply it by q or by u or by V and it will be exactly the same similar thing So it will always be This type of proof for any pair in this case It's a pair a c but it can be pair a d or bc etc. Any other pair So we see that this is Provable based on this Which means again if this is true Then our random variables are independent Let me just repeat what we have just proven that in a non-trivial case if you have Only random variables which take two values only two values no more than that then Covariance is equal to zero Actually implies which means from this follows Independence between these two and my attempt to find dependent random variables Which take only two different values Always failed and that's the explanation why it failed now. This is a probabilistic problem and and obviously The fact that I have proven this particular Theorem indicates that I cannot find any practical numbers a b c and g and whatever numbers are here Which would result in? Covariance equal to zero. So if I want to some Some dependent variables and I will design whatever the dependency between them is I will not have covariance Zero or if I want covariance zero then Inevitably my either a and b should be equal or c and d should be equal or the dependency would not exist Okay, that's it for today. I recommend you again to go through all the calculations and come up with this Which is my kind of result of lots of calculations Go and do it yourself and come up with this just by yourself very very strong recommendation That's it for today. Thank you very much and good luck