 Hi, I'm Zor. Welcome to Unisor Education. We are discussing random variables. Now this lecture is about the expectation and actually sum of two different random variables. Expectation of sum basically should be equal to the sum of expectations. That's the theorem, which I'm going to prove. I recommend you to go to Unisor.com and watch this lecture from this website as well as read the notes to this lecture, which are right on the side from the video itself. It's very important actually to read the notes because it presents you a certain different, maybe perspective to the same educational material. Alright, so sum of two random variables, that's what we're talking about today. First of all, we all remember what the sample space is. I use the letter omega about the sample space is basically a combination of elementary events. In this case, it's M elementary events. These are possible results of the random experiment. Now, each of these are mutually independent, of course, and there are certain probabilities associated with each one of them. It's called the probability measure, if you wish. So each element, probability of each elementary event with index i is equal to the number pi. Now all pi is non-negative and sum of them equals to 1 and they basically represent the frequency of occurrence of this particular element. If you are conducting the same experiment many many many times and the more you the more times you conduct the experiment, the closer the frequency of occurrence of certain event will be to this particular value. So that's kind of a statistical approach to probabilities. Now, let's assume we have a random variable, which is basically a function defined on each elementary event. Well, it can be, for instance, a winning amount if you are playing poker and these are all different results of the poker game or it can be a temperature of the person randomly chosen from the crowd. I mean, it can be anything. So there is certain random experiment which results in some elementary event and with this elementary event is associated with certain numerical function. As I was saying, it can be amount of winning or temperature or some reading of some gadget, etc. Now we are interested in the concept which I was explaining before as expectation. So expectation of the random variable which takes values x1, etc. xm on the corresponding elementary events depends, obviously, on probabilities of these events and the values the random variable takes for each event. So the expectation of random variable xc in this case is equal to probability of the first element times its value as a random variable plus the second etc. up to the very end. So that's an expectation or expected value or mathematical expectation of a random variable xc and I was actually putting these values. Basically, it means that the value of the random variable if random, if elementary events eis is happening is xis. That's what basically this means. So we have a certain number of elementary events. We have certain probabilities they occur and we have a certain random variable the value of which depends on the elementary event and this is the expectation of this random variable. Now, so that's number one. That's my first. Now, let's assume that I have a second sample space which contains completely different elementary events. Doesn't really matter. With its own measure of probabilities which is q. q of f j equals to qj. So all qj's are obviously the same as pj, spi, whatever. They are non-negative and their sum is equal to 1. These are probabilities. q1, q2, q3 etc. up to qn. And let's assume we have a different random variable eta, which is defined as y1 etc. yn. So eta of fj equals yj. So that's a completely different sample space with its own probabilities of elementary events and its own random variable eta defined on this sample space. So we have two different random variables and this random variable also has its own expectation, which is equal to q1 y1 plus q2 y2 plus etc. plus qn yn. So there are m-elementary events in the q in the omega-1 and n-elementary events in the omega-2. Now, so these are correspondingly expectations of two different variables, random variables, defined on two different sample spaces. Now, what I would like actually to prove is the following theorem, that expectation of sum of these two variables is equal to sum of their expectations. That's what I have to prove. Now, first of all, if you consider an expectation or expected value of the random variable as the value where its the values of random variables are concentrated and examples of this, for instance, an example is you measure the temperature of a person randomly picked from a group. Well, there is something which we call normal temperature and considering people in the group are all healthy, their temperature will be basically close to certain normal value, not exactly equal to, but more or less concentrated around this normal value. Now, if you take a completely different random variable, it can be either a different group of people which you measure temperature or it can be some kind of measurement of the voltage in the electric outlet, which is supposed to be, let's say in America, it's about 110 volt and in Europe it's 220. Basically, it's a normal value. It doesn't mean it's always the same, but all the values which you can measure will be concentrated around this normal value. So if you have one random variable with values concentrated around one variable, one particular value, and then another random variable with its values concentrated around another value, then obviously if you combine them, if you add them together, well, the sum of two random variables must actually be concentrated around some of these concentration numbers, right? It looks kind of natural and it sounds really plausible. Now, this is the intuitive approach to this, but I would like to prove it mathematically. To prove it mathematically, first of all, I have to understand what's the sum of two random variables is. Well, let's assume our example. One is a temperature and other is voltage. Can I add temperature to a voltage? It looks like it doesn't make sense. Well, actually, well, it doesn't make sense in physics, but this is a number and that is a number and there is nothing wrong in mathematics with adding two numbers. So in theory, we kind of understand what it is, but let's try to define it more properly. Okay, so these are not temperature and voltage. These are number and number. So in theory, we can combine them together. We can add them together, but does it make any sense? Does it make any kind of sense? I mean, we are adding one random variable, which is defined in one particular sample space to another, which is defined in another. It's like two different functions we are adding together, but their domains are completely different. Can we do this? Well, that's not an easy thing to do. And here is what I suggest you to basically understand as the explanation of what is actually C plus eta. Well, so we have this particular sample space and we have that particular sample space. Now, let's consider a sample space, a new one, which I will write as this one. It's basically something which is known as Cartesian product. So if this space contains E1, E2, Em, and this one contains F1, F2, Fm, then this combined is a set of pairs. E1, F1, E1, F2, etc. E1, Fm, E2, F1, etc. So you understand all the different pairs of this and this are considered to be elements, elementary events of the new sample space. So if, for instance, the result of the experiment of checking the temperature is such and such and result of checking the voltage is such and such, then I'm combining two experiments together. I'm measuring simultaneously a temperature of a random person and voltage of a random outlet and combine them into a pair. So a pair is actually the result of my experiment, a pair of two numbers. So a pair of two numbers, where this pair is the combination of any from this and any from that is the result of my combined experiment. Now, what's the probability? Well, I don't know the probabilities, but let's just assume that there is some measure of probability defined on this particular sample space R, which is defined, supposed to be defined on each pair, right? E, I, F, J and I have some number, R, I, J. I like to write the letter R this way, okay? So what are these R, I, J? These are probabilities of occurrence simultaneously the element E, I and the element F, J. So if we simultaneously combine these two experiments into one experiment and measure, then the result of one experiment is E, E, E, E, I's. Experiment number two will give me F, J's and the combination of these has the probability R, I, J. Nothing wrong with that. Now, my new random variable is that how it's written? That I'm defining basically a new random variable defined on this particular space in a new brand new space, which is a Cartesian product of two previous spaces. I'm defining this in a very simple manner. Zeta of E1, EI, F, J equals to Zeta of EI plus eta of F, J, which is equal to XI plus Y, J. So that's the definition. This is a new random variable defined on a new sample space with new probabilities, which I don't know actually, and the values which are equal to a sum. So whenever we have as a result of my two random experiments, one this and one this, I have a result EI, F, J, then the value of my random variable on this result of my combined experiment is X for I plus Y, J, where XI's are the values of random variable C defined on this one and I, J's are the results of the random variable eta on this particular space. So this brings me to basically a relatively rigorous definition of what is a sum of two random variables. So yes, we cannot just simply add two functions defined on two different domains, just as an addition. But using certain logic and using the Cartesian product, etc., we can. In this case, that's exactly how it is done. All right. Now I define basically this and now I can base, well, now I understand what I'm talking about, right? What is an expectation? What expectation is as any expectation of any random variable? The expectation of this particular variable equals two. Well, let's just use zeta. Well, that's sum of probabilities times values, right? So this is R11 times X1 plus Y1 plus R12 times X1 plus Y2 plus, etc., plus R1 N times X1 plus YN plus R21 X2 plus Y1 plus R22 X2 plus Y2 Now this would be R2N X2 plus YN plus, etc. And the last one would be RM1 XM plus Y1 plus RM2 XM plus Y2, etc. MN XM plus YN. So as we see, we have M times N, different elements in this sum, because M times N is the number of the elements in the combined sample space, because it's all of these with all of those M times N. So each one of them has the corresponding probability, which is R, let's say ij, and the value is x ixj, and I have to summarize it with i index i changing from 1 to M and index j changing from 1 to M. So that's basically what the expectation of this sum is about. But it looks complex, but it's really kind of a structural, it's very simple. You just use the indices. Much easier would be if I can use the sigma sign in mathematics. I just don't want to use it in this particular case. All right, so let me point one very interesting property of these probabilities, R. If I will combine R i1 plus R i2 plus etc. plus R iM, what does it mean? It's a combination of certain elementary events, which are first experiment results in E1, PEI, and the second experiment results in F1. Now this is the first experiment also in results in EI and the second in F2, etc., etc., and the last one is the first experiment results in EI and the second experiment in FN. So what does it mean when we add these probabilities together? That's the probability of the first experiment resulting in EI and the second experiment either in F1 or F2, etc., or FN, which means everything. So we don't really care, it can be anything. So basically it's equal to the probability of EI, which is PEI. If you remember that the probability is a feature elementary events. These are P1, PN, and this is Q1, QN. Now similarly, if I will fix the second index, R1J plus R2J plus, etc., plus RMJ. Now what is this? It's the probability of the first experiment ending in either E1 or E2, etc., or EM and the second one being FJ. So which means we don't really depend on the result of the first experiment and it's equal to probability, measure of probability. I used letter Q in that case of FJ, which is equal to QJ. So these are very important properties of our combined probabilities. So if you're combining the probability of two events, but you don't really depend on the second event, so the sum of the probabilities of the first event going to a particular result and the second experiment going to any event out of anything, that's basically the result is just the probability of the first or in this case the second experiment, all right? Now let me just point out very important property. What if these two events are completely independent? So the probability of the first being EI and the second being FJ. Let me write it down. So for independent events, probability of EI, EJ, FJ, which is R, I, J is actually the combination of probability of EI and probability of FJ. If you remember for independent events, and we were talking about the probability of the combined event, if these two events are independent, is equal to the product of their probabilities. Now if you don't remember it, I can refer you to one of the previous lectures which we did. So in this case, this is equal to PI QJ, right? In which case this becomes obvious because what is this? It's PI times Q1, PI times Q2, etc. PI times Qn, you factor out PI and in the parentheses you will have Q1 plus Q2, etc. Plus Qn, which is some of these probabilities, which is one and that's why it's equal to PI. So it's obvious for independent events. Now, and this is similar obviously. Now for non-independent events, it's still true, but you have to really apply some logic to make sure that this is really true. Now why did I decide to use these properties? Very simple. We just have to regroup these differently. Look at this. If I will open these parentheses, it will be R11x1, R11y1, R12x1, R12y2, etc. And I will factor out x1. What will I have? I have x1 and in parentheses R11, R12, etc. R1n, which is P1. Now, similarly, the rest of the members in the first line, I don't really look at them right now. I will take care of them later. Now let's consider the second line. Second line is, again, I will x2, x2 and x2, I will factor out and in the parentheses I will have R21, R22, etc. R2n, and some of them is equal to P2, etc. Up to the very last line where I can take xm and factor it out. And in the parentheses I will have Rm1, Rm2, etc. Rmn, which is using the same property, Pm. But that's not it. I only use the axes. Now let's use ys. Now ys I will group vertically. So this is y1, R11, this is y1, R21, etc. and y1, Rm1, which is sum of R11, R21, etc. Rm1. Using this property, this would be Q1. Now similarly, y2 would be sum of R12, R22, etc. Rm2. So the second thing, this is fixed to 2, so it would be Q2. And as you understand, the last one would be similar, which is yn Pm. So that's what this particular sum is all about. And this is equal to, this is no other but expectation of C and this one is no other but expectation of F. And that's the end of the proof. And as you see, in case of independent events, I don't really need these properties because I can put P1 times Q1 here, P1 times Q2 here, etc. And then again, I just regroup and factor out things and that will be exactly the same result. So the whole theorem is really very, very simple, except you just have to write maybe a relatively long set of additions here. But the idea is simple. Intuitively, it's very much understood that the expectation of the sum of two variables is the sum of expectations. What I would like actually to point out is for you to understand the concept of the sum of two random variables defined on completely two different sample spaces, that you have to really consider the pairs of elementary events from this and from that, from this and from that. So it's like a Cartesian product. So that's very important. And another important factor is I'm not using the independence of these two things. I'm logically deriving these two equalities, which are kind of obvious if you consider them, and just use this property to prove this theorem. Well, that's it for today. I suggest you to really gain the notes for this lecture on theunisor.com. Notes basically contain the same logic, the same set of thoughts, but when you read it again, it's better understood actually. So thanks very much and good luck.