 Welcome back. We were discussing properties of integrals, abstract integrals. So, all the properties of abstract integrals are proved in exactly the same way. You prove it for simple functions first, then you prove the property for non negative measurable functions and then you generate for any measurable function. So, that is the way all results are proved, all properties of integrals are proved. So, we were in the middle of discussing of number of properties. So, I think we were discussing property number 6, which said that if g is non negative and integral g d mu equals 0 equal to 0, then g equals 0 mu almost everywhere. This is the property we were discussing. So, the proof of this is as follows. Assume the contrary, let b be the set where g of omega is strictly bigger than 0 and assume mu b is. So, you want to prove that mu b is 0. To get a contradiction, you assume that mu b is not 0, that it strictly positive. Now, you want to show that this leads to a contradiction. So, then we defined, let b n be defined as the set of all omegas for which g of omega is greater than 1 over n. This is what we defined. Now, we also know that b n is increasing or decreasing. B n is contained in, is that correct? So, we have union i equals 1 through infinity b n is equal to b. We are saying that this set is equal to that set. In order to show this, you have to show that this is contained in this and this is contained in this. So, one way containment, the contain in that direction is very easy. Because, if you are contained in any of the b n's, it is immediately clear that g of omega is greater than 1 by n. Therefore, g of omega is bigger than 0. But on the other hand, if you want to show the containment that way that b is contained in this. For you take any omega for which g of omega is bigger than 0, then there exists a k for which g of omega is bigger than 1 by k. So, there exists a k for which omega is in b k. Therefore, it belongs to the union. So, these two sets are equal. Then we used, then what we said is we can use continuity of measures. So, we said mu b is equal to limit n tending to infinity, mu of. So, this is mu of the measure of that is equal to, I must actually write b i. So, the measure of that is equal to limit n tending to infinity mu of b n. This is the continuity theorem for measures for nested increasing sets, mu b n. So, this is an important relation. So, we have assumed that mu b is strictly positive. So, we have written a strictly positive mu b in terms of a limit of a sequence mu b n, which means that for all n greater than some n naught, mu b n should remain positive. So, you are approaching a positive number. This limit is a positive number you assumed. So, this guy, the sequence must be positive for all n greater than some n naught. So, actually all this we need is there exist thus there exist k such that mu b k is strictly positive. This has to be the case thus integral g d mu will be at least as big as see integral g d mu is the supremum of simple functions, the integral of supremum of the integral of simple functions. One such function would be to say that see the function 1 over k over b k will have this property. It is definitely smaller than g by definition. So, you will clearly have that g integral g d mu will be bigger than or equal to 1 over k times mu b k. Got that because this is a supremum of all simple, this is a supremum of the integral of simple functions. An example of a simple function which is less than or equal to g is the function which takes the value 1 by k over the set mu b k or the set b k rather correct with me. Because over the set b k g of omega is bigger than 1 by k correct. So, the function 1 by k the function which is constant at 1 by k over the set b k is a lower bound. This is a simple function which is a lower bound on g. So, this must hold. Now, what you know about that guy? This is bigger than 0. So, you take that k. So, this is a contradiction. Why is it a contradiction? You violate this. So, this assumption must be wrong. So, mu b equal to 0. Are there any questions in this proof? So, in particular for non negative random variable, if expectation of x is 0 then x is equal to 0 almost surely. Property number 7 is a very fundamental property of integrals called linearity. Integral g plus h d mu is equal to integral g d mu plus integral h d mu. So, this is the integral form of the statement. If you want to state it for the random variable, what will look like expectation of x plus y? Expectation of x plus y is equal to expectation of x plus expectation of y. This is always true. This is a very fundamental property. The integrals are linear operators on functions. So, linearity includes this, but the next one I am going to say is also is a scaling. So, you integrate integral a g d mu will be a integral g d mu. So, put together it is linearity. So, the proving this. So, again this is true for any measurable functions g and h as long as all the integrals are well defined. But, as usual the way you prove it is you first prove it for simple functions. Can you prove this for simple functions if g and h are simple? Can you prove this? The g you can write it as some of weighted indicators h also you write it as a weighted some of indicators. And you can from the definition itself you can verify that this is true. For simple functions this is very easy. Now, going beyond simple functions the next step will be to look for prove this for non negative functions which are not necessarily simple. So, that is very difficult to do if you take the supremum definition I gave. So, one of the things about say there are couple of ways of defining the integral. The way we defined it 6 of the 8 properties turn out to be very easy or 7 of the 8 properties turn out to be very easy, but linearity turns out to be very difficult to prove using the supremum definition. What we will do therefore, is we will prove monotone convergence theorem which is the very important theorem in integration theory. And then get back to proving linearity all right. So, because the definition we gave is not is conducive for proving many other properties, but it is not very conducive for proving linearity. So, we have to take a slightly difficult route through this fundamental theorem called monotone convergence theorem which we will do later actually we will do very soon may be next lecture. So, I will only say that this is easy to prove for simple functions non negative functions and more general functions we will get back later. Let me not worry about proving this and similarly for random variables this is always true as long as all the expectations are well defined. Again the proof will be along the same lines you prove it for simple random variables and then use monotone convergence theorem to prove linearity. So, this is. So, expectations are linear no matter what is what else is true x and y need not have any further structure to them x and y may be have any joint CDF what so ever they may have different distribution different marginals they may have any joint distribution what so ever they may not be independent or any such thing right. It is always true that expectation of x plus y is equal to expectation of x plus expectation of y. And similarly this generalizes to any finite sum right. If you have a finite sum of g 1 g 2 g n you can always write it as the sum of those n integrals. Similarly, if you have n random variables expectation of x 1 plus x 2 plus dot dot dot x n is equal to sum of the individual expectations prove for finite I am not saying about if this is finite if the number of terms you are summing is finite this is always true that is very easy. Because, if you have x plus y plus z you can write this as x plus y plus z and you invoke the same thing right. Any questions on this get to the proof later in scaling. So, this is scaling property of integration. So, you can this just says that you can bring the constant out of integration this is something you have been doing all along and it is true for abstract integrals as well right. And similarly for random variables. So, expectation of a is greater than or equal to 0 expectation of a x will be equal to a times expectation of x. How do you prove that? There is only one way to prove things right. So, you start off with simple functions again you take that g is equal to a weighted sum of indicators and write out. So, a g will also be a simple function whenever g is simple a g will also be simple that is because I assume a is bigger than or equal to 0 right. So, and then you write that integral out and then you verify that you write that integral out and verify that they are equal right. It is a very trivial exercise in. So, for simple functions it is easy. Now, for non negative functions. So, if so for non negative functions you have to apply the supremum definition. So, the way it works is this. So, a equal to 0 the result is obvious they are both 0 right. So, let us assume a bigger than 0. So, prove assume a bigger than 0 clearly if q is in S of S of g then a q is in S of a g correct. Actually it is not just clearly. So, it is actually if and only if a matter of fact correct. What does that mean? So, any simple function q which is less than or equal to g right. If this is the case then a q is a q is a simple function less than or equal to a g and vice versa right. So, that is very clear right. So, now you apply the supremum definition. So, you have integral a g d mu right. So, I have to be a little bit careful here. So, I have to look over the supremum of all q let us say q prime in S of a g is not it of integral q well q prime d mu is that right. So, that is by definition see, but this q prime is a q for some q in S g correct. So, I can write this as supremum over q in S g integral a q a q d mu correct. Maybe I am skipping one step you can write it as a q belongs to S of a g right and then write the same thing right. So, this is ok with me. So, now what happens? So, this is a simple function now right for simple functions I can bring the constants out because you can explicitly prove that to be the case right. I can bring the constant out here right can I bring it out here. I eventually have to bring it out here right. So, bringing it out here is not a problem, but since a is non negative I can bring it out here as well correct. So, that is equal to supremum. So, supremum q belongs to S g a integral q d mu this is because q is simple right. Now, this a is non negative. So, the supremum of that set of numbers will be same as a times the supremum of that right. So, this is equal to a times the supremum of. So, this is because a is non negative q belongs to S g integral q d mu and this is equal to a times integral g d mu correct by definition. So, I hope this is correct right I mean look over this argument very carefully it looks like everything is obvious, but each step requires a little bit of thinking right you have to be careful in writing this term. So, this comes out because of q is a simple function and the a eventually gets out of here because a is positive all right great. So, I think I am done with properties. So, I am done with the basic. So, the elementary properties of integration I am actually done now. So, the properties that remain are the more fundamental theorems of integration right. So, we will do actually there is just one fundamental theorem and that is the monotone convergence theorem. The monotone convergence theorem which is regarded as the cornerstone of integration theory and if there is one theorem in integration theory which is I mean which is the single most important one it is clearly the monotone convergence theorem. We will also do dominated convergence theorem which follows as a corollary to monotone convergence theorem and there is also a result a very well known result called Fatou's lemma that is a result we I will mention, but not prove it is not included in your syllabus. But, monotone convergence theorem and dominated convergence theorem are very useful in practice and they are very important theorems. So, I will those we will do in proper detail. Before we get into that. So, I think there is one more little unfinished business from the past if you remember I mentioned something called the inclusion exclusion principle for probability measures. Remember what that says it is a probability of a union b probability of a plus probability of b minus probability of a union b and similarly for n such events. If you have union over n events you have to pull out the intersection add back intersection and so on right remember that formula. So, I what is said back then when I was dealing with properties of probability measures was that you can prove it using induction right. It is true you can prove it using induction, but now that we have done expectations integration with there is a very straight forward proof of getting the inclusion exclusion principle. So, those of you who actually bothered to write down the induction proof will probably know how messy it was it is doable there is no intellectual challenge in it, but it was a it is you have to sit down and do it carefully right. So, very messy exercise, but we will see that using indicator random variables you can prove the inclusion exclusion principle very easily let me just indicate that right now. So, inclusion exclusion principle. So, inclusion exclusion formula if you like. So, recall. So, this is from your first chapter we said that probability of A i is equal to sum over i equals 1 through n probability of A i minus sum over i less than j probability of section A j dot dot dot minus. So, plus plus minus 1 to the n minus 1 probability of intersection. So, this is the inclusion exclusion rule right we just recall this from memory right. So, you have to add all the probabilities subtract the probabilities of intersection 2 at a time add 3 at a time subtract 4 at a time and. So, on right. So, in order to prove this. So, I am going to prove this and I am going to finish it here this is all the space we will take. So, you will agree with me that indicator of this set. So, I am looking at the indicator of that event. So, this is the indication this is the indicator of that at least 1 of the A i is occurs 1 of the A 1 through A n occurs. And that is equal to 1 minus the indicator that all the A i is fail to occur correct. So, this is the indicator that all the A i is fail to occur 1 minus the indicator of that is the indicator of at least 1 A i occurring right this you will agree right. Now, what is the indicator of. So, what is the indicator of A intersection B indicator of A times indicator of B correct indicator of A intersection B is indicator of A times indicator of B. So, this can be written as 1 minus the indicator of well actually let me write this as product indicator A i compliment I equals 1 through n right. Now, indicator of A compliment is 1 minus indicator of A i right. So, I am going to write this as 1 minus product 1 minus indicator A i agreed with me so far. Now, one more step is all that is needed. So, you have to go from indicator to probability how do you do that. So, how do you go from indicator of event to the probability of event take expectations right. Because, indicator of I of A is equal to probability of A correct. So, you take indicator of this what happens you get you get this correct and then you have to take expectation here right. So, what will happen here. So, if you expand this out. So, the 1 will hit there are bunch of 1 minus terms here right. So, the 1 will hit with the first 1 here. So, if you expand this product out what you will get is you will get a sum of the indicators right minus you will get indicator A i indicator A j right all the all the i j's you will essentially get terms like this you take you take expectations you will get this. So, you expand and take expectations that is it you are done expand out and take expectations right and you will be done. So, in order to write it may be I will take a little more space, but I am sure you can do that right. So, you just expand this product out this is like 1 minus this indicators this will be 1 minus I A 1 minus I A 2. So, 1 and then there will be 2 terms 2 at a time 3 at a time and so on. Finally, there will be a product of minus 1 to the n minus 1 indicator of all the product of the indicators of all the A i's which will finally, give you that term finally, if you take expectation you get that term. So, this is a very simple proof of inclusion exclusion principle actually this is a very this is the most transparent way to see why this inclusion exclusion principle is even true right. The induction proof while it is a mathematically correct proof it gives you does not give you much more intuition about what is going on right this gives you a very structural intuition on why this inclusion exclusion principle is true are there any questions if there are no more questions we will start discussing monotone convergence theorem. Maybe I will not I will not finish discussing today, but we will start of discussing all it is about. So, this monotone convergence theorem as I said is probably the most important theorem in all of integration theory. So, the monotone convergence theorem qualitatively what does it do it just gives you a way gives you conditions a condition rather under which you can interchange limit and integral. So, normally if you have limit n tending to infinity integral f n you have a sequence of functions f n let us say you have integral f n d mu where I can think of it as integral f n d x if you like if you just want to think of integrating on the real line you know you probably know that you cannot always integrate limit interchange in limit and integration right. So, the integral limit of an integral is not always equal to the integral of a limit we will see some examples, but the monotone convergence theorem says gives one sufficient condition under which you can interchange limit and integration namely you can say that the limit of the integral is same as the integral of the limit function. So, in particular if f n converges to some function f integral of f n need not always be equal to integral of f in particular the monotone convergence theorem says if f n converges to f in a monotone way right suppose f n monotonically increases and converges to f or f n monotonically decreases and converges to some f then you can interchange limit and integration. So, in that case you will in fact have that integral limit of the integral is equal to the integral of the limit. So, that is what the monotone convergence theorem says it is actually a to state it is fairly simple, but before I state this properly I want to define some concepts convergence of functions. Let f n from omega to r this could actually be r union plus or minus infinity as well be a sequence of measurable functions definition we say f n converges to f f is another function from omega to r the real line we say f n converges to f point wise if for all omega in omega f n of omega converges to f of omega. So, point wise convergence of function is a this is what this is it means right you fix an omega in the sample space in your set omega and for every omega f n of omega will be a sequence of what fix an omega f n of omega is a sequence of real numbers well not including plus minus infinity not quite real numbers. So, this is a sequence let us just say it is a sequence of real numbers if it is true that that sequence of real numbers converges to this real number f of omega. And if this holds for every omega in omega then we say f n converges to f point wise this clear. So, this f n. So, this could be a sample space this could be random variables for example, f n could be x n some random variables and in that case you say that random variables x n converges point wise to x if for every omega in the sample space x n of omega converges to x of omega for example. Now, it remains to be well the one little thing that I have not really spoken about is that if you have a sequence of functions converging point wise the limit function is also a measurable function whenever this f n is a measurable the limit function is always a measurable function. This is something that can be proven I have not done it in class, but it is always the case that if you have a sequence of measurable functions converging point wise then the limit is always a measurable function this can be proven. So, the definition clear and then I will just put one more definition which is closely aligned let us say that. So, omega let me say that omega f mu is my measure space omega is this guy and I have f n mu on that space as well we say that we say f n converges to f mu almost everywhere if what you suspect it is what you think this will mean this convergence holds not necessarily for every omega in the sample space, but instead it may fail to hold on a set of mu measure 0 if f n of omega converges to f of omega for all omega except perhaps on a set of mu measure 0 this is also definition. So, this is slightly weaker definition than this this is saying that for every omega in omega you need this convergence f n of omega converging to f of omega here you relax saying or may be it is not always true that. So, it is not true that for every omega it holds there may be a few omegas or a countable set of omegas or some mu measure 0 set of omegas some small set some there were 0 measure set where the measure is mu right on that set you do not have convergence or you may not have convergence right you do you cannot quite say what f n of omega is doing on a 0 measure set, but outside the 0 measure set you have convergence in other words there is a set a subset a of omega which has full mu measure where convergence happens. So, in the context of probabilities this has a very important interpretation which will come to a little bit later again. So, this will become almost sure convergence of random variables had the have you heard that term before almost sure convergence of random variables. So, there we have a sequence of random variables x n you have x n of omega converging to x of omega not necessarily for all omega in the sample space, but on a set of probability 1 right there may be 0 probability omegas where the convergence does not happen. So, this is a this is almost sure convergence when you are talking about convergence of random variables I will write that down explicitly a little bit later when we study convergence of random variables. So, far definitions everybody find see now. So, here so this is point wise convergence right now the issue is if you have a measurable function converging point wise to f of omega f of omega is always measurable. Now, the question is if you integrate f n d mu right if you integrate f n d mu you get some sequence of numbers right integral f 1 d mu will be some number integral f 2 d mu will be some number. So, integrating f n d mu gives you a sequence of real numbers right. Now, the question is does that sequence converges at all right that is question number 1 if it does converge does it converge to the integral of f d mu that is the question we are trying to address. The answer is no generally that is not true even if you have convergence of f for every omega in omega it is not true that integral f 1 d mu is equal to integral f d mu not true. So, the question we are trying to answer is this can we interchange limit and integration. So, that is the colloquial question right i e if let us say f n of omega converges to f of omega point wise is it true that integral f n d mu converges to integral f d mu answer is no. So, what I am saying is so rather say another way of writing is if you want to write it more explicitly is it true that the limit of the integral is equal to integral of the limit function if this is a little more explicit you can take this it is not true that this is not true to see an example I will just conclude with an example. Let us say you take let my omega f p be my favorite 0 1 integral 0 1 interval endowed with borel and Lebesgue measure that is my probability space I want to define f n of omega is equal to n for 0 less than or equal to 0 less than or equal to omega less than or equal to 1 over n or less than 1 over n and 0 otherwise. So, that is my sequence of functions. So, f n of omega will look like that. So, if I plot it so everything is in all the action is in 0 1 interval this is 0 1 interval. So, f 1 will be 1 in 0 to 1 f 2 will be 2 in 0 to half f n will be n. So, this will be 1 over n solid dot here 0 everywhere here hollow dot here solid dot here actually does not matter what the here it is a hollow dot here it is a solid dot the functional value is n here you will see that. So, if you want to so integral what is integral f n d lambda. So, I am going to integrate f n with Lebesgue measure integral f n d lambda will be equal to 1 for all n this is equal to 1 on the other hand if you look at the limit function. So, the limit function f n will be so for all omega lying. So, for all omega that strictly bigger than 0 the limit function will be 0 correct at 0 the function will be plus infinity actually because I put a less than or equal to here actually if I put a. So, if I put a strictly less than here the limit function will in fact be 0 everywhere right, but yeah I mean. So, if you like so to avoid confusion let us just take this. So, let me just put the function at 0 here just avoiding confusion. So, let me do that and at 0 it is 0 the function is 0. So, the limit function is equal to 0 for all omega and omega correct because you give me no matter which omega you give me there will be some n beyond which the function is 0 right. So, the limit function is identically 0 correct. So, if I look at f defined as limit f n this is 0 for all omega and integral f d mu will be equal to what 0 right and this is not equal to 1 right. So, the so integral f n d lambda is equal to 1. So, this implies limit n tending to infinity integral f n d lambda is also equal to 1 right. So, this integral is always 1. So, the limit is 1 the limit of the integral is 1, but the integral of the limit is 0. So, it is clearly not the case that you can blindly interchange limit and integration very simple example says that you cannot do it right. So, I made this strictly lesser. So, even if this was strictly less than or equal to the limit function will be 0 for all positive omega and it will take the value plus infinity at only at 0 and because it is a function that is 0 every almost everywhere integral of a function which is 0 almost everywhere is 0. So, it would not have mattered even if I had less than or equal to here there will just be 1.12 the function is plus infinity, but that 1 point has measure 0. So, there is no difference that is the clear. So, you cannot interchange limit and integration. So, what you will see next is next class we I will state the monotone convergence theorem which says that if this limit where to be a monotone limit if f n converges to f monotone equally then it is always the case that you can interchange limit and integration. So, in this case monotone city does not hold right because f n will look like this and f n plus 1 will look like that right. So, it is not true that they are monotone in n right. So, we will see this next class.