 Good morning. Today, we will begin our third chapter on integration and expectations. So, we have completed two chapters. The first one was on probability measures and probability spaces. The second one was on random variables. Now, this is that third chapter on integration and expectation. So, you are already familiar with integration and the integral you are familiar with is known as the Riemann integral. This is from your calculus course. So, when you say integral a to b f of x dx, you know what it means. So, you take some function from r to r and you know how to integrate it over an interval. So, everybody is familiar with this. So, in particular. So, you have some function f of x and you have some interval a to b. The way this integral a to b f x dx is defined is by dividing this x axis into several little particles little intervals. So, let us call this each of this. Let us call this delta x i. So, this is x naught is a x 1 x 2 and so on. This is x n minus 1 and then this is x n equal to b. So, you make a partition of the interval a b into this tiny little delta x i intervals. So, delta x i is simply x i minus x i minus 1. And then what you do? Just to give you a quick refresher on what Riemann integral is. So, you take the following. You take define. So, u n of f is equal to sum over i equals 1 to n supremum f of x over this interval times delta x i. But, delta x i is simply x i minus x i minus 1. So, you define that and then you define l n of f as the same thing with an infimum of the same thing. So, essentially what you are doing is if you consider a particular delta x i interval. You consider the supremum of all the values taken by the function in that little interval. It is the biggest value. So, in this case it is here. And then for each set delta x i interval, you consider this and some multiplied with delta x i and sum it over i equals 1 to n. Similarly, for the infimum, this sum is called the upper Riemann sum. This is called the lower Riemann sum. Now, what you can show is that when n becomes bigger and bigger and the partition becomes smaller and smaller, the upper Riemann sum is monotonically decreasing. Is that clear why it is the case? So, if you just if you want to draw a more magnified view of what is going on here, let me just draw a magnified view. Suppose, that is my delta x i. So, I have a particular n and a certain partition. So, in this case I will have the supremum is that is here, this value and infimum is let us say that value. So, in the supremum I will substitute that value for this particular interval. However, if I create a final partition with a larger n. So, let us say I will have a further partition of this. Now, what will happen is I have. So, for this interval, for this partition I have that as my supremum, but this interval I will have a lower value as my supremum. So, when I do the upper Riemann sum with the final partition, I get a value that is smaller at least no bigger. And similarly, for the lower Riemann sum as n becomes bigger and the partition becomes finer I will get a larger value. So, the lower Riemann sum can be shown to be a monotonically increasing sequence and the upper Riemann sum is monotonically decreasing. So, for this sum for these if you say n to infinity that is that each delta x i goes to 0. This sequence must necessarily attain a limit, because this is a monotonically decreasing sequence and this is a monotonically increasing sequence. For the moment let us assume that f is bounded is that clear. So, since u n of f is monotone decreasing and l n of f is monotone increasing in n limit n tending to infinity u n of f and limit n tending to infinity l n of f must both exist. Is that clear any questions on that. So, this is something you should already know I am just reviewing Riemann integration. So, this limit and this limit definitely exist, but it is also the case that for every n u n of f is bigger than or equal to l n of f for every n that is true why by definition. So, this limit exists and this limit exists as n tends to infinity. So, the limit of this must be greater than or equal to the limit of that correct. So, this limit is known as the upper Riemann integral of f over a to b and this is known as the lower Riemann integral of f this limit. This limit exists as real numbers and this is called the upper integral and this is called the lower integral of f and f the function f is said to be Riemann integrable if the upper Riemann integral and the lower Riemann integral have the same value. So, f is said to be Riemann integrable over a b if limit u n of f is equal to limit n tending to infinity l n of f. So, what we are saying is that l n of f is a increasing sequence in n and u n of f is a monotonically decreasing sequence in n. They must both have some limit the limit may not be the same, but if the limit is the same the limit and this limit are equal in other words if the upper integral and the lower integral are both equal. We say that f is Riemann integrable over the interval a b and this common value is known as the integral of f over a to b. This common value is called this is common value is denoted integral a to b f of x d x that is your Riemann integral. So, everybody familiar with this. So, one nice thing about this Riemann integral is that. So, if f is Riemann integrable it does not matter how you. So, how you create this partition is irrelevant you can create equal partitions unequal partitions any whichever way you like and as long as each delta x i goes to 0 you have the same value of the integral that can be shown any questions. And for Riemann integral if you have in something like integral a to infinity or integral 0 to infinity of x d x it is just like sending b to infinity. So, integral a to infinity f x d x is simply defined as limit b tending to infinity integral a to b f x d x that is an improper Riemann integral. So, these things you already know. So, now we know how to integrate a function over an integral with respect to a variable. So, now we are going to generalize this theory of integrals to what are known as abstract integrals. So, what we really want to do is to define something like this. So, this primer is over primer on integration is over suppose. So, this is called. So, we are getting into abstract integration here let omega f mu be any measure space not necessarily when a probability space be a measure space. And f from omega to R well actually I can have from omega to 0 infinity infinity included f from omega to infinity b measurable function. So, you have some omega f mu this is your omega and you have a sigma algebra on it and some measure on it this may not even be a probability measure some measure. And then you have a function that maps omega to real line or it can even some points can even map to plus or minus infinity that is allowed. So, f in certain points can even take plus or minus infinity as values we want to define. So, we want to define the following abstract integral. So, you want to define integral f d mu over a measurable set a. So, you have a measurable function from omega to R or omega to R union plus infinity minus infinity. You want to define the integral of a measurable function with respect to a measure over a measurable set f measurable set. So, that is what we will define I have not mean I have not defined it I am saying this is what I want to do. So, from now on you will not think of integrals as integral over an interval and integral with respect to a variable that is high school or under graduate stuff. So, as graduate students we will think of integral as integral of a measurable function with respect to a measure over a f measurable set. So, now on this is how we will think of it I have not told you what it is of course, but that is where I am building I am just saying what we want to do. So, this is called abstract integration now. So, why are we bothered doing this. So, the reason is this naturally generalizes the concept of a Riemann integral to include more general measures as well in particular. So, I will let me put down a couple of special cases I will put two special cases down which are very important. So, let us say that omega f mu is simply R Borel lambda. So, this guy is R B R lambda Lebesgue measure so in that case my integral will be integral f d lambda over a right. So, a is. So, a will be some what kind of a set a will be some Borel set right. So, integrating a measurable function with respect to the Lebesgue measure over a Borel set. So, here a is Borel so this integral is known as the Lebesgue integral. So, it is integral with respect to the Lebesgue measure therefore, the integral is known as the Lebesgue integral. So, this Lebesgue integral naturally generalizes the Riemann integral. So, this is like saying I am going to integrate f f is after all a function from R to R now right. So, you are integrating a function with respect to the Lebesgue measure over any Borel set. So, here you are doing the same thing except the set you are integrating over is it is a particular interval it is not some general Borel set right. So, what this does is generalizes the concept of a Riemann integral. So, in fact we will see that there are you can integrate. So, the Lebesgue integral is defined for certain functions over more complicated sets and it is defined even when the Riemann integral is not defined or when the Riemann integral may not exist even right. So, the Lebesgue integral exists sometimes even when the Riemann integral does not exist. But, however the nice thing is whenever the Riemann integral exists the Lebesgue integral always exists. Furthermore the value of the Riemann integral will be equal to the value of the Lebesgue integral. So, in that sense this is strictly more general notion of an integral of a function from R to R. I have not told you what it is of course, but I am just building up towards what it is right. So, this Lebesgue integral this generalizes. So, I will just say generalizes the Riemann integral in the sense that this may exist even when the Riemann integral may not exist. But, when the Riemann integral exists the Lebesgue integral always exists and the values will be equal right. So, in some sense we can from once I have told you what this is you can totally forget about Riemann integration because this is strictly more general. So, in the beginning this may seem like a little abstract because it is called abstract integration after all. But, once you learn Lebesgue integrals and abstract integrals in general you will actually feel that the Riemann integral is a very awkward integral. It is actually quite awkward and it does not exist for even fairly is simple functions. So, that is the first example of an abstract integral right it is the Lebesgue integral. The second the second example we consider the space omega f mu to be a probability space. So, mu will be a probability measure now. So, we will consider since we are doing probability theory. So, let us say this is a probability space mu is a probability measure now. So, I call it T and I am going to consider a measurable function from omega 2 let us say R consider a measurable function from omega 2 R. But, what is the measurable function from omega 2 R it is a random variable right now this is the probability space consider us. So, I will just say consider random variable right consider random variable x from omega 2 R this is a special case of a what we have written down there right fine. So, now the integral will look like what consider I am going to consider integral I am going to integrate over the whole space now. So, I this is you can integrate over any f measurable set you want right I am going to integrate over omega itself now omega is f measurable. So, I can integrate over omega this is a special case right what will the function be x d p right this p is my measure now by the way a matter of notation whenever I integrate over the whole space omega from now on I will not write omega here I will only write integral right I will not actually I will do that when I may when I write integral f d mu without saying what set I am integrating over it is to be understood that I am integrating over the whole space omega. So, let me just write integral x d p from now on it is the omega is implicit here. So, this integral is of great importance in probability theory. So, this integral is called the expectation expectation of the random variable x and it is denoted by E square bracket x. So, given a right. So, this integral will be some number right and this number is called the expectation of the random variable x. So, what is the expectation is just integral x d p, but I have not told you what integral x d p is right. So, once I finish telling you what integral f d mu is you would already know what a Lebesgue integral is you would already know what a expectation is with me. So, these are just special cases of the abstract integral integral f d mu any questions on this. So, now, we will actually go to build towards definitely defining this I have not told you what it is yet right. I only told you what special cases we have. So, I have. So, let us follow the following agenda for defining this abstract integral. So, the program will be as follows. So, the program is first you will define program. So, define integral f d mu. So, integral f d mu for simple functions what are known as simple functions. So, I have not told you what a simple function is either right. I will tell you that. So, this integral f d mu is it over the whole space then define f d mu for non negative f non negative measurable f then you define it for general for arbitrary measurable f f is always measurable arbitrary f then finally. So, finally, we will define integral over a f d mu once you define integral f d mu over the whole space we will finally define integral over a f d mu. This is the program we will follow first for simple functions for then for non negative f not necessarily simple then we will define it for arbitrary functions not necessarily non negative then finally, we will define integral over a f d mu. So, let us first. So, let us first define integral for simple functions it can be written as f of omega equal to sum over i equals 1 through n a i indicator a i of omega for or omega n omega where a i a non negative for i equals 1 2 dot n and a i belongs to f for i equals 1 2 dot n. So, that is the definition of a simple function. So, a simple function is first of all a non negative function which can be written as a finite sum a finite weighted sum of indicators ok. So, that is what a simple function is. So, in particular. So, for an example if you want to take let us say. So, let us say your omega f mu is simply. So, let us say my omega f is simply R Borel one example of a function which is simple will be let f is equal to indicator of the interval 0 1 plus some other constant let us say square root of 2 times indicator of rationals. This is an example of a simple function I just made something up. So, this is. So, you have a finite sum of some finite weighted sum of indicator functions this is an indicator of the 0 1 interval and this is an indication indicator function of the rationals right and I put some weight here. So, any non negative weight will do. So, this is an example of a simple function. So, if I give you some omega let us say I give you omega equal to half what will be f of omega b half lies in this interval. So, this indicator will fire is half rational it is. So, this will also fire. So, the value of this function will be 1 plus square root of 2 and if I give you omega equal to let us say 1 over square root of 2 then the value of the function will be 1 because this will not fire then right and if I give you omega equal to pi the function will be 0. So, all that is needed is that these a i's these sets here must be f measurable in this case boreal measurable right. So, I can even have crazy sets like canter sets here right. So, this is understood. So, this is an example of a simple function plotting this is a little difficult. So, indicator 0 1 we know how to plot it just takes the value 1 in the interval 0 1 and it takes values 1 at square root of 2 right this guy takes value square root of 2 at all rational points and you have to plot the sum of the 2 functions right. So, it is a little bit tricky to plot if this set were also an interval you can easily plot it right. So, if for example, if I want to plot some simple function. So, let us say. So, that is one example let g equal to some. So, let us say indicator of 0 1 plus 3 by 2 times the indicator of 1 3 for whatever reason this is the reason I am considering this is this is very easy to plot right. So, this will look like. So, if I plot g against omega by the way is simply from the real line now I will get. So, that is my 1 that is my 2 that is my 3 I will get. So, I will get something like that plus in the interval 1 to 3 I have the value 3 by 2 and 0 everywhere else right except at this point what will happen because both are closed I will have I will have the value here something 5 over 2 or something right does not matter. So, this is 1 that will be 3 by 2 see now you will realize that the representation of g in a certain form is not unique. So, this is my function this is the function I have plotted you could just as well write this as indicator of 0 to 3 plus half indicator of 1 to 3 or something like that is that the same thing right the same function as that right or you can write indicator of 0 to 1 plus 3 by 2 times indicator of 1 to 2 plus 3 by 2 times indicator of 2. So, many representations are possible right. So, they are all it is all the same function, but multiple representations in terms of simple functions is possible right multiple representations are possible. Now, just to fix ideas we will say that a particular representation is canonical if these little a i's are distinct and big a i's are disjoint. So, canonical representation so the multiple representations right. So, we are just going to say that I am going to consider canonical representations where little a i's are distinct and capital a i's are disjoint. So, in this for example, this function if I have to write it in the by the way the word canonical just means standard canonical just means standard representation. So, this for this function for example, you want small a i's are distinct and big a i's are disjoint. So, I will write it as indicator of so the canonical neither of this is canonical why this disjoint does not work right. So, you will write it as the indicator of 0 to 1 open at 1 then indicator of the single term 5 over 2 or something times the indicator of the single term plus open 1 close 3 indicator of this whole interval right. So, you can think about what it should be here. So, that is called the canonical representation. So, from now on I will assume that when I write a simple function f of omega equal to sum over a i of omega I will assume that it is already in the canonical representation. If it is not in the canonical representation you can put it into the canonical representation right. So, that is not a problem. Now, I am going to define integral f d mu for a simple function definition for this function is defined as follows it is defined as. So, integral f d mu is defined as sum over i equals 1 to n a i mu of a i. Now, why is mu of a i will defined a i's are f measurable. So, mu of a i is well defined and this sum a so mu of a i is some non negative bunch of numbers a i is already non negative. So, this is a finite summation. So, this is perfectly well defined right no problem right. So, integral f d mu is defined as that. So, this is definition. So, for example, in this case what will the what will that give you. So, if I consider not just. So, if I consider omega f mu to be r b r lambda. So, I am considering integration with respect to measure mu where mu is Lebesgue measure. So, integral of this g will be. So, it should be 1 times the measure of this which is 1 plus 3 by 2 times the measure of. So, 3 by 2 times 2 plus whatever value that is 5 by 2 times 0 right. So, you effectively see that you get the area under this. So, you will get. So, what I just spoke out is just the area under this function correct. So, what looks like some weird definition is for the case of the Lebesgue integral when mu is Lebesgue measure you get the area under the simple function right. So, if you integrate this function with respect to Lebesgue measure what will you get you will get what will you get. So, you will get 1 plus 0 right see 1 thing you should know what is that this integral does not really depend on whether you have a canonical representation or not that is something I have said that f has some canonical representation. But, it does not matter even if you do not have a canonical representation this the integrals value will be the same. For example, if you have some other say if you have. So, that was not canonical was this say this is certainly not canonical right. But, in this case also you will get you will first get 1 times 3 plus half times 2 which will be the same value you can check. So, it does not really matter whether I am writing things in canonical representation or not. It is just for I mean it is just for the sake of definiteness. So, this is not. So, this statement is totally unnecessary you can check that this definition is the same for even non canonical representations. So, you can just check that this is the same. So, example. So, examples 1. So, for that function over there integral f d lambda will be 1 times 1 plus square root of times 0 this is equal to 1. So, the integral of this function is 1. Similarly, for g integral g d mu g d lambda that guy is equal to I can write it as 1 times 1 plus 3 by 2 times 2 right 3 by 2 times 2 plus 5 by 2 times 0 that is also 0. So, this is equal to 4 is not it. So, that is getting that area right. So, this we have defined a simple integral. So, integral for a simple function which corresponds to at least when lambda is Lebesgue measure it corresponds to area under that simple curve simple function right. So, so far so good. So, of course, if the measure mu some other measure you will not get the area under the curve not at all right. So, far. So, let us now. So, do I have 5 minutes I have 2 minutes I think let me try and do one another nice example well actually I did that example kind of. So, I have let me consider the following function called Dirichlet's function. So, you have let say omega is 0 1 and I am going to consider Borel 0 1 and lambda. So, this is my space and define let f of omega is equal to indicator of Q intersection 0 1 Q intersection omega. So, this is equal to 1 with omega is rational and 0 otherwise. So, this is called Dirichlet's function. So, it is simply the indicator function of rational in 0 1. So, what I want to illustrate is. So, it is clearly integral f d lambda is equal to what f d lambda is equal to 0 right this is the Lebesgue integral. So, the Lebesgue integral is very trivially 0 right. Now, what the reason I am doing this example is even for such a simple function the Riemann integral does not exist you can show that the Riemann integral will be undefined does not exist. Do you see why that is the case. So, this Dirichlet's function is discontinuous everywhere right it has so many jumps. So, in any vicinity of a point. So, if you are at a real point which is not rational it will be 0, but it is very close vicinity there will be a rational. So, that the function will jump to 1 right. So, it has it is discontinuous everywhere. So, if you try doing the. So, if this is my 0 1 interval if I. So, no matter what partition I consider no matter how fine a partition I consider in that partition the supremum of that function will always be 1 correct and the infimum of the function will always be 0 right. So, my upper Riemann sum will be what will be 1 a upper Riemann sum is always 1 for any n lower Riemann sum is always 0. So, limits. So, the upper Riemann integral exists and it is equal to 1 lower Riemann integral exists it is equal to 0, but they are different the limits are not the same right. So, the function is not Riemann integral whereas, the Lebesgue integral is simply 0 right. So, you already see why the why I said the Riemann integral is actually quite an awkward integral right because it tries to look for too many details on where the function is in these little intervals right. Whereas, this only considers measures the Lebesgue integral only considers measures right it is defined in terms of the measure of certain sets right the Riemann integral is undefined. So, this function is not Riemann integral, but it is Lebesgue integral and the Lebesgue integral is 0 right, but of course, in nice cases like this the Lebesgue integral and Riemann integral will both exist and it will coincide right. So, I will stop here today.