 Welcome back. So we have now so far discussed about risk as an endemic feature of problems of decision making under uncertainty. The second feature that I wanted to talk to you about is the role of information. That information plays an extremely critical role in problems of decision making under uncertainty and the way and problems of and such problems differ in their characteristics, in differ in the kind of solutions you can get based on the way you based on what you have been told about the information about in the problem. So a simple illustration of this is the following example. So consider a system which has a scalar state, a system with a scalar state. Now this system evolves in the following linear fashion over two time steps. So the system starts with, suppose starts with some at time step 1 is given in this in the following way it is given as u0 plus w0 and at time step 2 is given by x1 plus u1. Now what are these variables here x1 and x2 are these are the this is these denote the state of the system at times 1, 2 respectively. This is x1 and x2. u0 and u1 these are decisions that you need to make. So these are variables whose values we need to find. w0 is an act of nature. w0 comes from the environment. So w0 is a random variable. So w0 is a random variable taking values plus 1 or minus 1 with probability half each. So it takes values plus 1 or minus 1 with probability half. So you can think of w0 as some kind of an initial state or you can think of it as a disturbance or noise that affects the system whatever you would interpretation you would prefer is something you can take. Point is that once you have w0 there is w0 and then there is an action u0 that you choose that results in x1, then there is another action u1 that you choose. So that x1 plus u1 then results in x2. The problem for us is our problem find u0, u1 minimize the expected value of the absolute value of x2. This is the problem. Now let us try to now see how we can approach this problem. Now here I have told you that what have I told you about u0 and u1 I have told you that these are decisions that we need to make. I have not yet told you what exactly do you know when you need to take those decisions. So as a result this problem what we are going to assume in the first version of this problem is that these are decisions that are going to be made without any knowledge at all without any knowledge. So these decisions are being made before knowing anything about the system. Notice that the system there are many stages at which you would get information about the system. For example you could get information about w0, you could then get information about x1, you could get an information about x2 itself. Now x2 is resulting from u1. So knowing x2 at this beforehand while choosing u0 and u1 does not make sense. However you could potentially know something else in between. So therefore use for example you could choose say you could choose u1 based on what is known about x1. A lot of these things can potentially be done but right now we are assuming that we do not know anything at all about u0 and u1 while choosing sorry we do not know anything at all while choosing u0 and u1. So therefore u0 and u1 therefore are deterministic. So because they are not functions of anything that any of the randomness in the problem they are essentially just deterministic the deterministic scalars that we need to choose whose values we need to choose. So let us try to solve this problem then. So our problem is to minimize this the absolute value of x2. So the absolute value of x2 let me write that as in terms of what we know. So the absolute value of x2 is x2 is just x1 plus u1 and x1 itself is u0 plus w0. So this is u0. So what we get is u0 plus u1 plus w0. Now w0 can take two possible values plus 1 or minus 1. So this expect each with probability half. So this expectation then evaluates to half times u0 plus u1 plus 1 plus another half times u0 plus u1 minus 1. This is what this expectation evaluates to. Now let us be careful here what is we have the way we have evaluated this expectation we have assumed that the only thing that is random in this the only entity that is random in this expression here is w0 the only random quantity is w0. So as a result we that is how we get to this expression. Now let us think of what this expression is basically saying it is this expression is saying well here is my here is suppose my real line here is minus 1 here is 1 and so this expression is basically asking you to evaluate the sum the sum of the distance of u0 plus u1 from 1 and the distance of u0 plus u1 from minus 1. So suppose u0 plus u1 suppose is here u0 plus u1 is here this quantity here is the distance of u0 plus u1 from minus 1. So this here is u0 plus u1 plus 1 and this distance here is u0 plus u1 my absolute value of u0 plus u1 minus 1 but u0 plus u1 could also be on the other side of minus 1 and 1 for example u0 plus u1 could be here. The other possibility is that u0 plus u1 could be on say the left of minus 1 for instance. So in that case this here is the distance this here would be the distance from u0 plus u1 to minus 1. So u0 plus u1 plus 1 absolute value of this and the distance of u0 plus u1 to 1 would be this. Now you can see there would be obviously a symmetric case with u0 plus u1 lying on the other side of 1. Now it is very evident by looking at this figure that whenever u0 plus u1 is either to the left of minus 1 or to the right of 1 in either of these cases this that would not be the right choice for u0 plus u1 because you are this this the sum of these distances that we are trying to minimize here would actually be much larger. So consequently the optimal one for us to choose would be one where u0 plus u1 is actually between plus and between minus 1 and plus 1 and in that case actually what is this the sum once u0 plus u1 is in fact between minus 1 to 1 the sum of this objective here which is half of this distance plus half of that distance would actually evaluate would actually evaluate to just eventually just 1. So what we conclude is that this the minimum of this minimum over u0 u1 of the expected absolute value of x2 is in is equal to 1 and the minimum is attained when u0 plus u1 is between plus 1 and minus 1. You can just take any u0 and u1 so long as their their sum is between plus 1 and minus 1 and that gives you the minimum the minimum of this of this particular problem. So you can you can see what the that that we can that here the object the the actual optimal decisions are are multiple that could that means there are there is a multitude of optimal decisions but the the the optimal value is is one for for any of them. Now what is now what here we remember what we assumed was that u0 and u1 are are deterministic. So u0 and u1 are chosen in in such a way that they they are chosen before the realization of any of of the uncertainty in the problem. So as a consequence when I took the expectation here the the the only thing that was random was was w0. Now what we will do is we will change we will change this assumption and we will we will assume that these u0 and u1 are actually chosen not before the realization of uncertainty but actually sequentially. So we will assume that u0 and u1 are chosen in a sequence and as you go along the sequence you get more information. So for example in this case so one particular way one one assumption we could we will make is so now assume so in this case here let me mark this with red earlier we had this was done assuming u0 u1 are chosen before w0 is realized so before this uncertainty is real. So here now I am going to assume that u0 and u1 are chosen sequentially now assume u0 u1 are chosen are chosen sequentially. So what does this mean? So u0 is chosen as before chosen as before but u1 is chosen after x1 is realized and x1 is known to the decision maker while choosing u1 now this is extremely important. So u0 here is chosen as before it is chosen before the realization of w0 but u1 while choosing u1 you actually have the knowledge of x1. So in order to for you to have the knowledge of x1 you have to wait for it to get realized so it is obviously chosen after x1 is realized and it is and after x1 is realized you in fact also know what x1 in fact is. Now this now changes the problem see what we now have is that you are not choosing u0 and u1 before the before the uncertainty is realized but you have the possibility now of adapting your choice of u1 to what situation actually arises. So this information actually changes the number of options you have effectively what you earlier you are required to pick only a particular value for u0 and u1. Now u1 can be chosen as a function of the information that you have. So u1 can therefore be can now be chosen in a way that it such that it adapts to the information that is that will get realized. So what would be the optimal thing to do in this case in this case we cannot really we should not really talk of one particular thing to do what we should really talk about is what we should do in each situation based on what gets realized. So the optimal thing to do is now not just an action or a particular value but rather an entire policy or an entire plan of actions that we need that we want to choose. So it turns out in this case that the optimal thing to do is let us try to think of what the optimal thing to do is. So again let me write out the expected value of this is equal to x2 remember was x1 plus u1 so this the expected value of x2 is x1 plus u1. Now x1 is known to me while I choose when I am choosing u1 x1 is known to me when I am choosing u1 so what should I take u1 as in order to minimize this particular expression. What I should do is the answer is simple I just take u1 to be negative of x1 I just take u1 to be negative. So since I know x1 when I am choosing u1 I can choose u1 as a function of x1 and so the answer the optimal thing for me to do is then choose u1 not as a specific value but something that is adapted to the information that I have since I have the information of x1 I am going to be choosing u1 as negative of x1. So choose u1 of x1. Now what happens to my choice of u0 well u0 did not even feature into the problem you can see that it does not matter what u0 is actually chosen as so u0 choose u0 as any scalar. So with this what I get is u1 equal with u1 equal to minus x1 my this would give me that the expected absolute value of x2 is actually equal to 0 and when this is and this is the obviously the least possible value the absolute value of x2 can take. So consequently this here is therefore the optimal solution. So notice here that u0 did not matter at all in the optimal solution you could have taken u0 to be 5, 10 whatever value you want the reason it did not matter is because you had all the information that you needed to make the objective value to take to get the objective value to its minimum possible value when you were choosing when you were choosing u1. So with the knowledge of x1 you could that it was possible to bring down the objective value eventually to 0 when you have that particular knowledge. So this is how you can see that there has been a difference in the optimal value of the objective based on the information that you had in our earlier example the optimal the optimal value of the objective turned out to be 1. The minimum value that you could get was 1 with in this in this case the minimum value in this case the minimum value is the minimum value in this case is equal to 0. Now notice here that we cannot just simply write this as an optimization problem because we are not looking at looking for just scalar values you need to write you need to be careful about what the what information is being supplied when while choosing every decision. So here this is being this is minimized such that u0 is any scalar and u1 is any function of x1 in that case the optimal value that we get is 0. So what we have see what we have seen here therefore is the role of information if you are given information if you change the information structure of the problem or if you change what is known at various time steps the nature of the problem changes the optimal value of the problem changes the optimal decisions that you can take also change. So with this what we have learnt is we have we have learnt two important aspects one is one is the issue of risk the other is the issue of information through some simple examples. What we will do in the in the subsequent lectures now is that we will start we will start coming writing out formal models of sequential decision making in which both of these aspects which is the issue of risk as well as the issue of uncertainty as well as the issue of information will play an important role. So in the following lectures we will see we will see a sequence a formal model of sequential decision making. Thank you.