 In the previous lecture we looked at the intrinsic model of stochastic control and there we showed that one can model a stochastic control problem without actually bringing in the concept of a state of a system. And as I mentioned that the this sort of model is very amenable to describing information structures. So in this model if you recall we had what is called the environmental noise. It is comprised of all the sources of noise whose distributions we cannot affect. So this includes the initial state, the measurement noise, system noise and so on. The information that we get at any time that we have at any time is written as a function of the action that are being chosen in the past and the environmental noise. So this function is determines all the information that we have at that agent I has at a time P. Then the agent I is action UIT is chosen as a function of this information. Notice that we can there are two different ways of modeling. One was to think of this as an observation equation in which case the information has to be carefully described by writing out what is the what exactly what part of the observations are actually available as information at any given point of time or you can think you can club together all those observations and write them out as one information equation. So it can also be considered as I said the information equation. The cost of the problem is also written without any state any reference to a state. So it is a function of only the actions that the agents take and the environmental noise that is present in the problem and the actions have to be chosen as a function of the information. So the action UIT which is the action of agent I at time t is a function of yIT. So the problem then is to find these functions gamma 1 to gamma n each gamma i is itself comprised of t functions gamma it running where t runs from 1 to t which is the time horizon of the problem. In order to minimize this cost j of gamma 1 to gamma n where j of gamma 1 to gamma n is the expected cost here. So this was the intrinsic form of our stochastic control problem. Now what I mentioned to you also was that these collection of these functions here describes the information structure of the problem. So the information structure is essentially is this collection of functions. So we can now talk of what it means for what various types of functions function classes or types of functions lead to various types of information structures. So the simplest of these is what is called a static information structure. So a static information structure and that leads to this particular problem the problem that emerges from there is often called a static team. Now one uses the word team to describe this problem because there are really n agents with a common goal. Now a team formally is comprised of exactly this it comprises of n agents with a common goal but with possibly different information. So they have to take a joint decide a joint set of actions or joint policies in order to minimize the total cost. But then they each have different pieces of information on which their policies are based and their actions have to therefore be chosen as a function of their local piece of information or local data or whatever. So the intrinsic model that I mentioned about of stochastic control is also often called a team model. It is used as the term often that is often used is that of that this is a description of a team problem. So what the problem that I will now describe is commonly known by in that sort of balance it is what is called a static team. So a static team involves essentially this. So y i t in a static team the information structure is as follows. So y i t which is the information of player i at time t is a function eta i t of psi alone. So the information that the agent i has at time t is a function of only the environmental variables. So what this means is at every time t every agent only gets some information about the noise in the system. It does not know anything about the actions of the other agents. So what this also means is that the actions of other agents cannot possibly affect the information of any other agent. So information of any agent is a function of only the noise or environmental noise remember of this in the system. It is not affected by the actions of other agents. So what this so how do we sort of visualize this kind of a system this kind of a setting the way to visualize this is to think that basically there is there are various sources of noise in that affect a system that is measurement noise as system noise there is probably the an environmental noise which is itself distributed and known only partially to different agents different agents. And what this function eta i t is capturing is what piece of that long environmental noise vector is actually known to agent i. So for example psi could be a vector which describes the weather conditions across in different locations on the earth. So it is some kind of meteorological information that is encoded in psi. So it is therefore a long vector which has components describing the velocity of wind velocity maybe the humidity, temperature, air pressure etc at each location and what is known at to agent i is only his own location is what is known only at his location. So then the function eta i t would then be essentially picking out the relevant components of that long vector and producing that as the information of agent i at time t. So this sort of model actually can be used say for example if you want to model a collection of distributed wind generators suppose there are wind generators located at different locations. So here is suppose one wind generator here is another wind generator at another location and these wind generators have to produce their outputs let us call these u i t and u j t the outputs of these two wind generator and they are produced as a function of the weather conditions at their respective locations. The weather conditions are this the information that is available to these agents. So they have to produce their outputs as a function of this as a function of this and now there could be a global goal such as for example they have to track a demand signal for instance so the they want to together make sure that they are not too far from from the prospective demand at that time. The demand itself could be time varying so they want to minimize the how far is their total generation from the demand so they want to minimize something like this. So then in this case the environmental variable psi would comprise of the as I mentioned the meteorological conditions meteorological conditions at all locations at all times. So these are the meteorological conditions at all locations and at all times and also in addition to this it also comprises of this the stochastic or uncertain demand that we want to follow. So demand at all times all of this can be clubbed together into the vector psi and the but only relevant only specific components of that psi are known to individual generators. So the so when UIT is being chosen only as a function of the weather conditions at location at the location of generator I at time t. So this is one sort of description one kind of motivation or application for this particular setting but this is also the reason this is studied is not only because this is most applicable but because this sort of setting has in it an underlying simplicity because there is really no effect of the actions of another agent on the information of any other agent. So there is no scope for agents to signal through actions or for there to be a dual effect or any of that. So thanks to this what happens is the actions of the actions that any agent is choosing can be understood more succinctly or more easily and the only effect at play then is the issue of decentralization. So the fact that multiple agents have different information is the issue at hand rather than there how they affect each other's information. So thanks to this this becomes an extremely useful first problem to study. So the where the information is only a function of the environmental random variable. So when one in addition to this one can also make the following reduction here notice that since y i t is a function of psi we instead of thinking of this problem as having n agents acting at n separate agents acting at time t at t different times. So that means n agents acting at t times instead of this one can think of one can think instead equivalently of n times t agents not n agents acting at t times but rather n times t separate agents. See remember all of these agents are part of a team. So as I had mentioned in one of my earlier lectures that one can either think of agents acting at different times as the same agent acting across at different time instance or one can think of it as if there is there are actually different agents but acting with different pieces of information. So one can equivalently have n times t different agents separate agents n times t separate agents. So as a result of this because and this is happening because there are actually the actions of the agents is not affecting the information of other agents we can really freely redefine what the what various agents what really we mean by an agent in the system. So thanks to this now what we find is that time is actually irrelevant time is irrelevant time in this problem becomes irrelevant. So in that is the reason why this problem is called a static team problem. So one can effectively just get rid of the index of time here and think of the problem in the following way that there is an agent i who receives information y i which is a functions i i sorry eta i of psi this here is our description of the information in a static team problem. So that the time axis has been removed one can one then has the the cost the cost remember is a function of u 1 to u n and also psi and each of these are chosen as a function of their respective information. So this here is gamma 1 of gamma 1 of y 1 all the way till gamma n of y n comma psi right and we so therefore the problem then is to minimize j of gamma 1 to gamma n in which is simply the expectation of l of gamma 1 of y 1 till gamma n of y n of psi. So we want to minimize this over gamma 1 to gamma n. So this is the description of a static team problem. So as I mentioned earlier so static in a static team problem there is no dual effect there is no dual effect in a static team problem. So the policy of any policy of any agent does not affect the information of any other agent. So the information of of an agent is a function of only the noise in the system. So it cannot be affected by the actions or the policies of any other agent right. Now this kind of requirement where the act the information is a function of only the noise this seems like a rather strong requirement because it seems to suggest that you know there is really no way agents can influence each other and therefore this problem might actually be reduced to something trivial. So the there are two the remarks I want to make about this. First is that it is not true that this problem is trivial it actually needs some work this problem as well and we will see an example to see with to get clarity on what how the form of this problem is and how does one actually go about solving such problems. So that is that is point one it is not at all true that this problem is trivial. Second is actually although there are although it is true here that the that we have sort of not allowed the influence of other agents to come up in this in this description there are actually a good number of problems with in the especially in the linear quadratic domain whose where the information where where we have actually classical information structures but those problems because of the linearity of the dynamics and so on can also can eventually be reduced to problems that that have a static information structure. So the structure of those problems is such that you can in fact by once the information structure is classical you can actually do a bit of elimination and manipulation and eventually write out information purely as a function of the noise in the system. Now you might recall that we had done something similar when we were trying to compute the optimal policy of a linear quadratic problem and linear quadratic problem with imperfect state information that there we there if you remember we wrote out the innovation or we wrote out that the error in the state estimate is in fact a function of only the noise in the system and there the argument was that one could back substitute the state equation and eventually get it to be a function of just the initial state and all the noise in the system. This this kind of argument is precisely what is needed in order to reduce the a problem with classical information structure and linear dynamics to a problem with static information structure. So the second lesson then is the first lesson is that the static information structure problems are not not necessarily easy and second is the static information structure problems actually generalize a lot of problems where which we have already studied which is of the where the dynamics are linear. So a problem where so this is this motivates why we should be looking at static information problems with static information structure more closely. If it does not have static information structure is it is said to have a dynamic information structure. So what is the negation of the information structure being static it is so in a static information structure the actions of any agent cannot affect the information of any other agent then in a dynamic information structure it just means that there are there is at least one pair of agents such that the action of one agent affects the other the information of the other agent. So in an information structure is not static is is called dynamic. The stochastic control problems with dynamic information structure are are called is are called dynamic team problems. So the resulting problems are called dynamic team problems. Now the if even though we have that that dynamic we if you look at dynamic team problems does not mean that the problem is necessarily hard of course all your all the stochastic control problems with with with classical information structure that we have studied they are also in general dynamic team problems of course some of them as I said can be reduced to static team problems but not all of them. So though they are in general dynamic team problems. So dynamic team problems may or may not be easy. So so the of course when they have classical information structure and they are enough when the cost is in in in the form that we have looked at where there is a stage wise separation and so on in that sort of form those problems tend to be easy. So this is something to keep in mind. So for the rest of this course we really would not have much time to go into the various intricacies of dynamic dynamic information structures because the the number of information structures in those problems in those problems is extremely vast and it will take an entirely separate course to to understand exactly what the you know what sort of information structures play out in various problems there. So what we will do now is in the in the rest of this course is is look at static team problems more deeply and do one example of a static team problems with static team problem but with various static information structures so that we understand exactly what the nature of this nature of these sort of problems is. One final historical remark I should make about static team problems is that static team problems were actually first studied by Marshak and Radner I believe this is the correct spelling Marshak and Radner. They were concerned about the this was in the 1950s I believe and they were concerned about the theory of organizations. So there for them the n agents were actually n individuals who are who are together forming an organization and the cost function of the organization was maybe the organization's profit or something like that but then because these were these were n different individuals they each had different different pieces of information about the about the underlying state of the world. So they may have different projections of where how a certain event would play out they may have certain projection different projections about say what the what a competitor is doing or what the what a certain you know maybe there is a strike in the factory and how this strike might play out or something like that. So they may each they effectively are n different individuals or n different age what we call agents and they each view the world differently but they all have a common goal and moreover they are in a static situation meaning that they their actions do not affect the information of other agents they just have to together choose a pair of a or a tuple of actions so as to minimize minimize the cost for the problem. So that is that is their their study of a static team. So what we what we see as I think as a result of this is that stochastic control problems are not really the are not limited to the control systems problems in the standard sense. So they are not necessarily about industrial control systems with physical systems and and so on they they can they are also just as equally applicable to situations where we to social situations to organizational situations to to situations of communication and so on. So any of these any of these settings whenever they which involves either multiple agents or decentralization or some kind of distributed observations they are essentially stochastic control problems. So one must not make the mistake of thinking of stochastic control in a very narrow way. So this is also the reason why at the start of the course I I promise that this this course is not really a conventional course in control or conventional course in stochastic control. It is really a course in which in which I try to view build a view where so that stochastic control, economics, communication, theory of teams, theory of organization all of these can be thought of in one kind of common framework. The the the the the dynamic in the the information structures framework that we have developed in the in the previous lecture and this one is in fact one such framework. It actually allows us to think of various types of settings including organizational organization behavior ranging from organizational behavior till to you know communication stochastic control etc all of these in in one sort of one sort of light and learn from the insights that we have developed in across these various various disciplines. So this is this is this I think is is the is the main takeaway from this course. So as we go further we will see more of more manifestations of this. First we will be doing static teams and doing a deep dive in static teams and understanding the role of information information in static teams and then from there we will go again to to a specific type of dynamic team which is the one that comes up in communication. So this this is the schedule for the rest of the course.