 So welcome to the Smaqury seminar. Our speaker today is Professor Alejandro Domingos Cascio from University of Illinois at Bernard's Champagne. He's going to talk about data driven control of distributed energy resources. Just to remind you that our next seminar is next week at the same time. His speaker is going to talk about cyber security. Professor Alejandro Domingos Cascio is a professor in the BC Department at UIUC where he is affiliated with the power and energy systems area. He's also a research professor in the coordinate science lab and information trust institute. He has been a Granger associate since 2011 and a William Aparit scholar since 2002. His research programs include control, distributed energy resources, grid data analytics, and uncertainty analysis in electrical energy systems. He received a degree of industrial engineering from the University of Oberto in 2001. This is in Spain and the PhD degree in W.E. and E.C.S. from MIT in 2007. So he received the NSF Career Award in 2010 and the Young Engineer Award from NSF Career Award. He's currently an editor of the Artillery Transactions on Power Systems and Artillery Power Engineering. So we've had our staff sharing our screens and that's because it's not the presentation. Thank you very much for coming in today. It's been two years since last time I gave a talk at the public venue like this so I'm very happy to be here. Two years to this day actually. It was my fifth the last time I was out there in the world. Anyway, okay so I would like to start the presentation by giving you some background on why the programs I'm going to be talking about are important and I'm going to start with this chart where you can basically see the breakdown in terms of the primary sources of energy for electricity generation in the United States and you see that about 65% of the electricity generated in the United States is done using fossil fuels about which 35% is natural gas, 27 coal and 1% oil and other things. So that's pretty high. So if you look at this other chart about the contribution of the electricity sector to greenhouse gas emissions about 28% of it just in the United States comes from electricity generation just trailing a transportation sector and worldwide it's about 42% the total emissions from electricity generation and so on. So we need to address this and the way to address this is by basically utilizing renewable based power generation. So that's fine. That's a great solution but renewable based generation also comes with its own challenges. So for example, a deep penetration of renewable based generation might cause trouble in power systems in terms of frequency regulation and distribution systems you might have issues of voltage control. So typically frequency regulation at the bulk level is done using large synchronous generators but those are being displaced because of renewable based generation so that's a problem. So on the other hand there are some resources at the distribution level that you could utilize for that task. In terms of voltage control on distribution networks typically that's done using LTCs and switch capacitors but the problem with those devices which are mechanical in nature is that they are not designed to expand the variability of the renewable based generation in case you have all these issues. So those are some challenges introduced by the integration of renewables. So how can we address those? So one potential solution is the number of distributed resources that are being integrated at the distribution level. Those typically come with inverters and that's how you interface them with a grid and you could potentially utilize those resources to provide the ancillary services to the grid. So that frequency regulation that I was talking about that could be done if you were to coordinate these resources appropriately. You could also use them to operate portions of a distribution network in islanded operation in case there is a blackout, the notion of microgrid and so on. So a lot of opportunities in space if you start utilizing the ERs. So the issue with the utilization of the ERs is that, let me see what's going on with the clicker. I don't need the battery. So the issue with utilizing the ERs is that you need to develop appropriate control schemes for the utilization, right? And that's something that we have not done in the past. I mean, these are relatively new technologies that are being integrated into the grid. So you need to come up with those schemes. The problem is that if you were to use the traditional approach of using model-based control schemes, you might run into issues because you may not have accurate models of the distribution network, right? So you are much better off using some sort of data-driven approach in which you assume that you don't know the whole model of the system to which the resources are connected. And basically what you try to do is learn it somehow. And that's basically the premise that my group has been taking, that the data-driven approach to control distributed energy resources. So in the talk today, I'm going to basically tell you about two problems that we've been working on in this space, okay, of control distributed energy resources. One of them is the problem of voltage regulation, where you are trying to modulate the voltage at the basis in our distribution network to reject the disturbances introduced by renewable generation another sources of uncertainty like load variability and so on. And then the other problem I want to talk about is how do you do generation control in a distribution network when it's operating in another landed mode and you are just relying on inverter-based ERs, okay? So think about that as a microarrange. In these two problems, like I said, we are not going to be relying on a model of the system but we are going to be trying to infer such a model as we go along with the design of the control, okay? So that's the roadmap for today's presentation. All right, so let's start with the first part. Let's talk about voltage regulation in distribution and networks. How many of you have the electrical background here? Okay, so do you know then what the... I think I'm going to switch to... Do you know then what the load-to-tap changer is? So in the distribution network, the way you do voltage control is via these load-to-tap changers, okay? So these are like transformers in which you can change the turn ratio and that's how you do voltage control along with switch capacitors and so on. So here what you can see is basically an electrical model of one of those devices. So if you know a little bit about the transformer modeling the way this is just model is voltage on the secondary side here is equal to the primary side divided by the square of the turn ratio and then you have all this stuff here that is basically taking into account the resistance and reactance of the transformer plus any resistance and reactance associated to the transmission line that is connected in series with the transformer. So this is the model you have. And basically this turn ratio is a change. You can change it if typically it takes 33 discrete values ranging from 0.9 to 1.1 in per unit, okay? So this is basically the way you adjust the voltage in the distribution network. You're going to change these setups. Okay, the way current LTC control schemes work are not optimal in many ways, okay? So first of all, they are myopic. They are just based on local voltage measurements and they definitely don't take into account any future uncertainty effects on the current control actions, okay? So that's a bit of a problem. And beyond the fact that these controllers are suboptimal the other issue is that you cannot really rely on them for handling the variability associated with the renewable base generation or new loads that are coming into the system as such electric vehicles because these are, like I said, mechanical devices and if you operate them too often to try to reject the disturbances associated with these sources, you're going to basically work them out really quick, right? And in addition to that, you have another source of variability which is you could be utilizing these to provide regulation services to the grid that adds another layer of viability to the system, okay? So then we want to address this issue. How do we address this problem of LTC control schemes not being really suitable anymore because of the new sources that are coming online? Whether it's that through a new ballpark control architecture that we have been designing over the years we rely on two types of actuators. One is that they act on a slower time scale and this includes the LTC devices that I was talking about switch capacitors and so on. So basically you're going to decide the position of those devices at this time T0 to 1 to 2 and then in between updates of those what you're going to utilize is inverter interface DTRs with the ability to provide reactive power with the ability to provide reactive power to do the fine tuning in terms of the voltage. So you're going to be utilizing this in between time instances to do the fine grain voltage control and try to track that faster variability, okay? So that's that. So today, not today, in the first part of the talk I'm going to be mostly focused on the slow time scale control and then on the second part of the talk I'm going to be talking about how you address the control of this inverter interface DTRs to achieve this architecture. All right. So let me tell you a little bit about this optimal LTC dispatch problem. So here's your distribution network connected to maybe some external system through some tie line. Here you have what you see there are the there's one there and there's one there. Those are your LTCs, your regulation transformers and the problem is to find a policy for determining the position of the LTCs based on the current position of the of the taps, okay? So the current turn ratio and then whatever mass voltage you can measure locally. Okay. So that's the idea. And then what you want to do is that policy needs to be such that you're minimizing voltage deviations from some reference value as power injections to actuate over time. Okay. So that's that's the idea. So that's the objective of the problem. So now let's see if we can formulate the problem mathematically. All right. So first of all, we need some sort of system model. I mean, I said that I was going to not utilize models, but if you want to formulate the problem, I need to tell you that there is some underlying model describing the physics of the system. And then I'll tell you we don't have access to that model. We're going to try to do things with data. So the model of the system is basically shown in this slide. So you can see that we can relate the squares of the voltage magnitudes to the active and reactive power injections and the top positions of the LPCs. Okay. So some sort of model that where this function is capturing network topology and transmission line parameters. So if you are familiar with the power flow, just think of this as the power flow equations. But of course that means that you would have to solve for them. But in an abstract sense, this function G, it exists. You can obtain it if you weren't able to solve the power flow and characterize things analytically that you can most of the time. Anyway, so that's at least conceptually the model. Now let's make some assumptions here. So we are going to assume that the changes in active and reactive power injections at the instant K, which would not by delta P and delta Q are random. So that's where our randomness comes into the picture. And we're going to assume that the actual active and reactive power injections P and Q, either without measurement, we don't measure it or we don't know their joint priority distribution. So we don't have a whole lot of information about them. We are going to assume the network topology is known. They are good algorithms for topology detection. So that's not a big issue. But we are going to assume that the line parameters are unknown. We are talking about the distribution network. So you don't know what's out there. So there could be like an underground cable over headlines, but you don't have a good understanding of what those parameters are. So you're going to assume that you don't know those. OK, so because this first assumption here, the natural way to do, the natural thing to do is to formulate the problem as a mark of decision process. Anyone familiar with MDPs here? OK, good. OK, if you'd be perfect. And then because this other two, these other three assumptions that I have here, we are going to have to resort to reinforcement learning techniques to solve the MDP. And I'll be more specific of why that's the case. All right, so that's the premise. So if we are going to formulate the problem as some MDP, I want to give you basically, I want to give you the ingredients of the MDP. So the ingredients of the MDP are the state space. The state here is going to become a price for the top ratios, T, and the square voltage and magnitude. OK, so that's going to be my state here. The action space are going to be the changes in the LTC top positions, not the top set themselves, but the changes. The delta is here. OK, that's one thing here by delta T. Those are my actions. OK, so the change from instant K to instant K plus 1. And then the reward function here is defined basically based on voltage deviations from some nominal value. So reward, if you are in the state A, you take action A and you transition to state S prime. It's just the voltage deviation. And for the new state at which you learn. OK, remember that the state contains both the top positions and the voltage. So that's how you define that reward. OK, so essentially we are penalizing voltage deviations from some reference point. That's the idea. And now that we have the ingredients of the MDP, we can go ahead and formulate it. So state transitions are going to be covered by random changes in active and reactive power injections delta P and delta Q. OK, remember that I told you about those before. So potentially, again, you can write the new state value S prime as some function of the current state, the action you take, meaning the change in the positions of the low top changes and then the delta P and the delta Q. So you have certain power injection active reactive power. You change that a little bit. And then you also change the top positions that's going to land you in a new state, even the previous state you were at. So that's the idea. And again, you can think of this function as something that you could conceptually obtain using the power flow model if you were able to solve it. OK, conceptually, potentially. All right, so the other thing that you have to note here is that the probability of transitioning from S to S prime under action A could be computed if the function H and the joint PDF of the disturbance is delta P and delta Q were known. But actually, we assume that these were not known and also the function H is not known. So that's basically the first problem we're running to here. So we don't have enough information here to solve the NDP analytically or even numerically because we don't have the part of the model. So now what is the objective here? What we want to do is find the control policy that acts based on the current state, meaning the top positions and the voltages and tell me what to do, meaning how to change the, how to modify the tops. So that we are minimizing the discounted some of the expected voltage deviations over time. So that's what we are doing. Or if you have throw a minus there, you are basically maximizing the disturbance. I mean, I'm doing basically everything based on maximization because I was talking about reward earlier, but if you, if you define it to be a cost function, then you are basically minimizing. Anyway, that's how they are the equivalent. So that's, that's my objective here. All right. So that's clear then. Okay. We go back once. This I did. Sorry. This computer is misbehaving a little bit. Now, let's say I'm denoted by our bar, the expected the reward for the pair. The pair s a so the action, the state s and the action a. So then the NDP is a soft. If you can find a function q star, which is the optimal action value function that satisfies this equation under this is some version of development equation. Okay. So, so what do you have here? That's the action value for the optimal action value function for the pair s a the. So technique. Precisely is the value of the oxygen of the optimal action value function for those two values. And then what do we have here? That's the discount factor. And then these are the transition probabilities of going from state s status prime. Right. You would get the optimal policies by obtaining the argument that maximizes this action value function. Okay. This is a standard NDP theory. Oh, that's all great. But the problem is that as I've been telling you, so all along we don't know the transition probability. So we don't know this part of the model here. This we don't know. Okay. And even if you need that, those transition probabilities, we could not solve it for the optimal action value function because of the curse of dimensionality. So the state and action spaces are so large that you cannot really solve this in a, in any way, in any efficient way. So what we do to address these issues is we're going to circumvent issue number one by applying a model theory, reinforcement learning algorithm that utilizes transition samples that we obtained using a virtual transition generator that utilizes data that you take from the system. And then there is some clever trick to exploit that data to create new data without really relying on the model of the system because that's always the issue. Right. How do you generate that data? Oh, I can use a model of the system. But if you can use a model of the system, why do you like resorting to a model free algorithm? Right. So that's a, there is a way to do that. I'm not going to spend too much time on it. If you're interested, then you can be the paper that where all this is written up or we can talk at the end of the, of the talk. Okay. And then the other issue that we want to address the, the fact that the fact that the state and action spaces are too large to solve this in any efficient way, what we use is we're going to have some learning scheme in which we can do a sequentially learn the action value function. Right. So that's a, that's what we do here. Okay. So let's see. So let's talk a little bit about the obtaining the estimate of the action value function. So if we have Q bar, sorry, you have to note in the approximate value, the approximate of the approximation of the optimal action value function. So there are a couple of ways in which you could get this estimate. One is by using parametric functions. And the other one is neural networks, which are really very fashionable. So we actually resort to a parametric characterization and we just stick actually with the linear parameterization because, you know, this is not such a complicated problem besides its size. So linear approximation works pretty well. You don't have to throw anything fancy to it like a neural network or anything. Okay. And then what we're going to do is we are going to learn the weights here, the weights on this linear parameterization by basically using the so-called least squares policy iteration algorithm. Okay. So this is an algorithm that all it's doing is just basically fitting this transition sample that you obtain from measuring things in the system and also from this virtual transition generator and then you fit them to this function and that's how you obtain the weights. Okay. So again, you know, if you are interested in the details, you can read the corresponding papers. Okay. So that's how we do this. Now what are some of the challenges here? This is the square error policy iteration algorithm. It requires adequate transition samples that spread all the action and state spaces. Okay. So how do we generate those? Like I said, using this virtual transition generator, okay, which exploits as I told you before the data that we have obtained from the system and then uses a clever trick to generate more data without really knowing the model of the, without really utilizing the model of the system. Okay. And we want to do that because you don't want to be exploring with the actual system because the system is online in operations. If you're exploring on a real system like that, you might cost a blackout. So you don't want to do that. Okay. The other thing that is problematic is that the LSP, the algorithm that we want to use to obtain the, the action value function approximation also suffers from the course of dimensionality when the action space is very large. Okay. So we came up with a heuristic, which is instead of trying to solve, instead of trying to figure out what is the optimal policy for all the LTCs at once using this, this LSP algorithm. What we do is we do it one by one in a round robin process. Assuming that the other LTCs are not really doing anything. I'm not moving. And then after a few rounds of that round robin process, you convert it to some policy. And while it's suboptimal, what we have seen from simulations is that it's pretty close to what you would get if you were trying to do the whole thing at once. Okay. But that way we can scale this up. All right. So let me now show you some simulation results. For a 123% system. So here we're going to have the four LTCs. Here, here, here, and here. All right. And what you see here are some simulation results. In which we are comparing basically the rewards. For example, here of our bunch reinforcement learning algorithm, the blue trace. So that compares to the optimal solution that you will obtain using exhaustive search. For this case, because it's not a super large system and then the actual conventional scheme and the myopic scheme that I was telling you before that is only active on local voltages. So you can see how obviously the bunch reinforcement learning algorithm and the optimal one, the exhaustive search outperform the conventional scheme. More interesting. I mean, what we see here is that the batch reinforcement algorithm. Okay. Our approach actually is very close to the exhaustive search. So it performs very well. You could see the same thing. If you look at the rewards over seven days, these remain close and so on. Whereas the conventional scheme is doing its own thing. You can also see how much variability you have on the rewards whereas here they stay flat. And then here you have basically the trajectories of the top position for all three schemes. So, so that's a, that these are basically the numerical results for this. Any questions so far? Yes, go ahead. What is the time scale that you're having to take these actions? So here we are simulating this over 24 hours. And then you could be that depends on how far you want to do that. Your, your analysis, right? You could basically dispense the maybe every hour. And then in between hours, you could do your find the tuning with the inverters. If that's not sufficient in your system because of variability, then you could shrink it maybe to half an hour. And then you'll define tuning with your guys. It depends a little bit on your system. But it doesn't need second level. No, no, no. That's really much. And then last question. What was the simulator you used? Let me try to think about it. I think it was just custom made. Code. Because they're mad up or Python. Because at some point we switched to Python. This is just straight out. This flow, power flow model. So that's what we are using for simulating the power system. And we coded that up. I suppose it wasn't like grid lab B or anything. No, no. No, we just custom made it. Yeah. Yeah. Go ahead. So you said that you generate the trajectories from like your own, without like an actual law. So you generate the trajectories. How do you like handle where. A. This. If the directory starts going off. It can have like this like compounding. It grows and grows and grows. It becomes very wrong. I'm trying to ask, like, how do you, how do you generate the trajectories? How do you generate the trajectories? Because it's very wrong. I'm trying to ask, like, how do you, how do you make sure that the samples that you generate is close enough that you can. You mean for the, the virtual samples that we are generating? Yeah. So the way it works is the, is the following that I just played without the, without the map. You have samples from your system and those samples correspond to measurements from your system. Those correspond to, to. Delta piece and Delta Q is variations in your system. That's actually what's exciting your system. So that's the excitation you're going to have in your system. And what you want to do is generate. New samples that use that excitation. You don't want to have a basically further excitation because that does not what you're going to be exhibiting your system. But what you could see is like a, a portage is going in different places because of the actions that you're taking. And that's what the virtual transition generator is created. Essentially. I can show you the, if you want to, at the end, because I mean, I had a couple of slides on that, but I decided to remove them because I wanted to talk more about the second part. But I can show you how it's done. Do you want to? Sure. Sounds good. There was a question really for you. Yeah. Was PV considered during the simulation? Yes. Yes. Anything else? Yeah. Very good. Are we good? All right. So let's move on and talk about the second part. So that's the first part of the talk. Program of voltage control on a distribution network. How do you address it without having a model of your system? The second part I'm going to be talking about. I'm pretty excited about this part of the talk, because this is a relatively recent result. So this is, in fact, the first time I'm going to be presenting this. So we'll see how it goes. So the problem here is that you have now a distribution network that is mostly based on inverters. Okay. So all the resources are interfaced via inverters. So no inertia in this distribution network. So you have a portion of the system with a bunch of distributed resources that is highlighted from the rest of the system. And what you want to do is control the system. So which means you have to control frequency. You have to control voltage at certain basis. And you may also want to control a flow of power for certain transmission lines. So it's just a smaller system based on this inverter. So how do you generation control? That's the problem. Okay. And then you also want to make sure that whatever you are doing with your control, the active and reactive power injections of the inverters are not exceed their capacity limits. Right. So that's it. So we want to design a controller that does it. So here the controls are the reference points, the active and reactive power set points for the inverters. And your output, what you're going to be trying to regulate is the system frequency, voltage magnitudes at all or a subset of the basis, and then flow of power to certain transmission lines. Okay. And again, conceptually, you have some sort of relation between these inputs, the set points of the inverters. Also, here side is describing the, I had it in the previous slide, but this is describing some disturbance to the system. So it could be a control load, or it could be a PV generation that you're not controlling whatever, but that's also affecting your, your system. Right. So again, you can write basically this model in which the, the, the, the value of the outputs at a certain instant, the tk plus one minus, depends on what you did previously. And here, I know that the notation is a bit confusing, perhaps, but why the k index for the measurements and the outputs and the k index for the use is exactly different. That's what I have here, this, this, what they represent. So for the inputs and the disturbances, those are enthusiastic or measure at instance tk minus one, tk and tk plus one. And then the outputs here are measured at tk plus one minus, which is a little bit later. So this is basically telling you how the system has evolved based on how you chase the controls. Okay. So they, then the time instances here and here, they don't line up, but this is what's the way this is written up. Okay. Sounds good. All right. So now how do we solve this problem? I'm going to use this idea of how optimization based feedback. Okay. So we have some optimization problem. Whereas what we are trying to do is minimize the deviation of my outputs, what we are trying to regulate with respect to some reference command subject to the model that I described to you being satisfied. So whatever controls that we are choosing and whatever disturbances you're measuring needs to satisfy this output input output equation and then the capacity needs to be satisfied. You solve this optimization and then you choose your control at instant base of that solution. Right. What are the issues here? Okay. This controller assumes that this input output model here is a no to you and also the capacity needs of the resources are also known to you. In practice, again, getting that model an analytical intractable model that you can work with. It is difficult because think about what this is doing. You would have to be getting some analytical expression for the solution to the powerful equation. So you don't really know this model, even if you can write it this way. That's my thing. And at this capacity limits, those in practice, again, you might have trouble getting them accurately characterized because they could, some of them could be associated with PD limits. Okay. So PD capacity limits and those can change over time depending on the installation you have in your system. So you don't know those accurately. All right. So what do we do about that? What we're going to be doing is formulating the problem in a slightly different way. I'm going to take my input output model that I show you and linearize it. So this delta Y now is basically denoting an incremental change in my output and then delta U is denoting an incremental change in my input. And then this matrix S here relating them is just the partial derivative of the function H with respect to the control evaluated at this guy and that guy. And I mean here there should be another term, but we are assuming that the change in the disturbances is small enough. So we drop that term here. So now let's say use this model to formulate any optimization. So this is the new optimization that I'm optimizing that I'm formulating. And here what we are trying to do is basically the incremental changes in your system need to track the incremental values that you need to track. And then that's again another subject because you are using them instead of the control actions, incremental control actions. What you need to make sure is that the incremental changes in your control are within incremental capacity limits, not the absolute capacity limits. And then we are going to hear this regularizer here to penalize large-scale control deviations. So you solve this optimization and then you set your control at instant K to V, whatever you have plus the solution of the optimization. It's just an incremental model at this point. Right? Now that's OK in theory, but in practice, you have still the same problem. If you didn't know the model H, you don't know this matrix either. I mean, OK, fine. The matrix S is slightly easier to calculate because if you have the powerful equations you took there, you could take partial derivatives, and potentially you could use the inverse function theorem to get that matrix S. It's just manipulating the sub matrices in the power flow Jacobian, so you could potentially get that. But again, if you don't know the model of your system, you don't know the power flow modeling in detail in the first place, so you cannot get those sensitivities in the first place. Right? So what we are going to do, take the same approach before and try to estimate them. OK? So we are going to utilize measurements of my output and my input to estimate this sensitivity matrix S. OK? And also I'm going to, at the same time, estimate some of the incremental capacity limits. OK? For those that are not aware about it. And now I'm going to use all those, again, those estimates in the feedback optimization that I show you. And this is basically a block diagram of how the whole approach works. You have your system, you take measurements, and those together with the inputs that are generating those measurements, you throw them at the sensitivity estimator, and then you use that in the formulation of your control, which is the optimization I show you. And then there is also this block here that basically utilizes also measurements from the system to generate incremental capacity limits for certain years, specifically the TV installation. Those are the ones that you typically don't know the incremental capacity limits well. So that's basically the approach. All right. So let's talk a little bit about the sensitivity estimation. First straightforward, we do it through regularized LSA. All you are doing is basically solving this optimization here. We are just trying to fit the changes in the inputs to the changes in the outputs to show matrix S, and that's basically your sensitivity estimate. And then again, we are basically, we have some regularizer here in the optimization. Not the big deal. You can actually solve this problem. It's a spray-stree forest analysis problem, so I'm sure you have done that in some homework before. The solution to regularize problem right here in this matrix form. And you might be wondering, why is this telling us all this stuff, which is text in any textbook? The reason why I want to do that is because in order to obtain the estimate of this matrix S, which is what we want to have, we need to compute the inverse of this matrix. And the inverse of this matrix, look at how it's built. It's the discounted sum of this outer product of the incremental changes in the delta use. And the problem is that this matrix may not be invertible if this sequence here of incremental changes is not persistently exciting. So if the delta use such that the sequence is not persistently exciting, then this matrix is not invertible. And that's a problem. Now, wait a minute. You are using now this delta use are actually the result of that optimization that I was telling you about. So how do you fix that problem? You're getting your incremental changes, your controls come out of that optimization and that's how you're doing this estimation. So how do you fix that issue, right? So the way to fix that issue is to build into the optimization some sort of mechanism that will ensure that this sequence of inputs is persistently exciting. And that's basically what I'm going to be telling you how to do. All right, so again, this is the optimization we had before except that what I did here is replace the matrix S with its estimate. Fine, all right. So if you take a nice approach to the problem, you could say, look, I'm just going to solve this optimization which identical to the one that I had before with this sensitivity matrix here, replace by its estimate and I'm going to set my control input to be this guy, right? To be the solution of the optimization and that's it. The issue with doing this is that you might get the sequence of controls that is not persistently exciting. So that's a problem. So that's one issue. And then the other issue that also contributes to the sequence of inputs not being persistently exciting is that this matrix S typically has more columns than rows. So even if the sequence of inputs perhaps is persistently exciting, you might not be exciting the null space of that matrix. So again, that compounds the problem, right? So that's what you have. That's what you need to think a bit about this. So how do we fix that problem? We're going to take again the feedback optimization that I told you before and I'm going to add this term WK. This is basically identical to what I had before but I added this WK. This WK is going to be zero if this sequence of incremental changes in what the variables that you are trying to track is persistently exciting and otherwise you're going to sample from this uniform distribution. So what is this term doing? This term is doing the following. Let's take the trivial case with this matrix S it's actually a square matrix. So this matrix S is a square matrix by adding this random perturbation here which ensures that this whole thing is persistently exciting. The solution is just basically that your control is going to be the inverse of this matrix times that guy, okay? This is a trivial case but because this sequence is persistently exciting and this matrix S is full run you get a sequence of control that is persistently exciting to your estimation. So that's basically the reason why you want to have that. However if the matrix S is part has more columns than rows by doing this you might also be you might only be getting excitation on the part of the control that is in the subspace that is orthogonal to the null space of the matrix S. So this matrix S if you have more columns than rows it has a null space and by taking this approach you're only getting persistently excitation on the part that is spanning the row space but not the null space of that matrix. So that's the problem. So how do you fix that? The way to fix that the way to fix that the way to fix that is very straightforward. You solve the optimization as before and then you look at the you look at the sequence of controls actually it should be delta you know delta way start so if the sequence of controls is persistently exciting then you don't do anything but if the sequence of controls is not persistently exciting you are going to add this time here and this time here is just basically a vector that you are taking at random from the in the null space of that matrix I'm multiplying by some constant data that you also take at random so you are adding this time here whenever the sequence of controls is not persistently exciting to make sure that you're exciting that the component of the input that part of the input Okay, so that's that. So the combined effect of this W in the cost function and this term in the null space of the matrix that gives you a mechanism to get enough excitation to do your estimation right that's that Okay, so there are a couple of slides here that tell you what happens if when you add this term here remember that if you solve this optimization here you are solving it when you solve the optimization you might get a solution that is actually hitting one of these limits so if you are trying to add this term you might be violating the capacity limits so the way to understand that is to add in these terms here in the capacity constant that you choose them at random so you are basically shrinking the feasibility of the optimization a little bit just a little bit so you get some new way to add that orthogonal term and then the last thing that I wanted to tell you is what about the capacity limits I told you that for PD installations those capacity limits might change because this is a typical PD care for a PD installation but this maximum here depends on the amount of installation you have so if the installation is lower this might be lower right so this is the current set point for the current output then you might get less incremental capacity so if you want to make sure that you are capturing that viability by estimating this curve and that's what that other block that I have in the block diagram I showed you earlier does anyway I don't want to bother you with the details on this part alright so so that's the other thing that I also did here that I don't want to bother you with the details either is that we also have model the residuals in the linear model that we used to predict what the outputs are doing based on those sensitivity estimates and that residual modeling we can incorporate it also in the optimization to make sure that if the sensitivity estimate is performing poorly you are not taking control actions that are unsafe those details are all here and at the end of the day what you get is some sort of some sort of constraints that are capturing all that anyway that's all a lot of details that I'm going to bother you with so what I want to show you now is how this approach works so here is a 13 bus system with 5 inverter interface resources so at buses 2, 3 and 6 we have what we call grid forming inverters and then at buses 1 and 8 we have grid following inverters so you can think of this as the PV as buses with PV installation and those are the objectives to make sure that the sensitivity to 60 Hz and regulate voltage at certain buses so here are how we generated the data for the PV generating resources at buses 1 and 8 we took some data from an NREL software measurement grid that's the data of the maximum solar power that is available and also the estimate that you would get based on the approach that we have to do that estimation and then this is the data we are using in the simulation so what you are seeing here is how the system reacts after you have a load perturbation so you are performing loads 34 seconds and here we are executing the load that happens with frequency after you perturb the load let's say at the T0 that's where your frequency lands and then as you start taking control actions you see how you bring the C frequency back so each of these blue dots are perturbations in the load that are causing frequency and then you see how the controller reacts to that you see the same thing here with voltage so you see that this approach is pretty well to control voltage and frequency decisions and that's pretty much it in terms of what I wanted to tell you about so here are some concluding remarks about the talk so I basically told you how to address a couple of problems when you have power systems with deep penetration of the ERs so one of them is voltage control the other one is generation control and then both of them we assume no prior information or very little information about the underlying model of the system so we are mostly relying on measurements to do control and if you want to know more about the details of this works you can read these two papers on my website just google me and then you'll find them so this is the one where we talk about voltage control and then this is the more recent one of the second part of the talk that I was telling you about so that's all I have for today thank you I actually have a few questions the first one is on the second part of your talk is that something that would need closer to real time control 30 minutes yeah so think about this as a replacement for a secondary control in a power system so the AGC type control that you are executing every 2 to 4 seconds that would be equivalent to this but instead of relying on the model to do the design of the controller you are just relying on this feedback on line optimization idea these are going to be distributed devices that are greater in theory that optimization that I show you you have to solve it in a centralized location which is not that different from what you would do in a bug system in practice a lot of the work that my group does which I have not talked about this a lot today is how to distribute those computations distributed optimization and control so a lot of this optimization problems that appear in these settings we have ways to solve them in a distributed fashion so if you do that then you can just have a computing devices attached to each of the resources they exchange information and that's how they do it so if you want to know what we do Luis is the guy to talk to because we work in our lab doing that stuff so he knows that stuff we work and the second question was this is very exciting work is there any I'm sure there's effort but are you working with any utilities to get pilot deployed not yet not there yet I mean this is pretty fresh but I would love to at least if we could get the data from some utilities to test the algorithms offline that would be that would be great but no we have not talked to anyone yet about it yes Luis so you mentioned the relationship between the sensitivity estimator and the input control so and how do you solve them independently you use like generating data for the input to get the estimator and then use that estimator in practice no that's the thing right that you're doing it all at once so you use the so that's the trick here right that you are using output data from your system together with the control you're offering to obtain the sensitivity estimate but then the controller that's the one that is generating that data so these two problems are coupled so that's why you need all those mechanisms to make sure that you're at a slightly better input so that the sensitivity estimation still works so they are really coupled it's not that I'm going to solve my my control problem for a while I know the matrix and then I use that to to do an estimate of the matrix no they are really coupled at this time you are basically doing both I have a quick question yes in your fast scale maximization does it depend on the scale you have to choose during the time you may have a lot of degrees and nine years above the average that requires updating more frequently or less frequently that's an excellent point now that you mentioned it makes a lot of sense that this rate in this case we are executing the controller every second but you were saying that you could actually execute it in a much lower time scale actually we don't know until we see exactly my guess is that that's a good question in the sense that maybe there is some way which is not including the optimization of course but basically it tells you how to adjust the rate on your controller some mechanism that is tracking the viability of the things that you are trying to control perhaps and it's basically adjusting the rate if you don't do it fast enough you can over repeat it yeah yeah yeah yeah so again I need to look into the literature of time figure control but I think there might be some ideas there basically tells you how to change the rate of your controller this was very exciting to have you all to be in person so that was great thank you