 which is relating to the previous diagram that I showed and it's essentially what we do with the actions or how do we make decisions regarding our actions on the system. The motivation is obvious so I'm not going to give any more information on this but along the lines of the previous presentation we're seeking basically a policy or a plan for optimal inspection and maintenance planning and coming back to the original influence diagram I will be discussing I will be looking in this problem by separating it in terms of three components the system itself which basically is described by condition or state the observations or structural health monitoring technologies that we can implement or use and what is called actions which is basically or essentially the decisions on how to maintain or intervene in the course of the management of the system and so if we look at it in terms of the simple graph I am basically discussing a problem where the system is at the state which might be known or unknown and on which we seek to perform an action which you see here noted in this block but before that there is the possibility to actually perform an observation and depending on the action and resulting state of the system then we are actually then we end up with a reward there this reward could also be a loss I will use the term reward in what I discuss here but obviously the two are related and so in solving this problem of decision-making or basically deciding upon a strategy we could look at it as a classical mark of decision let's say process where we have a system or an environment which is at a given state and we have an agent that seeks to take an action of which of the results of which he is actually uncertain and once this action is taken then he let's say receives this reward the market decision framework is actually quite flexible I have to say it of course makes the assumption that the state is only conditional on the present action and state and so it doesn't look back into the history it also can actually it is flexible in the sense that it can handle environments that are socast that are stationary or non-stationary it can handle let's say periodic or a periodic inspections but it makes a very important assumption the fact that the state is observable in the sense that once we have an observation we are certain about the state of the system in infrastructure management or structural health monitoring this is hardly ever the case when we make an observation then it also comes with some level of confidence and so instead what I will be discussing here is an approach where the state or our estimation of the condition of the system comes with a certain degree of confidence as I said before and this is described by the so-called belief and so we seek to determine the policy that which takes into account this belief in our state of the system as it progresses and which helps us decide on the actions to take for optimizing or maximizing our rewards or minimizing our costs these are the assumptions that I mentioned before so now the new thing is that the actions are uncertain and the observations are also do not come with certainty and if we would like to actually let me go here we would like to see it now in the system it would basically translate in the addition of this as we call it a mission probability which is basically the probability of the observation given the next the current state and the action that is taken as well as with this be which signifies the belief the fact that we do not know the state with certainty just in terms of history this partially observable Markov decision process approach is basically used in an extensive number of domain in other diverse domains such as the industry robotics troubleshooting marketing and so forth I have to say what I consider as a good example or parallel with the management infrastructure management problem is actually the example in robotics where quite commonly this approach is used for deciding on the optimal path that a robot needs to take given certain possible actions and under certain observations that might be available this is not too different from the problem that we are handling here in the end the pump to be framework basically comprises a couple of the these six components the system of states the set of actions the transition model which describes the evolution from one state to the other the set of discrete observations and the model of the observations and of course the reward function and basically our confidence in the state or condition of the system here it's actually very well connected to what was presented before maybe obtained using Bayes rule I will not stick to the details to the solution of the problem but I do want to say that essentially in figuring out the policy or path that ensures the maximum reward we basically here have to solve a recursive problem where we start from the end and so the problem is solved in terms of horizons where horizons are basically the number of decision steps that you have left so a horizon of zero means no decision step horizon of one means you have one decision step to take or an infinite horizon means you're looking in let's say the in a long term decision planning and so we solve in this recursive manner and we try to maximize the expected future reward which of course is connected to this belief state that I mentioned before and to the rewards of the previous time steps the problem can be expressed in two ways and there's two classes of methods that deal with the solution of this either in the continuous domain depending on the formulation of the problem that you have as you see here or you can also look at it in the discrete domain but basically what it boils down to is the fact that this value function is actually piecewise linear and convex and so in order to solve the problem in the end it is enough to figure out the set of characteristic vectors these a vectors or alpha vectors as they are called there are basically the gradients of this value function for different belief states and the solution algorithms that try to figure out this optimal policy basically try to figure out what these a vectors are and there's a number of algorithms that can be used depending on whether your problem is formulated in the discrete or the continuous domain recently we also presented an extension of existing algorithms for the continuous problem when you have transition or action mode models that are nonlinear in nature and so what I will present to you here as an application is based on this methodology but I would like to just simply go to the example if we assume that we have a structure or a component and we have a set of actions let's say of different intensity that you can take on this component here we have three action we have the possibility to do nothing a minor type of intervention such as maybe painting and then some more intensive type of interventions such as the replacement of the component and also you could choose to replace the system so there's different intensities to the set of actions and maybe you have observations that are again of different of heterogeneous nature either you can do nothing or you could visually inspect or you could perform perhaps a non-destructive type of evaluation then we can assume that we also have some sort of idea of what is the state or condition of the system but only an idea not a certainty about it here I am using an example with a one-dimensional model because it is a simple one but this does not mean however that you cannot include more components that to this state of the system it can be a vector of different indices so let us assume that you do have such an index that describes the state of the system described by this mean and variance here a good question is how do you get the index well you could actually get it from let's say strategies such as simple inspection even this would give you something like a categorization of the state in Switzerland it would be in five across five the grades and of course you could also give it excuse me you could also give it some sort of confidence there thereby associating some sort of standard deviation to this categorization which depends on the same experience of the inspector here I wanted to say that in the parallel action to you 14 0 6 there is a big discussion on what are actually indicators or indices that can be used and this would give maybe a source of let's say the metrics that one could use for these solutions but also you could use indicators that come from permanent monitoring solutions and in fact in this action there is a fact sheet already where we describe performance indicators from vibration based monitoring where again you get quantities such as this which is basically a mean and standard deviation that can be connected to the condition of the system so assuming you do have this quantity we can also assume you have these diverse type of observations that I mentioned before you see there is a more crude assumed observation here which could correspond to visual inspection in this case it's a simple good or bad situation if you're here your your system is in a good state here and if you're here of course it is a deteriorated state and there is an associated confidence with this decision or you could have a more refined and and more costly inspection approach such as non-destructive testing and of course there you will get a let's say a more refined assessment on the condition of your component or perhaps system again with some sort of confidence we also need to have the transition or action models as they are called and these are basically description of what is the effect of a specific action on to the state of your system here it is a conceptual example and we wanted to see how non-linear processes can be modeled but what we assume is that you you have a specific transition if you do nothing just a little system deteriorate if you have a minor intervention such as painting then maybe this would only lack acting the specific range of the state of your system might do nothing when you have significant deterioration and again might do no significant difference if your state is quite good and then we have a more severe intervention that could have an effect in a broader range of the states of your system and you could also choose to replace in which case of course you go back to the original state but of but all of these actions also come with their own and dedicated costs depending on their severity one might be more expensive than the other so the issue in the end is how to devise this policy planning and I mentioned before that here it is done in terms of horizons so here what I show you and the way we chose to depict it because this is also something that is a question for us how do you depict these things in a way that they are let's say that they are meaningful for the operator or the person who is to take the decision we chose to provide this in this sort of maps where in the x-axis you have the mean of the state of your system on the y-axis is the standard deviation those were the two variables in this case describing the state and in the different symbols here x cross and the circle you have the different kinds of observations that you can perform and the different color are the different types of actions that you could perform and this is conditioned on how many decision steps do you have left this plot is for only one horizon one decision step left and of course it is taken into account all of the possible combinations of observations that you need to take so in terms of solutions not that much different from the complexity discussed before here's the example for more horizons so this is with two decision steps left three decision steps left and so forth and you could also have an infinite horizon now the process for the decision-making in this simple example would be the following you have the system you have the condition indicator this gives you a mean and variance you go let us assume you have a number of two decision step left then you know you you're also we also have this plot which is let's say the policy plot and depending on where the index brings us then we can figure out or we are pointed to a specific action that we can undertake and a specific observation that we can perform next the observation will then tell us that after that action the state of the system is in this case I guess improved and it and you're pointed to a new let's say a point in your policy plan now you're if we assume that before we had two decision steps to make so two horizons to plan this would mean that the next in the next decision that you're asked to make you're left with the one horizon so you would go to the corresponding plot unless of course you want to change again you're depending on judgment if you want to change the number of decisions or horizons in which you want to plan but in this case it would be one horizon left and then the process simply continues in the same way of course this is an overly simplified let's say example but the methodologies actually can't be extended to account for higher dimensions of the states and also possibly the diverse diverse types of observations and diverse number of components to be inspected and so forth the idea is that if you were to let's say have this to look at the course of time where you're asked to inspect a specific component where maybe we can assume that this is the type of deterioration that the component under goes if you were to have a stochastic model of the transition of this deterioration then without any observation you would be let's say forced to take a decision based on this sort of rough gray graph once an observation is obtained depending also on the accuracy of the method every time you can redefine your confidence in the state of the system and accordingly if you judge this is necessary act at the point in time where it is needed the method comes with the number of considerations and actually there's research undergoing right now in order to extend this and look at what happens when the complexity increases when we want to retain more steps back in time so not just the simple mark of assumption and also how to deal with proper monitoring results that are continually let's say available and mean that you can update these maps on the basis of the inflow of new data there is some literature on this topic by the group of Mateo Pozzi and Costas Papa Costadino so and also some work of ours that you can refer to for further details and I would be happy to receive your questions thank you