 So, good morning and welcome from my side. So we basically just continue where we left off yesterday. So we're a little bit behind schedule but it's not critical. So this first hour or maybe until the coffee break we'll look at the influence diagrams which is basically nothing else but can be interpreted as nothing else but a Bayesian network where we add what we call decision nodes. So this is, we use here the square to show that and we also add utility nodes. The utility, in the example that we are looking at today or that Jochen gave yesterday, the utilities were just costs. Now there is a whole thing about utility theory and I don't want to go into this now. So for the moment we can just consider these utilities to be some expression of consequences. And in Jochen's example we use monetary consequences. I guess if there is some time, maybe also tomorrow or later today we can also discuss about what do we do or how do we consider non-monetary consequences. So obviously if we deal with structures often we have the case that consequences are safety related so potentially loss of human lives and there is at least a bit controversial to assign monetary values to those type of consequences. But for now we just accept that we can express our consequences by some measure, you can think of it as money or you can think of it as utility. Alright, now we come back to this example of Jochen a bit later. Now to start with I show this example that is also in the lecture notes that you have so I am going to rush through this a bit with the idea that if you want to study this yourself the example is completely documented in the lecture notes you can actually go afterwards and study it by yourself. So I have this problem and I introduce first the problem without any monitoring just to motivate it and then I'll show you how that develops how that goes into monitoring and how the influence diagram is used to represent that. A very simple problem I have to choose a foundation you want to build a house, this is on a slope and there is some uncertainty as to if the slope is actually stable or potentially moving. Now if you have a moving slope you want to have a deep foundation whereas with piles whereas if you have a stable slope a shallow foundation cheaper might be sufficient. Typical geotechnical problem and in geotechnics there is a lot of uncertainty and if we speak of inspection monitoring doing measurements geotechnics is really a field where this is done all the time. So but here at first we don't have a measurement we just have this case where we have to make a decision between deep foundation and shallow foundation and in this decision tree we have these as two branches and following those decisions we have a system state. Now this is not a temporal order necessarily because the slope is there before we make a decision but it's the order in which we kind of observe what will happen so that's why we put it in this way. So here is the decision and then we have a realization of the system and the consequences, the utilities here are associated with the combination of what we decide and how the system is behaving. So this is a very basic example for basic decision analysis what Sebastian presented yesterday. I'm just going to use that example later so I'm showing it. So you see here I have not put any numbers but I can also put numbers here so that we can calculate something. So we just assume at this cost you can multiply with 100,000 euros and you have the reasonable numbers for costs. So now if you want to make a decision what do we do? In Sebastian's show you just calculate expected costs or consequences, expected utility and you find the decision that gives according to the decision theory the higher expected utility or in this case the lower expected cost since utility would be minus cost. That's just an instance and this is a show here of the generic decision tree. So this is really what Sebastian showed yesterday. All right, good. The answer here if you look in the lecture notes you will see that in this case it's better to do a deep foundation because what you have to do is you have to multiply here. I'm thinking if I should let you do something later so what you do is you multiply the probability of a moving slope with the consequences associated here or the consequences. Okay, one by one. So if we want to figure out what is the expected consequences associated with the shallow foundation we take the costs here. Those are the 2-2 multiplied with the probability of this branch which is 0.7 so minus 1.4 plus the consequences here which are minus 10, minus 12 actually minus 12 multiplied with 0.3 0.36 and you sum them together. So minus... no, minus 3.6 so minus 3.6 so together we get there are minus 5 and when you do the same here you see that the consequences do not depend on what happens because once we choose the deep foundation no matter what happens the house is okay so in that case the cost is simply minus 4 so minus 5 expected cost minus 4 and the better choice here is to choose this lower expected cost. Good. And now let's continue and make an influence diagram of this problem. Obviously it's a very simple problem so completely I would use this decision tree here it's completely fine now how does the corresponding influence diagram look like in this example I'll give you the answer the influence diagram looks like this very simple indeed we have a a note for the decision in this case that's the foundation type we have a note for the random variables in this case only one the slope so is it stable or moving and we have a utility note and we use this diamond shaped note to show utilities those are the costs the interesting thing in this is again are the links how are things linked to each other now what we see here is both the foundation and the slope affect the consequences maybe one interesting is that there is no arrow here this means that this particular decision has no effect on our random variable our state of we call it state of nature also so the foundation does not change the slope in this example in a more general case however there could be a link from a decision to a random variable so if for example the action would be to stabilize the slope then there would be a link from the action to the slope so the links are in the same way as the base networks the links are the important thing that give the kind of how things are dependent now I already explained yesterday that the influence diagram is really important that you model it in a causal manner in principle again there are examples where you can do models that are not causal and are still okay as I showed yesterday for the base network but it's very easy to make a mistake and at least if you don't have a lot of experience it's not recommendable so always try to follow the causal flow so first thing second we can have of all this type of nodes these are the basic nodes but of all these nodes we can have one or multiple so we can have multiple decisions we can have multiple utility nodes and of course we can have multiple random variables so let me show you here so here is the base network we have multiple random variables and we have in this case one decision but we will of course have later more decisions because we have also decisions on nodes for example and we have we can easily have multiple utilities so we have a cost of a failure but we also have a cost of a design we have potentially a cost of doing tests so we can add different utilities nodes now we need so we know how the random variable parts we know already because this is just the rules from the base network that are still valid actually when we say that the influence diagram was developed in sort of parallel to the base network and when the base network was developed by people that come more from the artificial intelligence community the influence diagram was actually developed more by people that come from management theory background but these things kind of went in parallel and the way we use it today is consistent the base network can be directly extended to the influence diagram but we still need to understand the semantics of the rules for how to use these decision nodes here and the utility nodes what is the meaning of a link that goes from a decision node to a random variable like we have it here or from a decision node to a utility or from a random variable to a utility and so on utility I should mention that utility nodes cannot have children so utility nodes are always we call them terminal nodes so there is no link going out of a utility node there is no meaning to that links going into the utility nodes are quite easy to interpret so the utility node as we will see later is basically a table or if it is discreet it is a table, it can also be continuous but it is basically a function that assigns a utility or a cost in function of each of those parents so in this case we have two parents so each one has two states so there are four possible combination of parents and for each of those we have consequence so these are exactly those four combinations here this was minus 2 minus 12, minus 4, minus 4 the table that is describing this node contains exactly those four values then the links that come from a decision node to a random variable they can be interpreted in the same way as we already know when we have a link from a random variable to a random variable that means that this random variable here is defined conditional on the decision so in this case here the capacity is defined conditional on R and conditional on W in this special case here is just a function it is just a deterministic function more generally it can be also probabilistic finally this is not here, not here but there can also be links to a decision node this is an important part that we will that we will use an important feature what is the meaning what is the meaning of a link to a decision node somebody knows that? yes there can be both things you said two things one is that the outcome is known that is the first thing if I have a link from a random variable to a decision node it means that when I make this decision that is not the case I have a link here from R to W that would imply that when I make the decision on W I have full knowledge of R that would be great because that would mean that I know exactly the yield strength of the steel and I can I can optimize my W for the given value of R that is not the case in this example but if I were to make this link then that would mean that second case when we look at the sequential decision making is that if I have a link that goes from a decision to a decision node between two decisions I can have links the meaning is that there is a temporal order to the decisions and it's important to realize that in the decision in the influence diagram we have to specify a temporal order of decisions just to make an example let us assume that I will also make a decision on on what can I make a decision on okay on the tests that will come later but if you have a second decision here on the number of tests that will be a node here somewhere that points somewhere here inherently to the influence diagram in this in that case it's not clear I think which decision is taken first or is it I guess maybe okay let's let's a different example let's assume I can make a decision on Q for some reason I can choose the bridge so I can choose whether I make restrictions on traffic or not that would be a decision now it's not clear that there would be a link from a decision node here to Q now it's not clear which of those decisions is taken first and in the particular example also it wouldn't matter but the algorithms that are used to compute and need to know which of those two decisions is made first so we have to give an ordering of decisions and in order to tell the computer or also ourselves sometimes which of the decisions is taken first I would make a link from the decision that is taken first maybe this one to this here if you use the genie software I will show an example later if you use the software and you don't add this link the software will add a link by itself just assuming and some ordering because without the ordering it cannot make the computations which already points to one example of one type of decision problems that cannot be solved with the influence diagram directly and that is to find the optimal ordering of decisions so let's assume you can inspect multiple components in a bridge and say okay I'm going to inspect these components but should I inspect first this one or first this one that's actually a type of decision that cannot it can eventually be implemented in the influence diagram but it's actually quite complicated because you can't just put the decisions individually and then let the algorithm find the optimal ordering because it will not do that it requires you to say okay I make first the decision on inspecting the component number one and then I make a decision on inspecting component number two you can't let it decide what is better we come back to that also later for the moment we remember patient network semantics or rules stay the same utility nodes have no outgoing links and the in-going links mean that the cost here is a function of those in-going variables decision nodes can have children if they are random variables it means that these are defined conditional on the decision we take if it's a utility it's just cost is a function of that decision and you can have in-going links which means that the random variables that point to this node are known when I make the decision or other decisions are done previously temporal ordering okay please yes so that's some decision rule when you say decision rule you mean what exactly so that specifies what kind of action we should be taking remember what you call decision rule is what I also sometimes call decision rule but now I call it a policy so what you think is that whenever I observe A I'm doing B for example whenever I'm observing that my structure has some cracks I'm going to repair that would be a decision rule we don't implement that here because in principle the influence diagram just tries to search among all possible decision rules you just give options and maybe it comes clear when I show the implementation in the code you just say tell it okay you define the full problem and then you let it find the optimal policies or decision rules what you describe is already it's kind of something that helps us to solve the problem I mean here again is something that helps us to model the problem the solution to the problem the influence diagram does not really have or it's not attached to a specific way of solving the problem we still have to solve the problem and that can be computationally very costly the gene it solves the problem but it does not have a particularly efficient algorithm to solve the problem so if you have too many decisions it will stop working as we will see we'll come back to that okay so this is a kind of consequence of this there's also a you find also intellectual notes I think this graph here so these are three different type of situations and this is like a prior situation if you want this would be kind of a posterior situation the prior situation is this example we just saw looks like this in principle if you want to make it very general you also have a link from A to theta because A the decision could also be influencing example as I said in the slope or in the in the in other examples like here the W will affect the capacity so the but it's a situation where when I make the decision I say given information it means that I know the probabilistic description prior probabilistic description of my state of nature but I do not have any additional information we have to deal with what we know and we have to find the optimal decision that's the problem shown before the first problem is shown by Jochen yesterday now this here because of the semantics that we introduced implies that when I make the decision I know exactly what is the state of nature and there is no uncertainty in my problem because when I make the decision I know exactly what is the the random variable because we have only one random variable in this very simple scheme here so I know exactly what's going on and I can make a deterministic decision no need for probability and this is the situation that we have typically which is that in a posterior case which is that we have information but it's not deterministic so this would for example be a test so taking the slope example this is my slope now I'm geotechnical engineers are doing exactly that they are going and they do side tests they take some profiles and they try to figure out whether the slope is stable or not but this is not always based on directly measuring the quantity of interest but on some indirect observation and that indirect observation is what I have available to make my decision so in principle maybe we come back to this later here just to clarify the meaning of this arrow here this is all the theory that you need to know about inference diagrams no one thing I forgot to say last thing if you have multiple utility nodes as I said you can have easily multiple utility nodes because you have a cost of a failure, you have a cost of inspection then the assumption is that those utilities are additive they are independent and additive so it means that if you have a little bit experience with utility theory then you know what I mean by that essentially it means that we can just sum up the utilities coming from different utility nodes to calculate the total utility okay so this slide is not some of these slides are just copied from another lecture so this is something you know already and I don't have to repeat but I let this slide here anyway to remind us why we are here your information is not for free and this information set here that we assume to have comes at the cost and there is a decision associated like here with whether or not we should actually collect this information and this leads to what we want to do here, value of information optimization of inspection, monitoring could also be figuring out what is the appropriate level of an engineering model how much engineering we should how much we should spend on doing an analysis actually one of the first times I came across this problem was when I think I was still not sure if I was a PhD student or a master student but I wanted to do some work on I mentioned to somebody yesterday so it's a rock fall gallery so this is a structure that we have in the mountains basically it's like a concrete roof to protect from potentially falling rocks there's a rock cliff some rocks can be a common stable and we don't want them to fall on the highway so there is this rock fall gallery and there was a question whether that gallery is still ok or whether it needs to be improved or strengthened so I got this report which was made by a structural engineer expert in concrete structures and he wrote a thick report and made a very detailed analysis of this problem so he looked at the impact and he did a very detailed analysis it's not a simple problem obviously and he there's a lot of uncertainty associated with that so he tried to make a very good model and he wrote a very large report he made very detailed calculations and so on but he had of course to assume a load so he had to assume that the rock would fall so what did he do? he didn't really have good information from the geologists so he just assumed that there was one cubic meter rock falling 70 meters on that roof based on that assumption he made a very detailed analysis and I spent on how many months he spent on that but completely useless if I should say that because obviously a huge uncertainty on what type of rock will actually come how high it will fall it could easily be 3 cubic meters or 5 or just half and it could be falling from 50 meters or 70 meters or I don't know so making the model more accurate even if the uncertainty would be plus minus 100% doesn't make any sense at all there is no value in getting a better model because the uncertainty coming from the inputs on the load side is so large the decision on whether the structure is okay or not does not get anything any better by having a very detailed model on the resistant side that's also a kind of a value of information analysis that one should do and maybe save a few months of engineering time just to mention that okay and then that's just a more general thing now what I want you to do now is on a piece of paper go and take this example here this is a very general case and you can think of this as the slope problem here is a test for example the drilling some bore holes and can you extend this so that we can answer the question whether it makes sense to do a test or what type of test with what do I have to extend this influence diagram, how do I have to extend it so that I can answer this question should I do a test maybe what type of test should I do can we extend this from a posterior to a pre-posterior problem okay so I guess that by now we have either a solution or we have nothing but okay so how do we extend this influence diagram to consider the pre-posterior case so the optimization of the test itself from suggestion so there should be a choice note or decision note yes before here somewhere here you have a decision note that points to set yes that's it and the utility note that is a child to this decision note otherwise well we could have it without the utility note that would imply that this is for free and in that case as we learned yesterday is that it's never bad to have information so in that case it will always tell us okay get the information associated and that would then be represented like this so in the classical decision analysis at least the way it's used in civil engineering is that we have E standing for something that is an experiment so it's a decision on whether to collect information and A is this kind of action note which is decision on doing something but of course more generally it can be you know you can just write here test or whatever you want there's no need to restrict to that but that's why it's called E here and then there's the utility associated with E and there's the utility associated with this is to the same with what we decide to do and with what we what is the actual state of the system now we have these two utility notes and exactly as we as I said before it means that the total utility is just the sum of the two so there's a cost associated with the inspection and there's a or the test and the cost associated with what ultimately happens again if you look at these books mentioned yesterday by Benjamin Connell and so on they call it UT with T stands for terminal so this is the E stands the cost of the experiment alright so that's in theory quite straightforward of course as we will see later when we look at a specific case then it's not just TTA but we have more random variables and we might have multiple decisions also when we look tomorrow at sequential decisions we'll not have a decision only at one point in time but we have decisions at maybe every point in time so it can become much more complex but the basics are all here and you know yesterday Sebastian already showed this this is the corresponding decision tree and what you cannot really appreciate here is that this tree already has many many potentially at least many branches and we have only one random variable here we have only one action here and only one experiment E but let's assume we add additional decisions additional random variables this tree becomes larger and larger and becomes kind of it's difficult to see anything in that tree whereas here it's very concise this advantage is of course that here you can actually see the different states that can be taken you can see here the consequences that are you can directly see what is the consequence from a certain combination of decisions and outcomes which is here is hidden inside of the node so that's the kind of advantage of having this representation but yes very fast it becomes difficult to see but there is a second there is an additional advantage of this representation so this has information here that is not directly visible in this tree what else do I see from here that I cannot see from directly at least from the corresponding tree if I want to give evidence yes then so here so here we may still assume that we give no evidence so I will come back to that later here we still give no evidence of course we could then also add evidence okay now we observe this for a moment we are not putting evidence here we are just saying that when I get to make that decision I'm going to have that evidence but I don't yet have it but in principle I could add evidence here but that if I have evidence here it actually means that for example let's say I have evidence now on set it just means that I can just ignore all the evidence that corresponds to this evidence so that the evidence could actually also be looked at here but what is the advantage of this graph here sorry well it's also in the decision tree in the sense that the cost is all here and in principle when I make what should be at the end of each branch is the sum of ut and ue so if I choose this particular branch here it will put me here the cost associated with choosing the experiment so in that sense it's also there but what I wanted to say that here we can also see the causal or at least the kind of dependence among different variables for example what we see here is that A does not have theta which is something or in particular this becomes more evident here when we have a larger number of random variables if you would make a decision tree here it would just be with every additional node it would just be more branches but we will not see how those random variables and decisions are related in a causal manner you go you go with this to people who engineers who work on SHM who have no or limited background on decision analysis and optimization you can communicate this to them this are my random variables I have an action here and then we can add the nodes on the tests so similar to here or let's say here this is the problem and the test will tell me something about theta and that will influence my decision here I mean it's still not so easy to understand but at least here you can kind of see what happens how things are related to each other here you don't see that or you can see it if you look at the numbers that are written here but that will be very difficult so the advantage is that this gives me this shows me the structure of the problem I have an overview of the problem the only advantage of this representation is that it gives me the complete information but that's also the disadvantage because if that information is more than what fits on one page it will be too large so go back to this example and I'm showing you now the implementation for this example so the foundation concept the same as before prior problem as before now does it make sense to perform a test for deciding on the foundation type so what do I need additional information as opposed to what I already have here well if we compare this is the problem we have already looked at now the problem that we are trying to solve is this one so we need we need to define this part of the problem E well E is simple decision E has no parent is just do the test or not then we need this note here that will be the test or not doing it cause of not doing it is zero cause of doing it is something that we need to know and finally we need this note we need this note set set is the outcome of the test now this is conditional on E and on theta if E if the decision is to not inspect or not do a test the outcome is simply no outcome or nothing nothing observed irrespective what is theta if the decision here is yes we do test then set will be the outcome of the experiment the test conditional on what is the state of my slope how does that look in this example here this is the information so if I do a test then I can have three possible outcomes I can observe that there are no movements I can observe that there are slight movements or there are strong movements these are the conditional probability of observing these things given that this is my state if I have a stable slope probability is 0.9 that I do not observe anything but there is still a 0. there is 1% probability of observing strong movement there is some kind of measurement error if you want if I have a moving slope I still have a probability that 30% that I observe nothing maybe because in this particular period of the measurement there is no movement but that doesn't mean that there is no movement in general so the observation is not perfect if it was perfect I would have probability 1 here and probability 1 here and probability 0s everywhere else that would be a perfect observation but the reality is not perfect so I have an imperfect observation and I need this information here to describe the quality of my information of my test and when they come back to that later when I speak about generally how to model the quality of information this is nothing else but a likelihood function so if you know about Bayesian analysis or if you just know about general statistics maximum likelihood estimation this is a likelihood function where we give a probability of a certain outcome, test result given a certain state of the parameters that we want to know about given theta what I have said ok so this is the information we need and with this we can either go and do this is what Sebastian showed yesterday basically solve the decision tree the decision tree looks like this here in the lecture notes is the solution so you can find it there if you are interested or you can use the influence diagram to solve it and I am doing the second of course so we are using the influence diagram and I am showing you how for such a simple example how easy that is but solving this is a kind of solving and needs some work and also needs some it's not super difficult but with the influence diagram it becomes very simple so again I am using this software here and I have here the prior problem the prior problem is foundation yes or what type of foundation slope and then the utility and this is defined exactly as I said so if the foundation decision has just two options shallow and deep that nothing else specify no probability of course because that's just decision the slope has just these two probabilities 0.7 of stable 0.3 moving and the utility has these four values okay now it's before it was minus 200 so it's minus 200 for shallow foundation minus 400 for a deep foundation if the slope is stable and if the slope is moving the deep foundation is still minus 400 but the shallow foundation will be minus 1200 because first I pay 200 to build it and then I have 1000 in damages yes so total is minus 1200 and that's it and with this we can just go and run it so and then we check what it tells us it tells us the value for the decision node the value for the decision node is the expected utility so tells us that if I choose shallow foundation expected utility is minus 5 or minus 500 so the optimal choice is to choose the deep foundation well that's clear? that's of course also rather trivial but that's basically now okay let me maybe I have already the solution but maybe I will show I will now add it here manually so that you can see the process so now I am going to add the possibility of the test so how is it called? test outcome this is the test outcome and the test outcome had what happened now? I did something stupid I added this was not sorry this was not a random variable this was an object so this is what I said yesterday you can add an object here which is a Bayesian network in itself but this is not what I wanted so test outcome and let's first assume that we actually do the test no cost we just do the test the test outcome has no possible states no movement slight movement and the third one was strong movements this was what we could observe okay now it looks like this because there is no parent to it but when I add here with this thing I added this as a parent I can specify the outcome I can specify here the outcome conditional on the state I have to recall the numbers so where are we here where are we 0.9 0.09 0.01 0.9 0.09 0.01 and if it's moving it was 0.2 I think 0.3 0.5 0.2 0.3 0.5 0.2 okay alright so that's how it is and now there is no effect on the decision let's run it quickly and if we do it like this the decision is still the same because I'm assuming that I don't know the outcome when I do the foundation design so there's no benefit of having this test outcome if I want I can check what is the outcome but there's no meaning now okay maybe I should mention this because we spoke of evidence before so if I for example okay now I'm actually in the posterior case like we had yesterday and I have already observed something I have done the test and I have an outcome so then I can give evidence here set evidence for example we have observed that there is a strong movement I'm fixing this I'm updating and now I can check again the expected utility so you see that the optimal decision is obviously still the deep foundation the cost of this is the same but the expected utility associated with the shallow foundation has now gone the cost has gone up a lot because if we have observed the strong movement that indicates that it's rather likely that there is a problem and we should not build the shallow foundation but once I observe evidence the problem I mean just give it as evidence in the Bayesian network and it runs the analysis conditional on that evidence that's like forward but the posterior case is where we have not yet actually observed the evidence we are still in the situation where we are thinking of whether we should gather the evidence so I'm going to not do this, clear evidence instead I'm going to add an arrow from test outcome to foundation now here I don't need to specify the foundation I don't need to specify anything here in addition it's still just the two options that I have and this arrow for this reason is also dashed arrow so it is an arrow that represents information that flows but now when I do the optimization what does it do now it tells me that test outcome is a matrix yes now the optimal decision is given in function of the test outcome so if I and it tells me that if the test outcome is a bit small but if the test outcome is no movement I should build the shallow foundation we dissociated utility minus expected utility minus 325 if the test outcome is a slight or strong movement I should build the deep foundation often that's kind of what we would expect yes but it depends on the numbers we put there but that's directly the result of a posterior analysis but for all possible outcomes that we could have yes and now we're going to add the pre-posterior case so I want to actually figure out whether the test makes sense so I'm putting here a note that says test question mark and this has two possible choices or alternatives I can either test I say yes or I don't test no well this is associated with the cost so I put here this utility or value note so this is the cost of the test UE we called it and I'm going to make a link here and I have here to give the cost of the test so I don't remember what that was and I think it doesn't say here doesn't say here doesn't say anywhere but I think maybe here yes 0.2 ok so if I do a test it costs me 0.2 and this is minus no test is for free that's what you did and then I have a link from test to test outcome ok now what am I missing what have I not defined yet yes exactly you see that the software is smart it shows this arrow as gray which means that this arrow as of yet has no has no effect because I have not specified what happens what is the effect of this decision on this so I have to change this table here which is easy to do so what it does is when I add the link it just copies the table from before yes ok first of all the ordering is not convenient so let's change the ordering so what it did was it said ok if I do a test or not it doesn't change the probabilities of course it does change the probabilities so if I do a test I have exactly what I had before that's ok if I don't do a test then what should I have as outcome uniform although it sounds at first reasonable but I'm afraid no actually when you have don't do a test what do you observe you observe nothing so we have to actually add a test as a state no observation and in case that I don't do a test yes oops this is what I'm going to observe with probability one yeah that's how I it's kind of a trick this is how I include that so what the software does or the card the algorithm that is behind is that it's if you want implicitly it's doing a Bayesian updating here it's still doing a Bayesian updating but this probabilities will just result in the prior distribution so it will not change anything because no matter what is the state I always observe the same thing so that's how we do it and now I think we can run technical problem now we can run the whole thing okay we see here now the choice of the foundation still is a matrix now it's even a bigger matrix see here I'm going to go back to that what we actually want to know is the test whether we should do a test or not does it make sense and the answer is yes it makes sense we have a lower expected cost if we do the test and if we don't do the test this is the full preposterior analysis the value of information now is a bit hidden because we have here the cost of the test so basically the difference between these two things is what I call the net value of information so the net value of information already taking into account that we have to pay for the information as well so this is the cost if you want to know the value of information without that then you would have to put this to zero anyway it tells us that we should do the test and that the expected utility is significantly lower if I do the test and this is based on the assumption that when we do the when we choose the foundation we always pick of course the optimal choice and that means that the optimal choice would be now I'm opening this table here again value so this decision now depends on the not only on the test outcome but it depends also on the previous decision you see some of those combinations of test outcomes and the test decision are impossible so with test I cannot observe a slight movement so when I don't do a test I can just have no observation and in that case the utilities are exactly what we know minus 5 and minus 4 oh I made a mistake slight mistake but the utility here should have been defined as minus 0. minus 20 so so they still the same the prior decision in a way is still the same when we do the posterior it tells us that if I find in the test that there is no movement then I should choose the shallow foundation because that gives me a lower utility lower cost higher utility and otherwise I'm still choosing the deep foundation now that has become more expensive because I have to I added the cost of 20 for the test itself so in case I find some movements I actually end up paying more because I have to pay also the test so in a way after the fact so given that information I have no doubt that information I'm actually ending up paying more but initially because there is quite a high probability that I will actually observe no movement it makes sense to do the test but in retrospect you might end up paying more and still doing the deep foundation this is something we keep in mind so for 2 out of the 3 outcomes it turns out that I'm ending up paying more only in the case of no movement observed I get to pay less but that justifies the test ok anyway so you can solve this the most basic type of problem that we could have from creep posterior analysis you can solve it also with the decision tree and in the lecture notes you find both the normal form of the analysis but what I wanted to show here is that with the influence diagram it's straightforward to model and for these simple problems the software can do the analysis what I will discuss then tomorrow is that the software however is not going to be very efficient once we end up having many decisions to make let's assume we have a problem of inspection or monitoring whatever predictive maintenance these type of problems where we have to make decisions at every time interval for example every year we have to make a decision whether we should inspect or not or every year we have to make a decision on whether we should do some maintenance or not we end up having for example 50 decisions and what happens is that the software will try to do exactly what it does here namely that it will construct the last decision to be made will be conditional on the 49 previous decisions and that table again comes 2 to the power of 50 so that table would become impossibly large and therefore the software will just stop producing a result so we need if we have many decisions we need more efficient algorithms and I think at least in genie it's not implemented we need and we can actually only solve those problems actually we can solve them also in the general case but only approximately so we can either solve them only for special cases or only approximately but for simple problems you can use this software and you see that it works very intuitively and simple and for more difficult problems you can still use the influence diagram to represent your problem you just need to figure out a more effective way of solving the problem ok are there questions up to this point can be used as a modeling tool as well as a computational tool so what's the distinction between the two yes actually the influence diagram can be used as a modeling tool it's not really a computational tool so underneath it I mean the basically what at least what the genie here is doing is more or less doing exactly the same computations that you would do if you solve the decision tree by hand since this is a very small decision tree of course the computer has no problem to just very quickly solve these branches but it does not actually it does not actually have as this is not as well aware of unless they changed it very efficient ways of solving it the influence diagram does not provide me with an efficient way of solving the problem this is different for the in the Bayesian network the fact that we have this conditional independence gives rise to all these different algorithms actually if you look in the software here it says algorithms those are all different algorithms for solving the Bayesian network problem not the influence diagram problem so these are different algorithms some of them likely with sampling I mentioned yesterday these are the exact that I also showed yesterday those are algorithms that solve the Bayesian network problem to solve the influence diagram it basically just all the branches of the decision he kind of creates the decision tree and checks all the branches that's not efficient the only kind of some of the only special algorithms for solving influence diagrams in a way is basically what we call the Markovian decision processes or partially observable Markovian decision processes and I will mention that yet tomorrow so we'll speak more tomorrow a bit about how we can solve potentially solve larger problems so this really just for me is just a tool for modeling for understanding the problem for putting on paper to understand what is the information I need to specify but not to solve other questions or comments yes please so we have the results there are more or less similar so what to do if there are little better results not to do but I don't understand it's clear it's a very practical question but which I mean this is based on the clan also presented yesterday the classical decision analysis and it's based on the assumption that you choose your decision according to the expected utility criterion so whatever decision gives you the highest expected utility is the one you pick in practice of course we know that in engineering projects we have two options and one gives me a cost of total cost of say 400,000 the other one 410,000 it's almost the same now strictly if all my criteria that I have for decision are included in my analysis I should still pick the one that just gives me the highest utility even if the difference is very minor in practice we almost never include all the criterion in our formal analysis so for example maybe here we just include cost but then maybe there is also the fact that deeper foundation takes more time or actually doing a test takes more time and maybe that's not included in here then we would say okay but if you are smart we would also include the time as an additional cost here so then maybe it's already factored in but then we say for some reason maybe the owner here of the house is environmentally conscious and he doesn't want to have a big I don't know there can be additional reasons that are things that are not included as explicit what we call attributes in the utility analysis so we don't include them and say because aesthetics for example one option is nicer than the other and it's not included in these analysis and if they turn out to be almost the same cost we would say that okay we just pick the one that is more beautiful so this is not something related to the of course influence diagram but more to the general decision analysis which is at the end it's just a tool for informing decisions we are not bound to strictly relate to that and if we and in all the practical problems that I have seen it's that okay this gives me a first we try to include the main criteria here it gives me a result and if we turn out to be almost the same we typically in other criteria that might also play a role and then make a decision based on that we are not bound to do that decision however what we have to keep in mind is that when we do this kind of analysis and we have multiple decisions when we sort of look at the decision here is based on I mean factors in the decision that we do here automatically assumes that here we will do a decision according to the decision analysis so but for the first decision we can just afterwards look at the result and then pick I mean we are not bound by this it's just a tool yes how do you actually get to know the certainty of the probability you are attaching to those variables before doing the first like do you need to pass both criteria to account yes actually this is of course a good question because it's challenging or sometimes can be challenging to get those probabilities it's not so much related to whether it's before or after the test because even after the test you still don't know these probabilities after you do the test the test tells you there is a slight movement you still don't know I mean it doesn't tell you anything about the quality that you have to come up with based on information other than what you have on the specific side and we will discuss I mean it's the question how do I model the quality of my information it's actually what you want to speak about afterwards but just to quickly give some first answer I mean here we speak of geotechnical problems one good way there is actually to try to speak with experts they have a lot of experience on this thing and then they say okay when we look at this table that we need here let's say okay you know what is if the slope is stable then we could say okay so there is a probability that we have a slight or strong movement observed that is actually a measurement error because the slope is actually not moving so if you see something moving then it means there is a measurement error and that's something that we could try to understand from maybe other type other measurements or again the expert might know that this is not completely exact and this device might show some noise and this noise is like this the other thing here is if the slope is actually moving what is the probability that we might not observe it or might observe only a slight movement that's more related to the actual it's not really a measurement error maybe but it's more like maybe it's moving but not all the time maybe there is an estimate that even if it's moving 30% of the time it will not be moving at all or 50% but just a little bit and so there might be related more to the geology again you have to ask the expert, the geotechnical expert what do you think there could be and it is numbers I mean whether it's 0.5 or 0.6 probably nobody can tell you it's not the rocket science here in other cases we might have data that we can use I will show you later so there might be other ways of figuring this out but in this example here where the geotechnical is a lot related to expert knowledge but it's better to do that for the pre-posterior analysis you have to do it before you do the test but even if you are not doing a pre-posterior analysis let's say you are just doing a posterior analysis so it has been decided to do the test and now we want to do the posterior calculations it's recommendable to do this type of analysis before the test is done because engineers are thinking very deterministically not most of us here try to think more probabilistically but civil engineers are typically deterministic thinking people and other engineers even more so so once they observe and there is no movement they kind of start to believe that this is really the deterministic truth before they observe something it's more easy to get and to make these kind of statements and say ok maybe I might observe this it might not be exactly but once they have observed they just tend to believe that that's the truth and that's it in this case it's probably ok because we saw that the optimal decision when you observe nothing you should build the shallow foundation and if you observe a movement you should build the deep foundation so that the deterministic decision would be actually ok here whenever there is a movement to make a deep foundation but it's not always just like that you also have a question or comment on quantities or how many you mean are continuous good question to be honest I don't know I mean I'm not using the software for my work so I use it for teaching these kind of things I mean I know that it can handle there is a way of handling continuous random variables but I'm not sure if you can actually handle influence diagrams with continuous random variables so it might be that so you can do continuous patient networks to some degree but I'm not sure about whether you can do some influence diagrams with continuous random variables you have to find it out if you find out just let me know there is also just one software there are multiple tools there are other tools in python in matlab and so on the thing is that for the actual problems I mean so I'm using these as I said I'm modeling tools and so on and then we when we actually try to solve real problems particularly if they come with the sequential problems and so on anyway they know that this software cannot do it plus typically it's easier to just hard code at least for us it's easier to just hard code something in matlab or in python or something like that rather than using this existing software because if you think of this dynamic page network that I showed yesterday this is the type of models that we then often use for and there we have we we have a regular structure that's relatively easy to hard code and actually you can be more efficient than its generic algorithms so I haven't really I know that some of my students have used things for continuous page networks here but I'm not I don't know about continuous influence diagrams I mean this software can do many things that I'm not showing here so if you can just explore it yourself if this is of interest to you what time is the coffee break? no is it already? or is it in half an hour? 10 10 okay then I don't want to stand between you and your coffee break so we make a break I'm thinking okay let's open this good question so the question is what do you suggest is to put here just a uniform distribution yeah 0.3 0.3 no I will give 98% I think it will give the right result also because I mean the point is when you have no test it means that whatever is the reality the quantity you want to learn about whatever it is you can't see it in the test result so as long as and I think it doesn't even have to be uniform I think as long as the probabilities here and here are exactly the same so it could also be 0.8, 0.8, 0.2 0.2 as long as there is no difference between those two columns if you do the Bayesian analysis it will just tell you that the posterior is equal to the prior because there is no information whatsoever so what you observe does not actually dependant that's the whole thing actually 100% correct because if the distribution of the test result is independent of the quantity of interest it means that independent you learn nothing so yes, that was a bit too harsh I say this long time this has the so this is a bit more let's say clear otherwise you get some results that will tell you that there is a certain probability of finding strong movements even if you do no test the same result other questions or comments? no so I just want to come to an end I mean we'll get back to the influence diagram and maybe we do it now because we're still awake from the coffee so let's go back to the example of yesterday and now we'll think of the preposterior problem from Jochen yesterday and we try to complete this graph now so what do we need in order to to represent Jochen's problem here what additional nodes and links maybe I'll do them in red or blue okay so first a decision on the number of experiments so that was yes now this is you see already a bit tricky here okay the way I represent it of course we could also we could also do it by having n and then x bar but I wanted it to be more general so that it doesn't have to be all normal so the decision on the number of experiments is a bit it's not so easy to represent here but nevertheless we can do it first of all you have to forget about these E's because now in the preposterior analysis we don't have the evidence yet we potentially get it but we don't yet have it there is no E here so we have a decision on this let's call it n or how do you have a number for that n it was so it was n and the way we how could we introduce that here I can see a way to do that but maybe again this is a modeling task and there's not necessarily just one solution there could be possible most likely different ways of doing it but I want to n is the number of tests that we do I have to somehow connect it to here how could I do that how could I do that like this and so on yes okay and now it means that so how do I specify n is just 0, 1, 2, 3, 4 and so on these are the different options here so if I would do this in in genie I would give here 0, 1, 2, 3, 4 and so on and by the way this you can implement in genie you can this problem you can one possibility is to discretize those random variables that works for sure and this problem can be discretized or maybe it's possible but this you would have to figure out to do these continuous random variables but for sure in this case we discretized anyway so if we put in genie we would put here 0, 1, 2 up to a maximum number that we would consider and then we have to specify now the conditional probability table of this which is this is the what we have to without that it's just the likelihood and in this case the normal distribution with mean value r and standard deviation on the batch but now it's also conditional on n and how do I introduce this condition on n I think so basically if I don't have n here and I just had it like we had it before then x1 this leads actually over to the next lecture in a way so the the conditional distribution of x1 given mr would be the normal distribution with mean value mr and standard deviation 20 it was the within batch variability so because the mean value is the mean value of the batch and the standard deviation is the within batch variability so without this it would just be normal distribution with mean value this now I have it but now it needs to be conditional on mr and on n so let's say the number is 3 or 0 or 1 or 2 somehow this has to be modified here standard deviation nope the standard deviation will not change because n just tells us whether this test is actually done or not the quality of the information is stays the same what it does change is that it should be more like we had before that when n is let's take the first test when is this first test done whenever n is larger than 1 larger equal 1 we we can do the same thing as we do here so if n is 0 then the test outcome here is like 0 or no outcome is when it's 1 or more then the test outcome is this distribution that we had the conditional distribution of x1 given mr which is and the same here so here whenever n is larger equal then we have this distribution otherwise no outcome and in this way if you want to consider many large number of possible tests it becomes a bit large here but this does not add any computational problems for the Bayesian network maybe if you want to manually include 100 nodes with genies it's a bit annoying but in terms of computation it's not an issue just speak but for this consideration information actually comes only in terms of n and x bar in the normal distribution case in this case this is more general but in the case we discussed so we modify mr by knowledge about n so there has to be a direct relation by a direct arc in n and mr and only x bar is also modifying mr so one single node that represents x bar area x bar and decisional n would be enough so the decisional n would go directly into mr and n would also go into x bar and then there is an arc between mr and x bar and the arc between n and x bar we only need for the case of n equal to 0 sorry the arc between n and x bar we only need for the case 0 because otherwise no that's not true because this distribution depends on the number of tests that you do it does to specify this you have to specify the distribution of x bar conditional on mr and the standard deviation of this decreases with n let's put this to infinite then x bar is equal to mr and if you have yes no we do need it we will have evidence but we still have to specify here the conditional distribution and the conditional distribution of x bar between mr is a function of n because the standard deviation of this decreases with the square root of n we always have to specify this distribution you can put evidence but you still have to specify this distribution because you specify this note otherwise actually if you don't do that you would imply that n and n does not have any effect so whether this comes from one test the number of tests is the same but it's obviously not because this would be the normal distribution with mean value mr and standard deviation would then be mr divided by square root of sigma no 20 20 divided by square root of n so this here would be a normal distribution mean value mr so it's conditional mr and standard deviation is square root of n times 20 no space now 20 times this 20 times this is the standard deviation and then that will give me the actually one divided by n yes and then the rest the network can stay now so we can do it like this actually in this case so it's most that's easier to implement it's not normal Bayesian updating in world's drama the information is always coming into pecs but here it's only its bar and then we have to watch in our case it would be the standard standard deviation but always for the case of the normal distribution of course but actually for the computation it doesn't matter so of course we do it by hand in genius but if you program it in MATLAB you just include as many as you want and for the computation it doesn't matter so that's not the problem here so we had this alright we can also we can also go to the special case since it's a bit more requires less space so we had here r we had here the value somewhere then we had the general capacity and then we had here the performance of the failure and here was the Q so we have specified this this is I think trivial so this is a known distribution um wait what did we do I'm sorry what did we do I say because what did I do I'm not saying there should not be a link here this doesn't make sense because the number of tests does not affect the mean value of r this would imply that the number of tests has an effect on the mean value of r but it doesn't the number of observations double prime not the prior just the posterior so m, r is a normal distribution but depending on whether we have the prior or posterior okay maybe this point is good because you mentioned this because I'm standing with my point that there should not be no link here what you're saying is that when I observe these these changes that's basically what you're saying so once I observe something here I fix the number of n then I observe something here that will then affect the distribution of m, r that's basically what your statement is which is true but this is actually if I put here evidence this will propagate to here and we get a new distribution of m, r but if I would have a link here it would imply that the mean value of r the actual mean value of r would depend somehow on the test and the test does not change it changes our knowledge about it but we have to be careful so these arrows don't represent our knowledge our knowledge is represented by the probability distributions and if we put evidence somewhere those distributions if we put evidence here those distribution rules tell us that this one this one and this one will all be updated this one will not be updated but if I put here evidence then I will update this distribution will be updated hence this and this and this will be updated that's what happens I don't but I don't have to add looking again in this diagnostic idea that I observe this hence I should make an arrow like this but that's not the case yes we don't know anything that follows normally this is by means of manipulation exactly we know the characteristic conditional random variable only conditional on an X bar on an observation of X bar we put here a number yes this number in the standard deviation of the mean variable yes but together with n yes but when you look at the equation it's not enough to only have information about X bar also but what you have to realize is that this is the nice thing about the Bayesian network because this is automatically taken into account no but I can put here evidence if I put here evidence when I have evidence here this become connected so they become actually dependent and that's exactly what happens when you change n you get a different, it just changes because of exactly this fact, this thing it will actually be automatically like this because when you change n that's why the distribution of this here depends on n so I'm observing a value of I'm observing a value of 410 if that comes from only one test the effect on this will be minor as compared to when this comes from 10 tests that's because here the standard deviation it's like the table where we discretize it but it could be also a continuous rub so we have for all different observations of X we get these different distributions so how often do you specify distribution only based on information of X bar but wait if you think of the distribution analysis, forget about the rest just look at this problem now and we do distribution analysis and we want to update this here this is what you also have done and what we have here this here is the likelihood function this here is exactly the likelihood function so this here the input that goes to this node is the probability of X bar given MR and given N but N is in a way always fixed so what we have but for even value this is a function of MR and N so what is the input here this thing here is the distribution of X bar given MR and also given N N so it's depending on N but for fixed value if you want to fix N this is the likelihood function so X bar given MR comma N and this is the likelihood function of MR and it depends on N so when I do the Bayesian updating and I change the N because this function changes my posterior changes and this is exactly what you also use in your implicitly use in your normal, in your analysis it's exactly this function so that's why this information on this N actually changes the posterior distribution of this so this is exactly when I introduced yesterday the Bayesian network it's kind of logical and easy but then you realize that actually when you go and I mean I have a lot of experience, I've been using these things for more than 10 years so it's easy but easy to make a mistake because you confuse or you can mistake the way information flows from the way things are related to each other in a kind of causal sense and what you really try to strictly think of okay the arrow here doesn't mean it really implies that N would change MR and that would mean that the test will change your physical quantity unless you have a destructive test that should not, if you have a non-destructive test that should not happen so sometimes it's better to think less actually but difficult to think less and because you see I made a mistake I put this arrow because I didn't think about it so I realized so now this is the thing and now we still need these are the two decisions N and W and now we need the utilities where do we have utilities where is utility note observation here somewhere or here a cost of the test we call it C cost of observation cost of failure CF and design cost how do you call the design cost often there was no variable for it construction cost C construction okay hope you like this so I think that should be it that's it yeah please here yes it would not give the same result because the capacity is a function or it is dependent both on R which is the strength of the material here and W but at least in this example it depends only on W it does not depend on R but if you put it here as a child of here you can't separate the effect of those two so you would have the mean value of R yes so in this case they will double work okay so this is something that this I would say okay if you want for practical purposes you could actually implement that in the programming genie the only problem is you would have to discretize those random variables and that's not so you can discretize it in many states here you could discretize it in 100 states you don't do it manually I hope but if you could discretize this in 100 states there would not be a big problem in terms of computation your size of the conditional property tables that would be an option another option would be to another option is to use you can do this with discretization you can use also stochastic optimization algorithm which I will discuss tomorrow at least quickly discuss tomorrow what that one could use or you go and you take this and basically solve the problem by using the Bayesian network only to compute the probabilities so you basically calculate for given values of n and for given values of w if you fix n and w then you can once you fix those two you can just run the Bayesian network analysis here it gives you a probability of a failure which you get the corresponding risk and then you repeat these analysis for different values of w and n so basically you're using the Bayesian network here and for this you can use all these different Bayesian network algorithms and if you could do offline for example I could use this likelihood remember yesterday I showed you the code for this likelihood weighting algorithm I showed you the lines of code so I could use that if I want or any other algorithm that I have in a Python, Matlab whatever and then just do it for different values of n and w and run an optimization over n and w of course this is a mixed, discrete, continuous optimization so it's not so straightforward but just have two parameters which basically or you can also obviously since this is an analytical solution here you can use Johan's analytical solution to solve whatever we do, whatever we represent here with the Bayesian network you do that for different values of n and w this is basically what we saw yesterday and then get the optimal solution so this is an example where there is a relatively simple analytical solution and you don't get it from here what you get from here in my view is that one actually can better understand this relation one can hopefully better understand what is actually the meaning of these tests which I think was not so straightforward to understand initially but the computations are more easily done with something else the Bayesian network there is a Bayesian, I mean the one that I am aware of is called I mean you can just google MATLAB Bayesian network so I am there, what is it called MBNT or something like this MATLAB MATLAB yes I meant BNT it is the one yes it is interestingly quite an old code from 2003 or something like this it was done by this guy from Murphy that is in Berkeley and since then I am not aware that there is something else in MATLAB I know in Python there is also a number of them and then there is also you can find in R I think it is so there is a number of codes I am not the biggest expert on all these codes so I have used the MATLAB one which works quite well but I am not so familiar with other codes it is not like I could recommend you one that I always use but I am also not checking all the time I mean when I was checking 5 or 10 years ago or so more there was not much more things that were much better than this Bayesnet toolbox but there is also a lot of development in the community mostly in pythons so I am not sure if there is something better now okay what? what? in genie? there is a I already said before I am not a big expert in genie because I use it only for classes and I have not I know the students of mine have used it for continuous modeling continuous nodes and this is a normal distribution typically it should be possible to input I am not sure how it works I know that in yugin which is another software maybe this you can do in yugin yugin is another software that is commercial but there is a free version with limited capabilities there you can input the normal distributions influence diagrams continuous nodes also how is it that it tends to crash okay the thing is that here because we have this special case of everything in gaussian also in genie you can actually do exact inference also with gaussian so the genie I am not sure about the genie here yugin at least has this it is smart enough to realize that if everything is gaussian and linear as we have it here that there is an exact solution just calculates at least the base network calculates it exactly and I would assume that this also translates to the influence diagram that it can actually this particular case here use the fact that the base network can be solved exactly because everything is gaussian and linear so that it can solve so there at least in yugin you can input this as a conditional normal distribution and in genie you can also do it but I am not the biggest expert on this so maybe if you spend one hour today or two maybe you can figure it out and tell us tomorrow but the thing is this, when you are a PhD student when I was PhD student I was coding a lot and doing all kind of stuff and then when I was a postdoc I was still coding a lot and I was still coding quite a lot but then I had more and more and more students and now I have no time to a little time to actually sit down and do some coding so when I have an idea I just tell my students and they implement it so I know all the concepts but I have a very poor in implementing them or I always use the things I already know I use the tools that I already know so this tool I just use for teaching I limit myself to the simple type of problems but I mean there is a way of doing it somehow just don't ask me to show you now how creation, decision chance is the least I think it is to do with the equation you have to put the equation node and then somehow you can put the conditional distributions as distributions other question about this case the same last way about the mean value of R I am not sure I understand what you are implying yes you think that here in the Johans model the assumption is that if there is a failure there is a certain cost there is only one type of failure failure or no failure that is the assumption of course you could think of a case where you say the failure consequence do not depend just on whether you have failure or not but also on whether what was your load at the time of failure it could be reasonable to say on the bridge that the more load there is the higher the consequences because it means that there are more cars on the bridge or more trucks on the bridge which we could include how would we then change the network here if we would do that if we said that the consequences somehow depend also on the magnitude of the load we just have to put an arc here from arc from q to cf so if we do that then we can define the consequences in function of whether or not we have failure and what was the value of q at the time of failure so we could define that but in this example it is not defined like this but you could also consider monitoring the load there are many possible extensions that one can do okay yes alright so I will just do something very fast okay 10 more minutes 5 more minutes on this diagram and then I will speak maybe half an hour on this likelihood function so Sebastian introduced most of you know already this general idea of value of perfect information of information what we want to compute but there are also these intermediate concepts and I don't know if many of you are familiar with those it is value of perfect information value of partial perfect information and conditional value of information who has heard of value of perfect information somewhere I would expect this and that is actually quite a useful concept I find also in explaining this idea of value of information actually I will not discuss now here this value of partial perfect information is related to this so the value and now what we see here is this what is the value of perfect information it is basically the difference between the left side and the right side okay and the left side is the prior vision problem now let's go back to this problem of the slope and this was the prior problem now that we had this discussion before or the question before how do I specify my likelihood function the geotechnical engineer has to somehow specify the likelihood function it is challenging for him or her well prior to do that we can just calculate the value it is an expected value those are all expected values the value of perfect information we say hypothetically let us assume we have the possibility of making a perfect test what is the value that such a perfect test would have and that is very simple to at least in the Bayesian influence diagram what we have to consider is just saying that at the time of making the decision we know we know theta and it means that we can maybe make the example of this example so we assume hypothetically that we will know exactly the value of theta we still don't know what it is but you have to assume that when we make the decision we will always make the perfect decision so if we know that the slope is moving we will make a deep foundation and if it is not moving we will make the shallow foundation and what would the expected cost of that be compared to the expected cost of this and this difference is the value of perfect information and the good thing is this is an upper bound so it is an upper bound any information we collect our infinite amount of data we can collect can never give us more value than this upper bound and it is helpful because it gives us an indication what we potentially might maximally spend on collecting information and it is also helpful because it shows that that upper bound does not depend at all on the type of monitoring system we have it is purely decided by the decision problem by how much uncertainty we have and what is the decision and these consequences so we can quickly do it and check it for this problem of we had here so I am going back I am going back and I delete the posterior problem so we are back at the prior problem we remember that without additional information we should choose the deep foundation and it costs 400 that is the optimal choice now what happens if we have this well the problem is now that Gini itself does not directly give us the answer here he tells us that if we observe if we know that the slope is stable we should choose the shallow foundation which is something obviously we will also know ourselves and if it is moving we should choose the deep foundation we have to do the calculations by hand but it is not very difficult calculation because we also know that the probability of having a shallow foundation sorry a stable slope this probability is 0.7 so the expected cost or utility I am doing utility now the expected utility given that I have a perfect information how to write ok I do not want to write ok I want to do it a bit far I do not want to write the whole expression here I will just write the result so we have minus 200 times 0.7 plus minus 400 times 0.3 and that will give us 260 minus 260 so that is this value is the expected utility assuming that I have perfect information of course if I actually have perfect information I will even know that in this case I have the expected cost of minus 200 or I will have this and that is minus 400 but since I actually do not know yet what will be my future perfect knowledge it is this weighted average and then I compare that with 260 I compare to my minus that I get a priori and that means that my my value of perfect information is equal to 140 for any test that I have will give me a benefit of at maximum 140,000 euros any test that costs more than that is no point in doing it because it is not going to be even if it is perfect it will not give me more value than that this is kind of an upper bound the actual value of the information I think was the order of 60 because we can also understand when we calculate the actual value of information for example this is the order of 60 we can also see how close we are to the upper bound it makes sense to maybe say we should try a better test or we should try an additional test something I am going to mention tomorrow is that we might just consider to do a second test or a third test I can already say that if one test costs 20 it doesn't make sense to do more than 7 tests because more than 7 tests will not give me any positive net benefit so this is a very useful concept and it is much simpler in a way much simpler to implement because you don't need to do Bayesian updating and you don't need to know the likelihood or you don't need to know the quality of your information and it gives you a first indication and then there is something related to that and that is the there are some examples in the lecture notes there are some examples in the lecture notes so then there is the value of partial perfect information here the idea is that in most real life problems only a part of the uncertainty is reducible so we go back here we could think that maybe we can reduce maybe reducible means we can learn something about it so maybe we could say that maybe we can learn something about Q but typically for the load maybe we could learn about the statistics of Q but the load itself typically cannot be learned at least deterministically there is always a component here that we know we can never know in advance so let's assume now that we have the perfect statistics of Q but we don't know what will be the maximum value we don't know if a certain probability with which a certain high large truck will drive over this bridge but we don't know when it happens and if it happens this uncertainty cannot be reduced we cannot learn anything about this so when we would calculate the value of perfect information we would assume that we know exactly Q and R but that doesn't somehow make sense because I'm never going to learn about Q so instead we can calculate the value of partial perfect information which is we assume that for those random variables where we can collect information we know them perfectly but for those where it's not possible to collect information we still consider them as random and then we do the kind of same analysis that I showed before so in principle if I show it in terms of a Bayesian network it means that so this is now not just one parameter but it's multiple parameters it's a vector of parameters and I'm separating those into irreducible meaning we can actually learn something about them they are also sometimes called epistemic and those here are irreducible sometimes called aleatory we cannot learn anything about them and then we look at this problem so we assume that we have full knowledge of those but we have no knowledge of those and we can do it in the same way we just add this link here we calculate the expected cost associated utility associated with that compare it to the original situation and we get again an upper bound on what we can potentially gain from collecting information and yeah so that's for example this example that I mentioned with the rock fall gallery this engineer could have said there is uncertainty on the rock and the size the impact load and there is an uncertainty in the model the structural model of the structure and he could have looked at this and said okay this is what I can I can improve my model up to the point where I have a deterministic exact model of the impact and I will not learn by improving my model I will never learn anything about the load so he could have compared this to this and so realized that there is not much point in doing any analysis because you would have seen that the value of partial perfect information would be very small so this is often a useful concept and tool to do an initial analysis before you go and actually model the likelihood and try to understand the quality of your information ok, so, this is the last one here that just summarizes these ideas that basically this is the prior and you compare this prior to this to get the perfect information the value of perfect information you compare it to this to get the value of partial perfect information and you compare it to this situation to get the value of the information this summarizes this free type of analysis. Okay, yes, okay, this concludes this lecture for the influence diagram. We can practice it a bit. I will come back to it a bit tomorrow when we speak of sequential decision making. We'll start with this slide here where we look at, if we now consider multiple tests, not just one, we start to have a sequential problem and how do we extend this and this will be for tomorrow. So any final question so far? Good. Okay, so we have half an hour before the lunch break and I want to use this to discuss this topic of modeling this node here. What I already said is the likelihood. How do we represent, what is the representation of this node? We will try to make it very brief, hopefully, and then have maybe 10 minutes at the end, maybe two or 15 to discuss and see if some of you have, in your applications, assuming that you have already started working on specific applications in your PhD, what is the likelihood, how does the likelihood function look like in your application? I'll prepare some slides here. The material you have seen so far is almost entirely, in the lecture notes, except for a few slides I had on specific applications. So I'm not going to distribute those slides. What I'm presenting now is not in the... It's not in the... This is what other people do here. My daughter is doing there. Five minutes from here. This thing is not in the lecture notes, so I'm going to upload the slides, these few slides here. Elizabeth is a PhD student, and she has helped me to do some of these slides. Some of you were in the course in Como. She was presenting in the Como course. Let's jump right into it. I already said that in the Bayesian network, the information is typically a child of the quantity that we want to know about. Let's first look at the simple case here on the left side. I've already considered here the fact that we actually have... If we do monitoring over time, or if we do inspections at multiple points in time, we end up having a temporal network. But if you consider only an initial test, just forget about the rest, and you can just consider one-time slides. The initial test is just one-time slides. Of course, we can have multiple many-time slides. I'm kind of consistent with what Jochen explained yesterday. I'm distinguishing between a direct situation and an indirect situation. Direct means I'm able to directly monitor my condition. Let's say my condition is whether the structure is corroded or not. This is what I'm interested in, and I can directly mean that I have something that measures the corrosion. What I get is still, in most cases, an imperfect representation of my condition. So think of the health cell potential measurement. It cannot actually give you a complete exact picture. It just gives an indication, but it gives an indication directly of whether or not it's corroded. But let's assume that the condition of interest is not whether it's corroded or not, but what we're interested in is whether it's safe or not, whether it will fail or not. In this case, what I'm trying to observe is related to the corrosion, and that in turn is related to the condition. That's not a very good example, actually, I realize. We should make a different one. Let's make a different example. Let's say I want to know, again, whether my structure is healthy or not. I take the stiffness of the structure as an indication. So I'm saying if there is some change in the stiffness of the structure, it's an indication that I have a problem. So the stiffness of the structure is taken as an indicator. And the stiffness is something I can try to indirectly measure by looking at eigenfrequencies and so on. The distinction is not always clear. It's not a clear-cut distinction between this and this. You can also always reduce this model to this model. But just to make it clear that it can be that we define this likelihood function relating a monitoring result or inspection result to direct it to the condition or that it relates to an indicator and then I have to relate the indicator to my condition. In which case, in a way, I have here first a likelihood that describes my test and then I have an additional model that describes the relation between my indicator and my actual condition. And then the third case, again, you won't probably construct even more cases, but the third common case is that I actually observe not the condition of the structure but I make observations of the environment or the load. So it could be the loads. You could measure, try to measure the cues or loads because we measure chemical environment. For example, for corrosion, we might measure the chloride concentration or temperature, humidity. So we might measure that and then the model is a bit different so we have that this environmental parameter or the load affects the condition and we have the likelihood function. That is, again, here's the base network. So again, the causal relation is in this direction. But by learning something about this or by observing this, we learn something about this and we learn something internal about the condition. So you can think for your system which model best describes your particular case and it might be that you have to add additional variables and these are not the free only options. But you know now the base network or you can kind of relate to the fact that in any case what I'm trying to look at is how do we relate what we observe if it's a monitoring or inspection outcome or a test outcome to either the condition or to the indicator or to the environmental variables that we have. And depending on the situation I might need additional relations to get from that to the condition. But I'm now just looking at this green part and it's called the likelihood function. So who has used the likelihood function in their life? I think almost everybody has used the likelihood function. If you've ever tried to fit some distribution in MATLAB you have used the likelihood function. Because the maximum likelihood is the most common statistical estimator. And the likelihood function is nothing else as we already said. Typically we often write it like this where we say this is the likelihood of a parameter, this is in the mathematical textbook. Which is basically equal or strictly is proportional to the probability of a certain observation or data. So I'm calling this, see here, set. Sometimes called the D for data or Y or whatever. But this is my observation here or what I actually observe. Observation data given theta. And in the maximum likelihood we're trying to find the parameters theta that maximize this. We're trying to find the parameters of the model that best explain the observation that I'm doing. This is the idea of the likelihood, the maximum likelihood. And in the Bayesian analysis, as you also might know, hopefully know, that Jochen, the posterior distribution of theta is again proportional to the prior distribution of theta with the likelihood of theta. So this likelihood function appears both in the classical maximum likelihood estimator but also in the Bayesian estimator. So this was the prior model of Jochen, the normal distribution. And this is the likelihood that describes the observation. And in Jochen's formulation it's a bit hidden because he had this conjugate prior and all. But that's what is behind all this. Yes, exactly. So this is the thing. And so maybe this is my prior knowledge. It's what I know before I do the test, before I have the data. And this describes me the information of the observation, the data. And this is here. And in the, I mean, so basically that's why here's the Bayesian analysis because more frequent people prefer the maximum likelihood estimator because it does not require any assumption about the prior. We just try to maximize this function here. So this describes my data, my observation. And so this is, so all these models that we have to describe the quality of information of my outcome, be it the probability of detection, probability of false alarm, this measurement uncertainty, they are all likelihood functions. They are all of this form. This is what I want you to remember most. All of those are of this form. And when you have to construct your specific model for describing the quality of your SHM, of your inspection, it has to be, in a generic sense, it has to be of this form. So this is over in this slide here. Yeah. And this, that's okay. These are slides of Elizabeth, so I didn't look at them, but they are exactly what I just said. So you have it there. Okay. Now, the real, I mean, now we have to look at, okay, what is the, this is very general concept. Now how does it look for specific cases? And we can, I'll try to make a kind of a classification here of the different cases of types of data and model parameters that we can have. And in most, generally I distinguish between the condition itself can be binary, discrete or continuous. It's, what I don't say is that, okay, binary typically is obviously by nature. These are discrete, but the continuous one can also be vector-based. This can also be a vector, but in the discrete case it doesn't actually matter whether you have a vector or not, because the vector just adds number of states. So, but this could also be a vector of continuous. So this is the condition or the indicator. So we saw before that what could be here could be either the condition, or if I have this intermediate indicator, it could be the indicator. And then I have the observation result that can also be either binary, discrete or continuous. I mean, kind of ideally we are in the diagonal. So if we have a binary condition, we have a binary observation, but all these are possible combinations. And I'm just going quickly through those to see what happens. So if we have a binary-binary situation, that's the simplest case. It's binary-binary. So just an, this is just an example. We could have no damage or damage or crack or no crack, corrosion, no corrosion. And the method tells me corrosion, no corrosion or crack, no crack. So in that case I have this binary and this is what is called the confusion matrix. So I have, and this is the same type of probability table as we have in the Bayesian network. So given theta, given that I have no damage, given that I observe no damage. And that's one minus PFA, where PFA stands for, this is probability of false alarm. So I should actually start it here. So assuming I have no damage, the probability of getting an indication of a damage is the probability of false alarm. That's basically what we have here. That's the thing we don't want. On the other hand, if I have a damage, the probability of getting an indication of that damage is the probability of detection. That's what we want. And then these are just one minus these quantities. So this is a likelihood function because it's the conditional probability of set given theta. And I guess almost everybody knows these quantities. P or E, PFA. Okay. So go back to the case A up there. Now comes case B. Case B is that we... which is such an extension in a way of case A, is that either the set or the Y or both are not binary, but are just more generally discrete states. For example, no damage, slight damage, large damage, or number of cracks, zero, one, two, three, four, and so on. This is just kind of... And we have just an extension of what we have before. This is still a confusion matrix, just that it's not P or D, PFA, but there's no real name to it. So basically it's the probability of being correct, which is always the probability of these diagonal things. The probability of observing set equal one when theta is actually equal one. And then we have the off diagonals which kind of indicate different degrees of being wrong. So this confusion matrix is... it's not that common, because typically either the outcome is typically binary or it is continuous. This case of discrete is not that common. But in principle, if you have a case, then you need to fill out this confusion matrix to describe the quality of your measurement. I'm a bit confused. It's good because it's a confusion matrix. The second, the set two. I agree that there is something wrong. Yes. There should be set two and set two. They should be set equal two and they should be set equal two. That's just... You're not confused. This slide is confused. I will just fix it right now. These are slides that Elisabeth made for a lecture in last semester. So it seems that you are less confused than the students in that lecture. I think there were only five of them. And they were only master students. All right. So now we are not confused anymore. Then we have a case C. The C is that now we have a continuous state but our observation is only binary. And then that would be something that looks like this. So this is kind of continuous. And then we have here a function. We also call it typically POD which is the probability of detection or I'd rather call it probability of indication but people call it probability of detection. I don't... But most likely it's an indication because it's a bit difficult to say detection. For example, I thought it's in German here but this is a common example. Here we have a fatigue crack or a damaged size. But you can think of this as a size of a crack. And now we use a non-destructive testing method. And this will give some signal. And then the people who use that put some threshold on the signal and say, okay, the signal is above a certain threshold we will make an indication if there is something. And that, of course, can be calibrated and you come back to that. But once you put the threshold you just say, okay, there is something or there is nothing. And now because this is a continuous thing it's from zero to some maximum value and there's never zero crack. I mean there may be 0.0001 millimeter crack. But that's not something I want to know about. So I put some threshold that makes some sense and then the probability of getting an indication or detection as they call it increases with the size of the defect. So we have some continuous curve like this known as the POD curve. So that's, again, a likelihood function. And if you think of it, you can just say that, okay, SEP, which is binary is a child of the continuous state. They are also multi-dimensional PODs. So it can be that it doesn't depend only on one quantity but on two. Well, I've seen PODs that define the probability of detection both in terms of crack length and crack depth. So this is the case C. Okay, case D. Now the opposite. The observation result is continuous whereas the state is binary and here's another example of that. I mentioned already yesterday this case. So that is an example where the state is discrete. So this is whether we have no corrosion or corrosion. For passive means no corrosion. Active means corrosion. And what we see here is a result from a potential, half-size potential measurement. That method gives you a potential difference between basically you have to connect your device to the reinforcement and you measure if there's some current and the current will be different whether you have the potential will be different whether you have an active case or a corroded case or a passive case. So you measure a continuous quantity and you're trying to figure out whether it's corroded or not corroded. So the likelihood function in this case is a probability distribution or probability density conditional on the discrete condition. So we have one for active case and one for passive case. If you have multiple discrete states you could also have multiple of those. And again this is the likelihood function. It tells us whether what is the probability or the probability density of a specific outcome given to the condition. Now in principle and then now I come back to what I said earlier when I showed you these PODs we can derive a probability of detection from such a continuous case. Actually that's in a way what they do because, and I should have put, sorry I have, okay I don't, I will not find it so fast. I have some pictures of those things I forgot to put it here. So basically what you end up with is you have a concrete surface and on that surface you do the measurement at multiple points and at each point you get a potential. So at one point you have minus 500 and sometimes minus 300, minus 100. So you have this point. And then you have to make a decision whether something should be done or not. So for each point the engineer has to decide whether to do something or not to do something. He has to decide is it corroded or not corroded. And it's not actually the way you should do it but the way it is done is that people fix a certain potential and say okay, based on these curves we would say okay around here if we are on this side we have no corrosion and if we are on that side we have corrosion. That's how we interpret it. This is a deterministic interpretation. That's why I think it's not how you should actually do it. What you should do is you should use those two curves put them in your Bayesian analysis and just calculate the posterior probability of corrosion and use that. But the way they do it is they use this more deterministic interpretation and they say okay everything that has a potential here is not, we interpret as not corroded and here we interpret as corroded. And from this we can now construct the POD. How can we see in this figure this is the threshold that they put this is not corroded so this is the distribution given that it's not corroded this is the distribution given that it's corroded. Everything, every result on this side is interpreted as corroded every result on this side is interpreted as not corroded how can we see on this figure the probability of detection? Yes, that's the first point do you mean this one? That's one, okay. So you take this, you took the CTF up to this point and that gives you the exactly probability of detection. So because everything that is up to here you would actually interpret as being corroded and it is actually corroded so that's the probability of detection and if you take this little thing here it's the probability of not one minus the probability of detection this area here gives you the probability of missing a corroded area this is the probability of detection, correct and the probability of false alarm we take the other figure and we do the CTF again so this area here this small area here will be the probability of false alarm yes and now we come to what I mentioned yesterday and probably you were not able to understand from what I said maybe now it becomes more clear is that I can move this fresh all around okay and here I, this is a hypothetical example but so I have these two curves and in this case it's just turned around I mean it depends on the interpretation here it's just turned around so the PFA and the POD are now on the right side and what you see here is the threshold and this threshold is moved so I can start here so I'm very conservative and I'm interpreting everything here as an event as a failure or as a problem and then I'm very conservative and I move the threshold to the right and then I'm becoming less and less conservative and you see as we move those to this threshold we change both POD and PFA and that's how we get the receiver operator characteristic so and so I, and do you see these two situations correspond to the two points shown there and this point here corresponds to having the threshold here so nothing is interpreted as an event and that point here will correspond to the threshold being here where POD and PFA are both one because we interpret everything as a defect and you see this for this is a very common way of representing the the likelihood function of monitoring but also this also is very often used in statistics this RLC curve as a way of describing the quality of a predictive model and nowadays in these all these predictive modeling machine learning you see very often these receiver operator characteristics curve as a way to describe how good is my predictive model binary classification problem you want to say something yes thank you for introducing this concept I ask the question to you Daniel how should the threshold be determined should we what should we do should we use this receiver operating characteristic or could there be another method what people have as we know thank you it's a very good question this is a classical problem we have to now optimize the threshold how should we choose the threshold and I've seen some papers that people try to they say define some kind of distance measure because this is the optimal point try to find some kind of distance measure or whatever in principle we are still in the decision analysis problem and what we should do is we should make an influence diagram or the decision tree we should think of the decisions and the consequences make a decision tree and take this threshold as the decision variable the decision variable at which we either do an action or we don't do an action and we do an optimal result the expected utility problem because in some cases we do it if people do that of course so for example if you have an aircraft you do inspections on a critical component in an aircraft you obviously want to be very sure that there is an omen versus for aerospace or rockets you really want to make sure that there is no defect it doesn't when you produce a rocket it doesn't really matter if you have to produce the piece that matters but not so much if you have to produce the piece otherwise you are sure that afterwards it's not going to fail so you are conservative if you accept a large number of false alarms so you would want to be very somewhere here maybe if you have a low consequence type of failure failure is not so critical you rather don't have too many false alarms because those all produce costs that maybe you have a bit of lower probability of detection but this all comes out if you make often in many cases you can do very simple decision analysis just say what is the consequence of doing something meaning what is the cost of an indication in the case of the concrete that I have made before what is the cost of opening that fix or doing something with the concrete what is the cost of repairing the concrete on the one side and on the other side what is the cost of not identifying correctly a failed corroded element what happens if I it is corroded and I don't correctly identify it and then you can make a very simple decision tree and try to find the optimum threshold so that's the way you should approach that and I don't know how people come to the idea to suggest the optimum of course it's clear that the optimum is certainly here and the optimum is certainly here but you really have to consider the cost and decision analysis gives you the answer that's what I would think and in a way if we if you put in our analysis I would rather you can directly use this information here the threshold has an optimization parameter and you get all this that's why I'm saying what the people actually do is that they fix this potential without considering the decision problem of course this seems a reasonable range but without looking at the decision problem they just fix the threshold and then they use it like this it's going to be okay but it's going to be suboptimal they should use the complete curve put that in the analysis optimize and the analysis will tell them what is the optimal potential okay I think that is almost the end so there is the last case it's the continuous case so both what you measure as well as the quantity of interest is above continuous that's maybe almost the most common case and one example model is a very simple model it's just you know it's just a measurement uncertainty that's how it would look if you say you measure one over a millimeter so maybe this is a crack size you try to measure a crack size then your likelihood function is just similar to what Jochen had also it's just a probability distribution hopefully centered at the true value if you don't have bias true value with a certain standard deviation yes so this here sorry this here this this here will be the probability distribution probability density mean value the true value and the fixed standard deviation or standard deviation that represents your measurement error and that's also a likelihood function that's almost trivial okay so that basically that's it of course the question is in practice there might be as we discussed earlier it might be sometimes challenging to actually figure this out how can we actually determine this with something like this okay you produce a number of specimens in the lab with certain crack sizes then you you run your method and you crack repeated ten times and for each defect size you get a result and then you multiple results of your outcome and then you can make it conditional distribution of the outcome given the true value and then you can construct such something like this so that's so the I think the most example that we do what we're trying to do is to separate the likelihood function from the actual structure system that we're looking at meaning that I want this likelihood function to be a function of the method of the testing method but not of the structure where it applies because if that's the case then I can do some tests in the lab or maybe on previous structures where I first measure the true value and I can use that to make a statistics of the likelihood in the technical example that's more difficult because there the likelihood is always most likely dependent on the specific side and the specific conditions but in the structure if I make a test of the concrete strength that's independent of whether it's going to be for that bridge, for that bridge, for that bridge I just do it I have a certain test method I apply it 10 times 50 times I take those specific specimens to the lab and I test them with the more exact method to get a reference truth I compare my test method prediction with the reference truth and I have 50 samples of that that gives me an idea of the measurement uncertainty and I can then take that and use that for predicting the measurement uncertainty of all the concrete structures and the same will be done for crack sizes detecting crack sizes measuring deformations and so on of course there are environmental factors that affect the quality of the sometimes many of these devices are sensitive to the environmental factors so you have to understand also maybe that but when you are able to say these are independent of the application of the particular structure then relatively straightforwardly determine them simultaneous observations the amount outcome and the property of interest then you have a cloud of measurements and represent by the likelihood is nothing else that is additional to one realisation yes or something like this that is also very nice these two directions of interest so you either come from the observation observation to do something like this sorry but here this is what you I like to do it like this so this is my true value if you want and this is my set and then you have this cloud I mean ideally you are on this line and this cloud gives you an information and then the likelihood is basically the distribution something like this what is the interest in the environmental direction? well then you use this to update I mean most people plot this set here and see it here but this is again the diagnostic thinking so the diagnostic thinking is to put here set and here theta or say okay my theta in function of set the kind of causal thinking is that set is a function of theta I mean it doesn't matter to here but I find it helpful to plot it in this way of course it can also be that this standard deviation might be like this measurement error increases as the size increases then you have a multiplicative error model or you have an additive one this is pretty good regression problem nothing else one last thing here one of the reasons why here you could also say the monitoring result set is directly we can formulate the set which is a function of theta here also but typically what we have is that for example if I take this example what I mentioned let's say no good example right now but the condition of the structure is whether the structure is safe or not and what you measure are some where that relation between the deformation and the condition of that particular structure is not structure independent this will depend if you have a 500 meters long span you would expect a few millimeters of deformation are not critical if you have a one meter span and a few millimeters of deformation maybe are critical so if you would take the relation directly from this to this will not be structure independent but if you say that my indicator is something for example material property stiffness value or something like that that might the relation between that and what I measure might be structure independent and then the relation between that local stiffness and the sorry too late too low sugar in my blood cannot I'll think of a better example but the point I wanted to make is that the separation can help us to take the structural specific part here this is where I need a structural model for example to relate my condition to something I can observe and here is the thing that this describes the actual measurement device the exactness the quality of my measurement device to make to make sure that this is structure independent and only this here depends on depending on the structure if it is a structural identification for example this part here might be the only the accuracy of your of your accelerometer that might be pretty good compared to the uncertainty you have in your structural model that relates the accelerations to the condition of the structure and the accuracy of the accelerometer is something you can take from one structure and put it to the next okay so I think we are almost running over time for lunch so I wanted to to this cast maybe 10 minutes but I will not do it now but we'll do it maybe tomorrow take 10 minutes or 15 minutes so my idea was to ask you the type of applications that you are working on in your research what are the likelihood functions or what are the how do you describe in your application this quality of the information maybe you are using these things already all the time maybe something that is you or maybe you are using it but you are not aware of exactly what format it is so maybe if you think about it a little bit until tomorrow and then we spend tomorrow maybe 10-15 minutes just to to discuss how you can relate this to this likelihood you get feedback from us yes to announce in one course and then we have an overview of how many it will be and then we somehow sketch it with the burden thing so it's a nice sort of thinking to expose yourself to make food out of yourself not at all the opposite right we have the most room here and we know that others are of course entirely right to protect you so it's a good opportunity to discuss and learn more about your project that's a very good idea okay there is no urgent question until the end of the day you can just announce if you want to do that and then perfect