 Sorry. So we wanted to understand the the the risk factor. So here you can see in this network the risk factor related to antibiotic resistance. And here each of those variables is one antibiotic and this is the encoding of if if a certain subject is resistant or not. So zero one. In this example, the the purpose was after an introduction of a new legislation in in Sweden was to understand what are the protective factor and risk factor for for animal welfare. So so we use we use the the the Bayesian network in in order to to understand this complex relationship. And and the last one here it's it's about a so-called technopathy. So this is a disease that happened to animals. Only I mean this disease is not present in the wild. This is just because we breed the animal that they have a certain disease. So this is proto dermatitis. And so where I want to go is to say that the scientific question between behind those papers. So here the antimicrobial resistance Bayesian network was a was suitable for this this problem because we have multiple outcome. So we want to to have a holistic view of multiple outcome on the single analysis. The second one is because I mean by nature animal welfare is multidimensional. And we don't want because the classical way in statistics to to to study multidimensional outcome is usually to do a score for example. So you say welfare is a score between zero and one. And yeah. And here we do not want to reduce the complexity of the relationship between the variable to just one score. And this is why we think that Bayesian network was suitable. And in the in the last example we use the ability of Bayesian network to generate hypothesis. So this is biological hypothesis about OK. We have a certain technopathy and we want to understand why it happens. So which are the factors that affect the the the podothermitis in industrial breeding rabbits. OK. So this was this was for for the motivation why we can or in which setting a Bayesian network analysis could be could be used and what which kind of of scientific question you can address using Bayesian network. OK. So now I want to situate ABN or the Bayesian network into the big landscape of machine learning. So you can see here that you have the the three pillars of of machine learning. So the super visit learning with regression classification. You have the reinforcement learning and we you have the super visit learning. So ABN has a data analysis niche near to this dimensionality reduction where we speak about structure discovery. So using a statistical model to try to to to understand the underlying structure of a complex data sets. OK. So what is the objective of this workshop. So how to learn Bayesian network from observational data. So actually if you are unfamiliar with this term. So in this case learn is synonym as to select. So first thing first what is a Bayesian network. It's defined by two elements. So first the network structure you probably have already heard this word dag for directed acyclic graph which is the set of the vertices. So so so the the random variable the column of your your your data set and the arcs which are the the relationship between those variable and the probability distribution. So this is two phase of the same coin and and actually the Bayesian network encode the factorization of the joint distribution. So a Bayesian network per se is a schematic representation of a statistical model. So how how it works in practice. So from observational data sets. So the idea is to deduce a probabilistic model and from this probabilistic model is to deduce the structure. So the the the feature of of ABN compared to other software to learn a Bayesian network is the fact that ABN accept multiple kind data type. So binomial Poisson distributed so counts multinomial and and and the Gaussian and the the trick to put that all together is to use the so-called exponential family. So profit from what have been developed about the classical GLM regression setting in order to make all those data live in the same statistical environment and being able to construct the global network from that. So you start with an observational data sets and the idea is to learn the probabilistic model and from the probabilistic model you deduce the the structure. So now let's speak about the limitation of this technique. So here I put you the the number of nodes and the expected so the maximum number of that link. So you can see that this number increase quite quite fast and here it's the level of inference you can hope to do with a certain number of nodes. So of course any Bayesian network modeling approach should have implies some approximation. So the very first one is to limit the number of parental nodes and this will be part of the of the hands-on tutorial. You will you will see how we do that in practice. So the idea is to control the level of complexity of your of your of your model in order to reduce the computational burden. The other thing is to use the composable score. So scored yet that you can compute separately on each of the nodes and another feature is to have the so-called score equivalent. So now I want to give you with this slide it's a quite technical but so the purpose is not to enter to the to the detail the mathematical detail is just to give you the feeling how we deduce the the structure from the probabilistic model. So the idea is you take the the definition of the of the conditional probability here at the top in number one you take the base theorem from those two you sorry you can deduce the definition so the so-called conditional independent. So if you have a set of three nodes from those two lines you can deduce the factorization of the joint probability in order that for for conditionally independence and there is a theorem from verma and pearl 1988 that make a link between a statistical statement so the conditional independence and a graph property which is the de-separation and any or to the best of my knowledge all the algorithm that tries to learn Bayesian network so select the Bayesian network from observational data take profit from this so essentially take profit from this this theorem and here you can see that if you have the the all the possible direction of the arrow for a set of three nodes one colliding arrows so this one do not implies conditional independence the three other implies conditional independence so if you have access to this information which variable are conditionally independent or not then you can start to construct the network so this was just this brief introduction just to give you a taste of of how in in practice so what under the hood how do we do we construct the structure for from from the probabilistic model and now the so can I just ask a question yeah so it's there is no independence in the previous slide because you can't know the relationship between a and b on the left ah you mean you mean in the here on the left you have the you know the relationship between a and c and b and c but you talk about between a and b so that's why there is a no you have no conditional independence oh my okay so so I'm not entirely because so the point is um if you if you want to factorize this set of three nodes so if you write the probability of a b c and you decompose this probability knowing that this is the structure you expect so so the joint probability of a b c is the probability of c given a times the probability of c given b and if you if you um so if you if you continue to work on this this expression you can show that it implies that there you you cannot factorize that using conditional independence no or did I no actually it was just a question why why is it this part is not on the right because you have no no I mean from what I see that you can't infer from the scrap the relationship between b and a you don't know I don't understand what this it means that a causes c no so so maybe if you if you want with during the anzone exercise we can yeah we can have a further chat on on on that sorry thank you so um so there is so the idea of the learning Bayesian network there are two class of algorithms to do that either you you directly test the conditional independence and you can construct the structure from there and this is called the constraint based algorithms or and abn lies on this side of the of the of the slide this is the so-called search and score algorithm so the idea is to maximize the so-called a posteriori score so essentially you you take a score in this case it could be b i c the log marginal likelihood or there are some Bayesian Dirichlet score that if they are well designed um you you you if you maximize this score the structure on the conditionally to a structure the structure will represent well your data so the task of learning a Bayesian network it's a it's a two-step process so you have the model selection those so the structure learning and you have the parameter learning so parameter estimation and here it's the it's the the the Bayesian view of this two-step process um for so everything I said up to now is quite general so now I want to go more focused on abn so how abn is working in this case so again it's a search and score algorithm so the idea is you construct a big list with all atomic possible structure you use a model in this case a glm with the exponential family in order to have the score for each of those atomic network then after you use a certain uh algorithm so in the hunson you will see that there are an exacts algorithm and there are some heuristic algorithm as an example and you try from the atomic so from this big list to to put all the nodes and to maximize the sum of the score so the global score of your network and finally you have a structure that has the highest posterior probability and from conditionally to this network you uh compute the marginal posterior density so essentially the regression estimate so conditionally to the structure you estimate the parameter for the rx um so in r using abn it's the sequential application of uh three functions so bird score cache to construct the cache here in gray uh most probable in order to uh compute to maximize the the posteriori uh uh network and fit abn to compute the the parameter estimate there are other structure that we will uh see in the exercise so one thing is to ban or retain some some structures or some rx so you can imagine that for example if you in your data set you have um biological sex of a subject or animal no viable can influence those um those uh viable so the idea is to restrict your problem to um relationship that are plausible from a biological or from a from a from a from a subject perspective so you can ban some some some rx or you can retain some rx because you can say yeah from a prior knowledge so uh we know for the last 10 years that um this there should be an a relationship between these nodes and these nodes and two other possible uh feature are the so-called random effect for accounting for clustering using mix model we are not going to discuss that in the hand zone and adjustment um here it's a big map of uh the different function implemented in abn so um you have the core function so the model learning with belscore cache it has a a Bayesian implementation using Bayesian irrigation and using the classical glm uh you have some search functions so most probable for doing the exact search uh search uh there is a hill climber and there is a a heuristic that implement different heuristic algorithm in order to maximize the score of your network and then after for the parameter learning there is the fit abn function that returns you the the the value of the of the coefficient and again this is there is a Bayesian and an mle implementation um um additionally to that there are function called ancillary function for analysis that you can you can use so for example what is in this one it's a there is just a mle implementation it's uh with what is the contribution of each of the observation to the global score and uh there are some function to do some simulation so you can simulate uh DAGs or you can simulate uh abn data excuse me can i ask something yes so is there a need for a prior in this process um yes so abn so so there are multiple priors that use so um you can have a so when you do the the most probable you need to have a prior for the network so how do you determine the prior in this case uh two prior are already implemented so you use a default value yeah there there are there are to if i remember well uh there are purely an informative prior um and we implemented two informative semi-informative priors yep so so there are some default them and and uh so this is for the prior for the structure and there is priors for a parameter estimation in the Bayesian setting of uh base core cache of fit abn and uh here uh they are implemented uh they are hard implemented in the code so this is a weekly informative priors so uh excuse me for asking again but um uh i've seen some um that some reviewers always when there is a Bayesian application they ask how you determine the priors and ask why you are using the the default ones and not something else so i i'm not sure i understand the way you'd select the the priors so so for for the parameter default so for the parameter estimation there is no possibility except if you if you modify the hard code there is no possibility to choose another prior so the prior have been uh selected because um this is the only possibility so using those prior is the only possibility in order to make the computational burden accessible on a on a laptop machine so essentially the choice of the prior is driven by um the the the computational complexity of the problem uh and and um i think in the vignette of the paper there is a discussion about uh because the only prior you can you can select uh is the prior for for for the for the search in in most probable and there is a discussion about the advantage of the two different priors yeah uh if you have time later on can you explain us how you end up with the selected prior so what's the rationale for for using for selecting a prior for yeah yeah uh yeah i will i will if you if you put that in the chat i can come to your breakout room okay thank you thank you um okay so uh this is the end so uh this is the end of this uh brief introduction um so it's a very selected bibliography um so now i um want to go in the in the in the presentation for the hands-on exercise it's a very good point so so we are quite a lot in this in this tutorial so if you have any question i think the best is uh put it on on slack and i i will come to your breakout room and uh we will try at the end to collect all the all the question and to make an FAQ and put that on the website for the for the tutorial um for all the question we have an answer and yeah so for the hands-on session um relatively quickly uh so this is the software requirements this is the way the same uh you have on the on the on the website uh so the data we are going to use is a veterinary epidemiological research they come from this book um this is the uh uh data about pig so this is growth performance in and abattoir findings of pigs from the selected farms in canada this is a quite old data set um the advantage is it has a mixture of different viable um so the the data were collected in order to study the entire relationship among respiratory disease so you have a multiple disease and the daily weight gain uh so this is adg and this is what you are interested in so this is the your your target variable um this is the the data as described in the book um so we will not use all the data set so we simplify a little bit the the data so here you can see the encoding of the of the variable so this one is binomial because as as i said abn can um support different data kinds data type so binomial poisson counts uh multinomial and Gaussian but you need to provide this information to abn so the very first thing you do when you do an abn uh analysis is to construct a list a name list with the the variable you have and which um uh distribution you want for for this variable so so all those ones are bi are binomial the word worm count uh is a is a number so this is a poisson uh this one is assumed to be to be uh uh continuous as well as as this one so the average daily weight gain and the the age of the the pigs um so just to give you some insights the idea of an abn modeling step so first thing you search for the optimal model um you always it's a it's a good idea to try to visualize the data and and then after once you did this first analysis and you know the level of complexity you need uh to to have an optimal model then you do the so-called bootstrapping so in the in the tutorial in the in the hands on uh we provide you with the code to do the parametric bootstrapping and the idea here is to prune the the unnecessary arcs so um to to adjust the network in order that they you don't overfit your data and then after from from there you you you from from the prune dag you extract the marginal density uh you present the final dag and you can start to do the interpretation yeah very good so so so as a as a as an id so the id is is is um uh it's not an academical exercise it's more like uh you have uh you have the code and you you have the the output and and the id is to to to understand how an analysis is is done which step are necessary how you perform them and again this is quite debatable because uh we present for the bootstrapping part we present uh parametric bootstrapping but you can also do non parametric so essentially sampling inside the data set in order to to to prune your network hi everyone again back to the main uh back to the main room so i did not have the chance to pass to all the all the room but i hope that you you if you have a some question you can you you you have the chance to ask that to them to to ta um so what we would like to do in the in the next 10 minutes is now now you you have seen this exercise this handsome um to go through the handsome a little bit together and discuss important points so just from the from the from the table of content so um this is this is a the preliminary so so the the id of this i i put that in the chat in the slack so the id of this hands-on exercise is more to provide you with an entire code that could be used in another context for another analysis if you want to reproduce that with your own data so all those points here it's more like the preliminary steps you need to follow in order to format the data in order to a b n to understand your your data and this one is to do the first run um and to um identify which complexity level is required by your data um and and from this point here it's about parametric bootstrapping so i saw during the exercise and during the the discussion that um it was a it was a hard part of the of the of the exercise so here the id of the parametric bootstrapping is so you identify the model you think that represent well your data and the question from there is to know if this model is too complex in a sense that it contains some arc that are unnecessary so if you overfit your data from this question you have two directions possible the first one is to do parametric bootstrapping so essentially generating data from your model and doing a search over and over on the generated data so this is what is presented here so this is the parametric bootstrapping another approach would have been and for the next version of the tutorial i think i will go in this direction would have been to use a non parametric bootstrapping so essentially if i may say a for loop where you sample your data set so essentially you you sample 80 percent of your data and you do a search on that and you do another search on on another 80 percent and etc and etc and you generate many many many many search so you have lots of tags and at a certain point you try to combine all those tags to identify which arc are well supported by your data and which are arc are not and and the last part is more about presenting the the the result so we can maybe quickly go into the into the the html page so this is to plot the data i always advise you to do that before doing any vision network analysis so this is the point where you you you choose the distribution so you need to assign every variable to a distribution and this is the part where you give the band and retain so which variable are going to be which arc are going to be banned so discarded from the analysis or which are retained so always present in the analysis on which basis is that prior knowledge so there is no so this is something as a subject expert matter this is your job to know what should be banned and what should be retained okay so there is no general rule of course the more you ban and retain the more external input you put in the process the the more feasible the analysis would be and sensible because the advantage of banning an arc will prevent a bn from returning and a dag that that is from from from the subject expert matter nonsense so if you know for sure that an arc could not exist then you should provide this information to abn before okay yeah um so one maybe one important point is this function burl score cache which create the cache and again here the naming is maybe a little bit limited because um this is a complex object that is a list that contain all the information needed for most probable and other function to optimize your network so essentially this is not just the cache it's much more than this so once you have this object you you fit that into the into the most probable so in this case an exact search you you have the network and you fit this network again this function returns you a complex object which is a list that contain more information and everything that is needed for fit abn to return you the the coefficient here we just extract the the the network score and from there this is where you start to identify the level of complexity that is needed in order to represent your data well so it's a it's a it's a naive for loop and again it works because we have a very limited number of data maybe if you have a more data more valuable than this you need to have a smarter um a procedure in order to to identify this level of complexity so here it's a it's a for loop where you do all that and you plot the marginal likelihood uh in function of the number of parents and you see that it reach a maximum and and it never uh increase again so you say that the good level of complexity in order to explain the data is is for parents um so you plot this dag and again here you do one search so you have your data you put everything inside you have the level of complexity then poof you generate the dag the problem and this is what is discussed here it's it's um highly probable because this model is very complex and the number of data is very limited so in this case we have 341 observation which is not a lot and this is very likely that you will overfit your data and to avoid that it is strongly advised to to use some pruning methods so in this case we we we propose the parametric bootstrapping but again any other approach including some that will be presented in the next hands-on exercise could be suitable and maybe the non-parametric approach would be easier to to put in practice so here essentially the idea is you have a certain model from this model you can generate some data and what we are going to do is to to to do that sequentially over and over using jag and it implies to to encode your model in a certain language called bug uh bug language to run jag on that over and over in order to generate data from your model so when you say pruning you refer to the variables prune your variables narrow down your variables pruning no this is this is the arc so it's because the the the variable this is a this is a this is your your your your data but your modeling is about the arcs so essentially the structure so how many arcs do you see in your data and and here what we want to do and this is uh this step is about okay we identify the best network with for example this arc here yeah you can see my um so am i sharing my screen or not yeah okay okay um so you see for example this arc okay the question is how robust this arc is or is it just there because this model score a little bit better than another one but it's not very robust because it's driven by some outliers or some some few observations okay and this is what we want to do now it's to know and to return a more prune dag which which keep only the robust part of your of your modeling um what's the least uh minimum uh sample size you could accept so it's a very it's a very complicated it's a very complicated question so so the thing you need to keep in mind is from a practical perspective the more data you have the better it is because vision network modeling is heavily complex so implying a lot of parameter so now another question is how many search are you going to do so if you do non-parametric bootstrapping are you going to bootstrap that a hundred thousand ten thousand hundred thousand or one million time here it's hard to say um I advise you to test but uh in this example we use thousand if it was for a publication that would go for at minimum ten thousand so and and here so so you can see so this is the original data this is the simulated data with your model so you can see that except for example for web count which where we do not have the same uh tail and probably for for for the those variable they are quite similar and the idea is to iterate the simulation of your data the search the simulation the search and etc and do that many many many many time in this case thousand but probably more if you want something robust and from this big list of that so you have a a big array of of of DAC from this you can compute for example the distribution of the arcs so so it means that sometimes for some of the simulation a five you have five arcs and sometimes you have 11 so you can see that by doing that again and again and again you have a certain diversity of model from your data and this is for me the main justification why it's super important to not rely on the best possible DAC you have but to average out that on many simulation um so here the strategy I propose we propose in this tutorial in this exercise it's to take everything that is supported by at least 50 percent of the bootstrap sample but again it's a another strategy could be also valid so for example you could say no no no I think I think everything that is supported at least by 30 percent of the of the of the simulation would be sufficient because we want to see something or you can say no we just want to have the very very very robust and then we just keep everything that is supported by 90 percent of your of your data so here you can see it's a it's a the percentage of arc retrieved within the bootstrap sample so you can see that for example this arc appears only four times over the thousand this one is always there this one is always there yeah so from this matrix which come from the bootstrapping you can generate a prune DAC and then after from this DAC so now we work in this DAC so we say that okay this DAC is more robust than the other from this DAC so so here it's showing how how you can see the the the the distribution for for each of the parameter okay and more importantly here it's a Bayesian approach so by default ABN is generating a Bayesian analysis here we can compute the different quantiles for the regression estimate so if we go yeah with the different interpretation so if we go to the final result so this is the DAC here I present you so so this was one of the question that someone asked me during the exercise is how we know that an arc has a a positive so an odds ratio more than one or less than one so here the dashed one encodes negative correlation and the plane one are positive and the sickness of the arrow is proportional to the bootstrap sample so essentially you can see that some arrow are visibly more supported by your data than other and you have the table of coefficient here okay for the sake of time I think I have to close that there I know that this is quite fast and so give me one question is there yeah please so when you select the when you have your the prior do you do many iterations on the Bayesian on the network so you have the prior you get an outcome then do you do another iteration with this outcome as prior and carry on or no no no we always use the default prior which are weekly informative okay thanks thank you so so now what I propose is we start with the with the second presentation I will go quickly over and so in this presentation I want to discuss two two points that have been discussed during during the hands-on in some of the room it's what do you do because most probable works with 20 variables let's say 25 if you have access to a cluster and lots of time but not more than this so if you have a problem where you have more than 25 variables then you need to have another solution the solution is to use heuristic you have no guarantee that you will have the maximum so the best scoring bag but at least you have you can optimize your dag in order to have some high scoring dag and after I will discuss the robustness of of the dag uh using MCMC over structure here is a list I don't want to speak about that it's it's the list of possible thing that is not discussed in this tutorial but still could be considered as an advanced feature of ABN so up to now you did uh work with the exact search so you have the guarantee that it works and that it works that it give you returns you the best fitting dag now um to give you an example so this is an example of an algorithm that optimize the dag using greedy hill climbing so sorry so you start with with a given network so so for example uh you start with a with a empty network at each iteration you evaluate all possible change so for example you you you see here for example you have a random dag then you change you change the direction of one arc or you add an arc or you delete an arc and the score will change and um you apply the the change that lead to an improvement of your score and you do that again and again and again and again so we will see that and you stop when you see no improvement of the score and we will see that indeed i mean in detail there will be an exercise in the in the handsome uh to show you how this works and and how you plot the results um of course it comes was with pitfalls so there is a risk of being stuck in local maxima uh there is a risk of not uh getting out of plateau and for those and i don't want to enter in the detail but you have uh many possible uh algorithm so you have the tabu search with this card so so for example at each step you record which move you already did then you know that you you are not going to do this move again so you you are forced to move in in another place where you you've never been uh so there is another solution it's the random restart so at a certain point you you you you restart your search and there are some more um complex algorithm which for example some of you may know the simulated anne league okay so i go quickly there um now i want to speak in the 10 last minutes for this uh tutorial i uh want to speak about the robustness of the output outcome of an abn analysis so the trustworthiness so here it's an example of of one uh beijing network i i did uh in uh so here it's the it's the it's the paper we published this material so so you have this analysis so now and i think everyone who do in practice beijing network analysis you will encounter this problem so imagine you present that to your peer your supervisor or or a subject expert matter and suddenly he say yeah but look i don't believe that this arc is plausible from a biological perspective or you say yeah yeah i mean this this is very unlikely based on my experience but why don't you have an arc there i mean um we know for sure for the last 20 years of research that there should be an arc here or worst-case scenario you start to draw on your on your on your output and say no no no no no we we know for sure that uh there should be something here nothing from there and and yeah and here you are stuck because i mean there is a software that give you uh a DAC you did all the math correctly so you you prune you you bootstrap you you do everything but still people are not convinced and ask you question about what is what is do you really believe that this arc should be there and and and you want to discuss this so the idea you have when you do a Bayesian network analysis or abn it's that the best possible dag so the one you present it's there is a huge gap between this dag and the alternative network so this is the idea we all have in mind yeah so we identify the best possible dag and and it's it's best in the sense that it's well separated from all the other possible network but what about the situation where you have a competitive network so a network that score quasi equally well which is very different but have a a a smaller score so in your paper on your research or or to your peers you present this uh network but this network visibly count also because it it quasi does the same score but with a very different interpretation so what do you do and one possible solution for for addressing this issue it's to do the so-called MCMC over structure so we will have a small worker like this and the idea is again like we have in the heuristic search it can move from structure to structure so we can add an arc delete an arc reverse or shuffle part of your network and the idea is the uphill steps are always accepted moderate downhill is usually accepted and you almost never accept the steep downhill and by letting this worker moving from structure to structure you can sample the posterior distribution of all the possible network so here you have the dag score in in this part of the graph the gray zone here is the burning so this is something you discard after because you you think that your MCMC chain chains so MCMC stands for Monte Carlo Markov chain so Monte Carlo it's a it's a it means a random and Markov chains means that the the next step you are going to to do is only influenced by the last step so you forget about the past so the MCMC chains so so here you can see that we have I think four MCMC runs so here yeah and you can see them moving from dag to dag and what you have here it's the posterior distribution of all the possible model and and here in the in the in the zoom out this is the same but with a with a much much larger number of steps just to show you that it converge to the to the maximum okay from this distribution what you can do is instead of presenting the best unique possible network where essentially an arc is a zero one relationship so you say there is a relationship there isn't a relationship instead of presenting a zero one dag you can start to have all your network stack one to the other and you can count the prevalence of the arc and by doing that you can do the so-called structural queries so essentially you can start to compute the probability of having a certain structure so what is the probability of having an arc or what is the probability of one node being in the neighborhood of I don't know which node or more interestingly I think what is the probability of not having a relationship between two nodes so so the MCMC over structure so the idea is select the most probable structure control for overfitting so so with this so this is implemented in MCMC ABN so it works well with the output of ABN so the idea is you can select the most probable structure so you can use that as a heuristic tool you can control for overfitting because as you have access to this dag you can start to select which arc are needed and from each arc are not needed but most importantly this is the dose line here by sampling the landscape of high high scoring structure so in an applied perspective you avoid to reduce the richness of a Bayesian network modeling to only one structure so you don't present one Bayesian network you present the entire posterior distribution of the Bayesian network and you and then you can quantify the marginal impact of a relationship by marginalizing over structure so it means that by having all those dags you can have an idea of what is the effect of of one variable over another in the global context of all the possible network so yeah so now we will jump on the hands-on exercise so yeah what I propose is we can we can do as before and I will we the TA will will try to pass from from one breakout room to to another Reina do you know if the the co-host you you you put last time they are still co-host or they're still co-host that didn't take those away and so hence I would reassign to the same breakout rooms there have been some leaving or not so the breakout rooms may not be as equally distributed as they were before but in interest of of time we keep it that way and then I would open all the rooms correct nothing else to add from your side shield no nothing yep perfect and we reconvene here approximately at 48 yes about 20 20 minutes until reconvene yep yeah which document is for this one sorry which document is for this ah yeah so so excellent question if you go to the website there is so the website is sorry I'm I'm so if you go I close it so if you go to the to the to the website of the of the tutorial there will be second hands-on and and here you have again the the the html page that you can discuss or and you have a zip file next to that where you can reproduce the results okay um could I just ask a follow-up question about the lecture yeah sure sure so in in the results for the heuristic modeling you would present all the different probabilities for the different arcs not just one final model so so you mean for the for the advanced feature so instead of presenting the best fitting dag with the with the with the with the arcs your question is are we going to present all the possible probability is it correct yeah yeah so I think I mean I mean of course it's possible I think this is not meant to be done like this in the sense that I think the most useful here information is about the dedicated a question so for example if you know that you're you are super interested into the relationship between your variable a and b here it works to speak about what are the probability to be to have an arc and what is the the the because another question could be for example if you have another variable c that could modulate the presence of an arc so you can start to ask those kind of question to the to the to the to the to the data but I would not go for presenting the entire probability or or maybe in the annex for example with uh with uh with the mattress yeah so I propose that we we go in the breakout room and and for those who has a big question about lecture you can stay there and ask if you want so sorry a practical thing I think in the folder in github I don't I can't find a second document hands on hands on too ah yeah but so yeah yeah so give me one second can you see my screen I think so so so here if you go to the website yeah so so you've been there correct so if you if you so this is the the html page for the second ads on yeah and where is the rmd if you go to the zip folder I really hope that I put that there otherwise I need to quickly change it's in there I have it from downloading okay so if you download this inside you open and you have the rmd here okay and and normally if you execute that in this uh you you will be able to reduce the html page okay thanks uh can you put again the link for this uh so which one is it the the last one you put yeah uh so the links are in the chat and you would just take then the corresponding slot uh for for the corresponding time slot yep can you can you see in the chat of this tutorial yeah I can see the chat but I can't find the the folder the second folder because when I open the last link you sent I only have the yeah so so yeah so if you look at so look at the screen I I am so if you if you go to this it's the zip folder from the second so the advanced feature and yeah yeah and the link is I don't have the link is there so the link brings them out to the root page and okay now I found it okay thank you so much thank you very much