 The tools of Bayesian modeling to understand how, you know, as a kind of like a normative reference point for perceptual processes and kind of like that was kind of our main framework. We discussed kind of this general approach of Bayesian modeling in terms of how do you set up the problem when you have an inference problem you start by building a forward model of the world which involves essentially giving a story like a probabilistic explanation of how the data could arise from hidden or latent states of the world and then we saw how to invert that forward model to take some data that you actually have from some measurement and then use it to extract some knowledge about these latent states of the world and then we saw how to apply this technique to a perceptual problem. Now we're going to kind of go back to the perceptual problem in a second but just before we move on I wanted to kind of at this time I forgot to give you so this is an important reference is a general reference that you can use for this whole thing that we're discussing it's very well covered in this book that I have put a reference here by Mark, Gordon and Goldreich it's a new book it's I think I believe the officially the book is not published yet but you can find a PDF from the author's website so this is if you want to learn more about this type of approaches this is a great reference that you can look at. Okay so last time we were talking about this problem of inferring a continuous variable so we were looking at this kind of auditory localization task right and we were essentially we finished the at the end of the lecture we arrived at the derivation of the posterior so of the posterior of the subject so we got the the fact that the you know after performing our Bayesian inference we got the posterior over the stimulus given by the given the measurement was going to be essentially actually let me write it so let me write it in a different way so we wrote that essentially the posterior on the measurement given the measurement was going to be distributed as a as a Gaussian centered around some u p with some standard deviation sigma p where essentially the mean of this of this of this of this Gaussian was given by x over sigma squared plus mu over sigma squared over one over sigma squared plus mu over sorry plus one over sigma squared essentially there was this idea that the standard deviation was going to be something like one over sigma p squared equals to one over sigma squared plus one over sigma squared so the idea was that the center of the the mean of the posterior was some sort of weighted average between the center of the prior which is this one and the location of the measurement and the weights that were going into this weighted average were essentially these factors one over sigma squared that we could also call for instance one over sigma squared we could also call it j and one over sigma s squared we could call it j s is a notion of it's an inverse in inverse variance is a notion of precision of these uh Gaussian distributions right so the idea was that essentially this integration process between prior knowledge or beliefs about the stimulus and the sensory information was weighing the two the two elements that we were integrating according to some notion of you know essentially how confident we are in the two in the two elements okay now the one thing that I think the next thing that we had to do was essentially so so if you reflect here we have what we have is we have derived a essentially a you know the posterior probability distribution for the stimulus right but the point is that you know we still have to derive uh something that would tell us what the behavior of the subject is right so we would have to derive a readout that I think you should remember from last time we discussed this readout as a mapping from a posterior probability distribution to an actual value uh of the uh of the stimulus and last time you should remember we discussed one particular one particular type of readout which was the maximum posterior readout which is a very convenient and very natural choice for a readout which is essentially just saying oh if you have a some probability distribution you pick the value that has the highest probability right so this is a great choice especially for discrete probability distributions because in some sense picking the the value with the maximum probability as there is a sense in which it's essentially it's trying it's the same as saying that you're trying to get you know you're trying to you're going for which you think is most likely to be the correct answer right you're picking the the most likely value however this doesn't necessarily make as much sense for continuous variables like the one that we have here where s is this you know this notion of the angle of this of this auditory stimulus because in some sense if you think about it getting the correct answer for a continuous variable is in some sense mathematically impossible right because if it's a continuous variable you kind of have to get it right at infinite precision right so there is no sense in which you can actually get it right um so uh the you know kind of the you know if you want to generalize this discussion a little bit we we can say oh we want to derive a readout that allows us to pick an answer that gets close to the or as close as possible to the true value of the stimulus and then the idea is that you know you will get the specific form of the readout will depend on exactly what you mean by close right exactly how you're going to measure your error essentially with respect to the original value of the stimulus um so uh so there are multiple ways of doing this but one very simple approach to uh kind of characterizing the error is just measuring the error in terms of the square distance right square distance so um you want to for instance you want to pick um s hat pick some uh response or report s hat uh such that essentially minimizing over s hat the expected value over the posterior over s given x of say s hat minus s square okay so does this make sense for everybody the idea that you know you don't know what s is but you have an idea you have like you have x right and you can choose s hat right and and you don't know what s is but you have an idea of kind of a probabilistic idea based on this posterior distribution that you have just computed using Bayesian inference and what you're going to do is you're going to pick the s hat that minimizes this notion of of error right and this is a very kind of a standard notion of error is a square distance um now it is very easy to show it's basically like if you take uh just a matter of taking a you know if you you try to minimize this expression with respect to to s hat is just a matter of taking a derivative of this of this expression with respect to s hat and uh you you're gonna see to see that this corresponds to choosing the mean of the posterior we can actually do it very easily so if you consider derivative with respect to s hat of that expression over there i should have me write it directly as an integral over the s of p of s given x times um s hat minus s squared so the derivative of this expression here so this s hat has nothing to do with this part of the expression so it just comes inside the integral and it becomes just integral over the s p of s given x times two times s hat minus s okay and then we want to put this equal to zero in order to minimize uh this notion of error but then if we put this equal to zero we can just drop the factor of two and then we can kind of separate this expression by taking one of the one of the terms here and taking it to the other side of the of the equal sign and this becomes um so this becomes integral in the s of p of s given x of say s hat equals integral in the s p of s given x of s okay but so here s hat has nothing to do with s so this just comes out of the integral this is just the integral over s for probability distribution in s so this is equal to one okay so this means that s hat is equal to this is just the definition of the average of s according to the condition of probability distribution of s given x right so this will be just sorry this okay so uh and this is what we call the posterior mean okay so if you want to minimize the error in the square in the square distance sense you you're going to pick the the mean of your posterior distribution so um in this case this is actually very easy to compute this mean because we know our posterior is a Gaussian and actually we have already written what the what the mean is so that means that in our case s hat will be simply equal to mu p okay which is that expression uh over there um note that of course if you change your notion of error you're going to get a different readout for instance if you wanted to minimize the error in the sense of a say of an absolute absolute value of the distance you're going to get that the the optimal readout is a is the posterior median um and uh and so on and so forth note also note also that uh in our specific case because our posterior is a is is a Gaussian it actually I mean this is just kind of like a pretty uh I mean it doesn't doesn't really matter that much because the mean of the of the distribution the median and the mode which is to say the maximum posteriori readout would all coincide because of the the mean of the posterior is also the median etc etc but this is like the gen in in general sense this is what you want to do um okay so that's great now we have essentially a deterministic mapping that tells us whenever the subject receives a measurement x what is going to be the answer the report of the subject right the report of the subject will be this particular value here which is just a deterministic function of the measurement okay um quick um but now we're still we're still missing yet one more component to model exactly what's going on in the lab because um in the lab uh we only have access to the stimulus s and the response s-hat now what we have is a theory that tells us you know what s-hat is what the response is in terms of the measurement x but we don't know what the measurement is right the measurement is something that happens inside the head of the subject um so we need a way to connect um this uh this response to the initial value of the of the of the of the of the true the true value of the stimulus right so in practice what we want to compute is the what is called the the the response distribution response distribution of s-hat given s okay um now uh again fortunately this is quite this is quite easy to do uh because if you if you notice that so now let me write s-hat here for simplicity okay so you notice that s-hat is this very simple function of x so this is just a linear function of x right because these these terms here are all constants so mu is just the the mean of the prior sigma s is the width of the prior sigma is the value of the sensory noise etc etc so this is just a simple function simple linear function of x so whenever you have x which is a this is a random variable it's a Gaussian random variable and then we have s-hat which is a linear function of a Gaussian random variable whenever you have this situation your your s-hat here will inherit the variability of x-hat and will itself be a Gaussian random variable right because whenever you have say y equals ax plus b with x you know say random variable uh distributed according to some mu and sigma uh you will have that y is also um some uh some Gaussian random variable distributed according to a mu plus b and the standard deviation will be uh something like a sigma okay something like this I think this is to be I mean is this kind of clear for everybody I mean this is just you know if you take a Gaussian and then you kind of use you scale it and you kind of shift it the result is still a Gaussian only with that you know centered around the value which is shifted and scaled and with a standard deviation which is scaled okay um great so let me just keep doing here so now if we do this calculation for the actually maybe there is some more space here um uh if we do just this calculation for for our case over there we get that the um actually let me so we get that s-hat will be distributed according to some uh Gaussian random variable of mean mu s-hat and standard deviation sigma s-hat where mu of s-hat is equal to um so this would be um okay so this would be s over uh sigma squared plus mu over sigma s squared over one over sigma squared plus one over sigma s squared we can also write it using those that j notation over data precision notation we can write the j um s plus js mu over j plus js okay and the um standard deviation of the of this conditional distribution let me so look at this so the standard deviation of this conditional distribution will be uh simply so that the transformation is over there so it will be something like um so uh sigma sigma s-hat squared would be equal to um one over sigma squared over one over actually let me use j sorry so it would be j over j plus js everything squared times uh sigma um sigma squared okay but then uh sigma squared is just one over j right sigma squared is one over j so this we can write it as j over j plus js squared okay um so this is great because we have um i mean i think this is is this the artist passage is fine for everybody we just uh you know just we have a Gaussian and we're just transforming linearly and this is what we're getting um okay so one thing that i want to focus on now so here what we have is really you know we we have just computed this probability distribution right p of s-hat given s okay so why is this why is it important to connect these two quantities right so i just want to stress this i already said it but i want to stress it it is important to build the model that connects these two quantities because these are the only quantities that we can actually have access to in the lab right these are the only two things that we measure and the kind of the general idea that we're going to see more in detail later but the general idea is that so this is the model for the data that you record in the lab and this model will have some free parameters right in this case can anybody tell me what the what the free parameters are for this model from the way in which i describe the experiment remember in the experiment there is a prior for the for the measure for the for the for the stimulus which we choose as experimenters and then there is this process of the generation of the measurement inside the head of the subject and then there is another process of of Bayesian inference and then there is a report right and this is the model for the report given uh the stimulus can anybody tell me what the what the free parameters of this theory are up to this point i'm sorry yeah it's it's related to the so yeah the the measurement itself is not quite a parameter the measurement is more like a variable right but it's connected to the measurement right what would what is the parameter that that is unknown and it is related to the measurement according so let me just rewrite because i don't have it on the blackboard but so the probability distribution of the measurement is this right according to our theory mean and variance it's actually just the variance because the mean according to the theory the mean is just the true value of the stimulus right and that we know right is just the variance so and actually sigma is the only free parameter in our theory is a very very simplified theory of course it's a very very simple model uh but so the idea is that you know if we want to apply this model to explain the data we may want to you know uh for instance use this uh this uh this this this this probabilistic description of what we see in the lab and essentially fit this description by tuning the free parameters in our theory which in this case is the sigma okay so it's kind of general idea but we're going to see it more concretely in a second so now i want to kind of draw your attention to another thing so until now we have discussed um a bunch of different uh kind of probabilistic objects um let me see what is a good location for okay i mean maybe let me delete this yes let me check if it's okay this one second it's turning on and wait wait wait wait okay it takes a few seconds and one two three go i if you want to model memory in your purse in our perception can we say the measurement is a hidden variable that includes the effect of memory in perception or we should have another parameter yeah so um i think typically what you want to do is for instance one good uh i mean depends exactly what you mean by memory but one good one good way of modeling uh memory could be for instance to include what you think is the memory somehow in your prior right so because you know what if you have some knowledge about your environment comes from your previous experience and you want to characterize that as memory you basically want you can build you can build that information into your prior and they can't be say uh given measurements uh stimuli and memory are completely uh independent given the measurement stimuli and memory can you can you i mean if you have the all the information about the measurement about the x then the random parameter for the memory for example m yeah and this stimuli is completely independent random variables it depends on how you set up like so basically you're serious thinking of something like where you have a something like it's gonna look something like this yeah and yeah i mean um yeah as i understood from the first uh session x is the uh is the measurement that maybe we can say all the normal activity in our brain so if we have all this information there's not there's nothing else that we call it memory so we have all the information from memory in our measurements yeah so i think you can actually there is probably a like a sensible way of setting up of setting up a model that kind of looks like that where you have essentially a memory contribution which is which is essentially independent of the of what the measurement is and what what the measurement is on any given on any given trial right um so that's the yeah i think you can you can do it that way yeah thanks um okay so i wanted to other questions no question i wanted to uh maybe make a little drawing because we've been throwing around a little bit of notation and um okay um and i wanted to kind of compare three quantities that we've been looking at so imagine consider uh the following so imagine this graph where we put on the x axis we put the amount of sensory noise sensory noise which is again our free parameter is sigma okay is the thing that we don't know about is the degree to which the variability of the of the measurement for a fixed value of the stimulus okay and then i wanted to uh plot three standard deviations standard deviations here on this plot so the first one is just a standard deviation of the likelihood function right the likelihood function uh is essentially remind you is just this conditional probability seen as uh seen as a function of s right but we know that the standard that standard deviation is severely it's exactly it's exactly what we have on the x axis right so this um right so this is you know the standard deviation of the likelihood is the same thing as you know the the the standard deviation of the of x in in this conditional probability distribution uh and so this is kind of an entity line okay um then we have and i'm going to ask you initially to believe me that this is how these functions look and then we're gonna try to interpret them together so uh we're gonna have that the um the value of the the standard deviation of the posterior does something like this so i'm sure let me write so this is just sigma this is what we have called sigma p okay and then we have the standard deviation of the response which actually does something like that okay response so this is what we have called sigma s hat okay so you can actually if you look through if you look at for instance this expression here that i have somewhere uh and if i if you look through your notes and the the previous expression for instance that we derive for for sigma p they behave like this okay so um you know take this as a given okay and you can check this is the case uh i wanted to see if you know beyond looking at just the formulas if uh you know we can kind of get an intuitive sense for why uh this is the this is the behavior so what is happening here is that as you increase the amount of sensory noise in your model um the the posterior the the variability of the posterior at some point just you know initially it increases as you increase the amount of sensory noise and then it just stops increasing okay and even perhaps even more possibly as you increase the amount of sensory noise initially the variability in the response will go up and then it will go down and it will actually end up going to zero okay so um apart from the i mean of course you can check that this is the case in the formula but apart from that can you can you think of any kind of intuitive reason i mean intuitively what is happening here can can anybody give me a kind of an intuitive sense for why this could be the case why does the variability of the of the response goes to zero as i increase the amount of sensory noise and why does the variability of the posterior stop increasing when the sensory noise is very large um maybe uh you can grab the yeah sorry no i don't have an answer just reasoning but if like the sensory noise is noise sigma increases a lot is like the the the subject whatever the noise that the signal is perceives like random noise basically and um so yeah that's a fact that's definitely like a good uh accepting the good direction yes uh so once okay let's assume that the the subject just says whatever nonsense when perceives whatever nonsense whatever the stimulus is then the posterior standardizations no i don't know i just it was already like a step in the good direction anybody else wants to add something so remember that our perception our inference process is a process of integration there's not just the the sensory information right we're integrating the sensory information with the prior okay as the sensory noise increases and remember the subject has we hypothesize that the subject has a good internal model for how their own kind of their own you know measurement works okay so as the amount of sensory noise increases this means that the kind of the reliability of this of the measurement kind of goes down okay so because you know if essentially you know in the limit over here you're you're you're getting blind and deaf right you know that you you cannot rely on your senses like your senses give you no information at all so essentially what you what what what do you have left when you have no sensory information just the prior the model you have exactly exactly just a prior so just your prior knowledge about the task so essentially in this limit here the your posterior is going to be just your prior you're going to be okay like okay this this sensory observation brought me no extra information after getting my observation i know exactly as much as i knew before you know getting my observation so my posterior is going to be exactly the same as my prior and that's why this asymptote here is actually just the variance of the of the prior okay this value here um and by the same token the variability of the response then goes down because why should my if i if on any trial my posterior over the stimuli is always the same is always equals to the prior why should my response change i'm just going to essentially respond according to what i think the you know according to my prior so according to what i think that a priori what is the most likely stimulus so essentially the response in this in this limit here my response is always going to be say i mean if i ever say a prior centered around here my response will always be kind of like the most likely or you know the kind of average of the of the prior right does it does it make sense okay okay um good so this is just for a little bit of intuition um and okay excellent now i wanted to show you a little application um that was a new time um great a little application of of this kind of this procedure in an experiment here this one so yeah okay for people on zoom i switched to my i'm still sharing my screen so you can look at the shared uh monitor so this is just a summary of uh some figures from uh from a paper this is a fissure and peña in 2011 um so and this is an example of you know studying perception using these tools that we have developed so this is this paper is about integrating a prior with some sensor information um so this is a study in barn house the little animals that you can see here there's a figure um so the idea of this experiment this is a sound localization task very much like the one that we have used as a as a running example so the idea of the task is that you have you know the owl is sitting there and then there is some noise coming from a certain angle and the owl will turn their head towards the noise okay so with the click or i don't know i forget exactly what type of sound is it's a sound so the owl will turn their head and um by using this behavior they use it to estimate you know what what is the location that the owl thinks the sound is coming from okay um now sound localization is very important for owls ecologically because they use it as part of the senses that they use to locate their prey when they're out hunting um now one interesting phenomenon that people observe in this type of experiments is that so you can see it here representing this graph so here on the x-axis you have the true location of the sound in degrees so here's zero being the central stimulus and then you go out to the right and out to the left and on the y-axis you have the location reported by the owl okay so you can see that nearby the center the owl estimates very accurately the location of the stimulus but then as you go far out to the right or far out to the left there is a systematic underestimation of the angle so the owl will tend to report an angle which is smaller it's kind of closer to the to the central midline okay so one possible hypothesis that you know that uh that that one could do about you know to explain this behavior is that uh these animals have a strong uh kind of central prior for where they think that the sounds should come from okay because if you think because of you know even if you if you think about it in in in in the invasion terms given what we have said if you have a strong prior they center their own zero your response would be biased towards that towards that location um and essentially in the you know in this paper they kind of try to test uh these hypotheses they also did other things but um so the reason why ecologically these hypotheses could make sense is that uh essentially during you know during hunting uh so people have actually tried to quantify um during hunting behavior the kind of relative position of the owl and the prey and they have seen that during like kind of most of the kind of approach of the owl to the prey so during most of the time during which uh kind of sound information is useful for for its ecological purpose the the prey is actually more or less in front of the owl right the eye is because the sound information is useful while the owl is approaching so the owl is not flying at the right angle with respect to the prey it's kind of flying in the general direction of the of the prey so typically in you know in these useful moments the sound typically comes from right in front of the owl so that's why the you know this kind of makes sense as a ecological hypothesis is there a question yeah and yeah sorry the the question is is this uh or question or statement is is this true for for any for any animals I mean this is yeah this kind of makes sense the the different thing about owls is that they you know the idea is that they you know they they're nocturnal animals they rely quite a bit on their sense of sound to locate prey and so on so they kind of make for a good uh test uh bed for this hypothesis um and in fact I mean similar biases I mean apart from the hunting part the similar biases like like this underestimation thing here has also been reported in other animals including humans um yeah yeah yeah yeah uh and yeah these panels down here provide some sort of quantification to what I said with you know about the relative position of the owl and the prey that sorry Jenny just just double check if I'm getting this correctly so you so here there are maybe multiple causes for these change independence right so you were saying that the prior is responsible yeah what is that the hypothesis is that they have a strong prior that is centered like but they could have also not an alternative hypothesis could be that they have a uniform prior about the angle but that their way in which they produce a response depends a lot on the fact that they have to turn at their head yeah like a cost a sense of a cost of the fact yeah um that there is there is also an alternative explanation I guess the fuck is at owls that they I think they they they they they can turn their head very easily compared to other animals they're apart from the case uh and uh I mean logically yes logically that things could contribute to yeah yeah exactly exactly uh other question uh from the back uh based on the uh uh explanation I think uh uh sigma of s hat should be approximately equal to uh the sigma of the model right sigma s I'm sorry you said it again so uh when the sensory noise increases to a great extent yeah yeah then based on what we discussed uh sigma s hat should approximately equal to sigma s now sigma s hat should go to zero should go to zero because if you think about it this way you know sigma s hat is is the variability of your response trial over trial but if trial over trial you have no information right there is no sensor information the trials are all identical right there is nothing that changes from one trial to the next does it make sense but uh since the a priori distribution is already known yes yes and that is uh the distribution would I choose something from the distribution or would I just take the mean value yeah so that that's that okay I see I see that so that's according to the to the readout rule that we have written here you would pick the average of your posterior which is you know which would always be the same value you're so you're not sampling from your posterior you can also do this type of models and and assume that your subjects are sampling from your posterior instead of saying that so that's another type of readout uh that's yeah that's another valid way of thinking about things in which case you would have yes they would you say exactly exactly thank you um so thanks so uh back to the house so the idea here is that uh so you know to test the hypothesis they do exactly you know type of experiment that we discussed the only difference is that essentially they have one extra so they have two differences one is that they have a one more free parameter so of course they have the sensory noise is a free parameter because like us they are not inside the head of the house they don't know how much that is and the other one is essentially a parameterization of the prior because they assume that they don't know the prior and they want to infer the prior of the animal and they have it here it's what is uh schematized here in the in the drawing is this you know there's this this distribution that is schematized here there's this dark line here for instance is an example of one one one particular uh possible um kind of value of the prior um and uh so this is the first difference with what we have and the second difference is that the uh distribution of the measurement so remember in our case it was uh this this p of x equal uh given s was you know a Gaussian uh centered around the true value of the stimulus right okay what they have in their case is that their measurement is actually centered around some deterministic and known function of the stimulus which i'm going to comment which i'm going to explain with you so basically that's the second difference there is an f there is a known f here um and the known f is this function this kind of sinusoidal looking function that they have here so to explain where this comes from um so i have to say that so when you're actually locating a sound one of the main mechanisms for locating a sound is uh what is called the the inter-arrow time difference so essentially the difference of time that it takes for a sound to for instance hit your right ear with respect to your left ear so if it's a sound arrived arrived first to your right ear then to your left ear it means that the sound is somewhere on the right okay so that's kind of the general principle and there's the sense in which uh so the idea is that if this is the kind of the main mediator of the measurement because if this if the transduction from the location of the sound to some sort of neural representation of this location of the sound should has to go through this uh inter-arrow time difference um essentially you can imagine that uh so this should kind of enter here right so um the reason why this function has this shape the is that so this is the the function here is the on the x-axis you have the location of the sound and on the y-axis you have this actual time this this time difference um so uh imagine that you have um so a sound that is located to the you know centrally in front of you and then the sound starts moving a little bit to the right of course it will start arriving a little bit earlier to your right here okay so it for for kind of smaller angles there will be this kind of linear it turns out to be linear relationship between the location in angle in degrees and the actual time difference but at a certain point and the specific location of this certain point will depend in a weird way on the location of the size and the shape of your head at some point you know if you think about when the sound actually goes behind you if you keep increasing the angle then the the the time difference will start decreasing right because it has to start decreasing right so this is the reason why this function has its general shape right if you increase the angle going away from zero it goes up and then it comes down okay so that's kind of the general shape um and so this of course is a complicated function and what they do in this experiment they actually measure it actually I think they use some measurements then in a previous study but the idea is that you put two little microphones in the year of the of the owl and then you play a bunch of sounds and then you measure the the internal time differences okay and this is the plot that you have here is measured here in dark okay um and essentially so the idea is that then instead of s here you just have f of s where f is this kind of sinusoidal looking function um excellent so I think maybe there is a I don't okay I thought there was a it was a question um so uh then the idea is that so they they they perform this experiment and then they fit the model so again the model remember has two free parameters one is the amount of sensory noise the other one is the shape of the prior and what they get is that they get some level of sensory noise and they get a prior that looks like this so indeed they look a prior that kind of seems pretty centered and pretty narrow around uh the kind of straight straight in front of the out and you know with this with this particular model fit what they get is that the model reproduces quite well the behavior of the animal which is this here so here you have that the dark gray so the black is the model the black squares are the model it's pretty hard to see let me see if I can zoom in yeah okay maybe this is better so the the dot is still still not that great uh so the the black squares are the model and the the the the kind of slightly lighter dots are the measured behavior of the animal so you can see that the the the model fits the behavior of the animal well but okay when that's kind of more like a sanity checker essentially we're fitting a a behavior which is kind of slightly sinusoidal in shape and we have two free parameters you know it's not surprising that a model with two free parameters can reproduce this type of behavior the cool thing about this experiment is then that is the then they use these two parameters that they have inferred in this condition to make a parameter free prediction in a different conditions uh in a different condition so what they do is they actually modify the shape of the likelihood function so they go and modify this f by actually using owls where they have removed this the facial rough which is this set of uh of feathers around the head which essentially helps to kind of shape the sound is like the equivalent you know owls don't have external ears but they're basically they're the equivalent of their ears like taking away your ears and this of course changes the shape of the head and so it changes the shape of the interarital time difference function right so it changes the shape of this function that enters here in our model uh but the cool thing is that you can actually measure this function so this is not a parameter of your model is this something that you can change and you can measure and then you know just by changing this function here you can then make again predictions about what should be the behavior of the animal if the other parameters that you have learned in the previous iteration of the experiment stay the same so if then if the sensory noise stays the same and if the prior stays the same because those two parameters have no reasons for changing just because you have plucked uh the feathers of the from the face of the of the owl um and then you can essentially formulate this parameter free prediction about the behavior of the animal and you can compare it to an experiment where you take these owls without the facial rough and you make them you know locate the sound and what happens is that oops sorry about the checkerboard and what happened is that uh I can show you is that the again you have an excellent agreement between the the the behavior of the animal and this parameter free prediction accounts from the theory so uh and this is really the kind of like the cool bit of results that I'm showing you right um I mean the here the the idea is that this result is really showing that is really lending kind of support to the idea that these owls have this kind of narrow prior centered in front of them and they are integrating it optimally with their sensor information according to you know and the this process of integration is well described by what you would expect in the Bayesian setting um comments questions on this yes here microphone misperception when the location of the source the noise source is in angle zero or exactly 180 we can easily try when you when you stand on a train line and close your eyes it's really hard to notice that train is in front of you or you know I'm behind because the distance of your is exactly the same and I didn't see here that they report 180 because 150 is about the same but from the model can we talk about this huge error when the angle is 180 and describe from the uh yeah from our perception model what happened so so there is actually an interesting thing to thank you there is actually an interesting thing to say about how the model deals with this because so yeah it is true that uh actually thank you really thank you for asking because it's yes I think it's really important so it is true that from the purely kind of sensory point of view if you just look at the the likelihood function here um it is true that maybe you have a completely ambiguous and maybe it's not going to be exactly ambiguous but let's assume that it's completely ambiguous sensor information coming from in front or behind okay and one kind of advantage of this type of model is that you know it allows you to kind of conceptualize the situation where thanks to your prior information and thanks for what you know about the context you can actually disambiguate this perfectly ambiguous stimulus you see what I mean even in this case even in the limited case that they have here where the angle doesn't go the way to to say 180 you can see that um the for instance the interarital time difference when you are say I don't know what number is this like 120 is the same as when you are at 70 something like that right so the sensory information will be exactly the same but but because that you expect that the stimulus to be in front of you are actually able to say oh of these two actually it's this one right so this is a way in which you can use um your prior to to disambiguate ambiguous sensory information okay thanks other questions about owls okay no more questions about owls um okay so great let me now switch to talking about a different topic where am I going to talk about this different topic here and so up to now we've been talking about um a situation where we have some stimulus and that stimulus gives rise uh to one particular measurement right that we have called x we have one stimulus s and we have a measurement x what I want to say you know I want to say something about the different situation where you have um one stimulus x s but actually you have two different measurements but which we're going to call x1 and x2 so these we're going to call them cues cues and this is the topic of cue combination okay so the idea is that for instance when you're listening to somebody speak for instance you have say you can imagine the stimulus to be something like the words or the sounds they're uttered by that person and the two cues that you're trying to integrate uh would be for instance the sound that you're hearing and kind of the you know the visual information that is coming at you from from looking at their mouth moving and it is kind of like it's easy to understand how kind of putting together information from two different cues in this sense can help you kind of getting at what the what the initial what the initial stimulus was so the reason why I want to I'm bringing up your combination is that so first of all it's kind of like a very natural operation that that the brain is constantly doing so the brain is completely kind of fusing information that are coming from different we would say like in neuroscience we say modalities right the modality is like a sensory modality for instance sound or vision like those are more than what we call modalities so constantly fusing for instance information from these different modalities but even you know from within a modality you're constantly also fusing information from across different channels for instance when you're proceeding color you're you know you're you're you're merging together information that is coming from say you know information channels that ultimately originate with different photoreceptors in the retina for instance things like that um also it is a kind of like a classic domain of application of a Bayesian modeling of perception so there's some cool results that that that we can look at historically important um and um also it's a nice uh it's a nice way of showing the kind of the Bayesian modeling framework at work uh in a setting where kind of the predictions become non-trivial uh and it's not just because we're integrating some priors so because until now we always talked about you know the essentially effect of priors right the fact that the Bayesian framework allowed us to to use priors but now we can also you know forget about priors for a moment and still do interesting things that are afforded to us by the Bayesian approach um so the I'll check for questions okay um so in terms of a concrete example that we're going to use we're going to recycle our our running example here only the difference is that so if you remember the the previous example that we had here is that we had this kind of this screen with some kind of loud speakers behind it right and there was some sound coming at the subject from one of these locations so what we're going to have now is we're going to add some kind of some light stimulus also coming from the same range of locations so you can you can you know I can draw this as little light bulbs I cannot draw of course but this is the idea right you have some some little light bulbs you know and you can have you know the you can have a sound stimulus that could come from from some location and perhaps at the same time you can have the light kind of turning up okay so you have two different you have one location which is the stimulus and then you have the two cues that are coming from from the two across the different modalities um okay now I mean we can just model this uh this uh this setting using exactly the same tools that we have developed last time um so you know to to kind of keep a long story short we have you know remember the first step of our modeling approach was to write a generative model for the data sensor for the for the measurement and we can write this uh as follows we can write that p of say x1 x2 um so actually we can first of all we can write that you know there's going to be some prior p of s p of s is going to be some prior and uh in our case actually in this case we're going to take it as a as a flat prior okay because we don't want to as I said we don't want to do anything particularly interesting with the prior so we're going to say that this is equal to some constant okay so it's going to be a very boring prior so say that it's imagine that it is a constant that goes from this value so say from minus 180 to plus 180 something like that no not the minus 90 plus 90 um and then we're going to have that the um additional probability distribution of the two um measurements uh given the stimulus is going to be as follows it's going to be pure x1 given s times p of x2 given s so this is kind of our main hypothesis and we are assuming it's essentially which means that we are assuming that if you condition on the stimulus then the two uh queues are conditional independent so this does not mean that the queues are independent overall it just means that the variability associated with the abstract kind of neural representation of the information in one modality is independent from the variability of the the corresponding variability in the other modality right if you remember the right time we're talking about measurements imagine that this variability comes from this you know noise in these neural representations and if this if one modality is say vision here what you're saying is that oh the noise in this uh you know for instance representation in your visual cortex is independent from the noise from the representation of the the same stimulus but in your auditory cortex for the sound component of the stimulus so this is kind of like a reasonable assumption I mean you can can make it more complex but there's there's a simple way of looking at it um and then I mean just like last time we're going to assume that each of these p of xi given s is also going to be always our friendly friendly gaussians okay each of these is going to is going to be a separate gaussian um so that's count this is our forward model for the for the task and then the second second step of the inference is just essentially just to apply uh Bayes rule so um what we're going to do it is just we're going to write the probability of the stimulus given x1 x2 is equal to probability of x1 x2 given stimulus times probability of the stimulus over probability of x1 x2 right now um this probability of the stimulus we said that so now we're interested in this thing as a function of s right we're interested in this as a posterior probability of s so um let's kind of get rid of some terms here which we don't care about so this p of s we know it's a constant we have assumed that it's a constant okay so we can drop it because it doesn't really matter it doesn't depend on s we can drop it from our expression uh so here we're going to say constant constant um and this denominator also here is just a normalization factor which also does not depend on s so we don't really care too much about it it's also constant in as a function of s so we can just write that this essentially the posterior probability of s is proportional just to the likelihood just the x1 x2 given s basically in this case because we have a flat prior when we end up with a posterior that is proportional to the likelihood and then because we have assumed this conditional independence we get that this is just the product of these two that we will call kind of elementary likelihood functions so our likelihood is just the product of the kind of two kind of likelihoods for the two individual components great nothing too complicated questions on this on these passages here okay um so this is just the um kind of the the computation of the of the of the posterior but now the interesting thing is that um if you look at the so this posterior here is just going to be a product of these two likelihoods but these two likelihoods we have written them here they're both Gaussians okay uh but then we recognize that from a mathematical standpoint the situation that we are in right now is exactly the same as we had in just the the previous case where we only had one queue but we had a Gaussian prior you remember when we were deriving the the posterior for the previous case just with the sound localization task we did our Bayesian inference and in the end we had the posterior was equal to some likelihood which was a Gaussian times the prior there was also a Gaussian here we don't have the prior anymore because we have a simplified and we have assumed it's a it's flat but we still have the products between two Gaussians because one is the the likelihood for one modality and the other one is the likelihood for sorry for one queue and the other is the likelihood for the other queue um so this means that we can essentially recycle our solution that we had before and we can you know or if you don't believe me you can do it yourself you can basically plug take these expressions plug them into here multiply the two Gaussians together we arrange the exponent and you can find that the posterior of s given x1 and x2 is given by um one over two pi again let's create sigma p for posterior again e to the minus uh what is it the s minus uh mu p over two sigma p squared where we have the mu p this is exactly the same the same uh expressions that we had uh the other the previous example would be j1 x1 plus j2 x2 over j1 plus j2 and then we have the one over sigma p squared which we can also call jp is equal to uh one over uh what is it sigma one squared plus one over sigma two squared equal which is also equal to j1 plus j2 okay remember that kind of ji is equal to one over sigma i squared okay so um so this is exactly the same form as the expression we had we had previously um where we see that essentially the posterior is going to be uh a weighted average uh between the essentially the contribution of the two measurements and the weights again i'm saying over and over the same things the weights of the averaging are given by these precision factors which are essentially convey the amount of information that is provided by each of these uh by each of these individual measurements um so uh we can also the same way we can also compute uh the um the the um i mean the same way the of course the this is also going to be equal to s hat right s hat will also be equal to this because again we're if we're doing a readout which is a posterior mean readout this is the mean of the posterior so this will you know given x given x1 and x2 this will be the value of s hat and then if we want to you know if we want to compute um a uh a response distribution so s hat given s will also be given by some you know mu of s hat uh sigma of s hat um where mu by exactly the same reason we did before mu of s hat will be given by what essentially by this expression where instead of each x we substitute s okay the corresponding s so the for instance the here we substitute the mean of x1 instead of x1 and here this the we will substitute the mean of x2 instead of x2 but the mean of x1 is can you like let's see what is the mean of x1 the what is the center distribution for x1 s better yeah s so x1 is centered around the derivative of the stimulus right so s plus what is the mean of x2 s exactly still s okay over j1 plus j2 okay but here we can we can collect s and this becomes just s okay so in this case we have that um the mean of the response distribution given s will be given which is again is actually the the true value of the stimulus another way of saying this this is that unlike our previous example in this case the response is unbiased can anybody think of why it's actually the response is unbiased in this case what what did we do differently from the previous case exactly our priori is flat so there is no reason why it should be biased anywhere than anywhere else than the true value of the stimulus correct um and um the okay so that's that's uh that's great and then the the value of the variance of sigma s hat um will be what will be something like uh j1 say variance j1 over j1 plus j2 um times sigma squared plus j2 over j1 plus j2 squared uh sigma 2 squared um and this is what this is j1 plus j2 over j1 plus j2 squared which is 1 over j1 plus this is just a simple manipulation of the is it like this in calculation we did last time so um okay that's fine we have a little a little theory that tells us you know what is the optimal way of integrating the two the two different um um q's but uh one kind of disappointing thing is that the only difference that we have with respect to the case with only one q is that you know if you assume a constant prior anyway is just that basically what we're saying is that the okay you should have your variance should be a little bit lower than what you would have with either just you know either only x1 or only x2 um and okay it's because you're just you know integrating information from two sources so hopefully you're getting as you're doing a slightly better job your your variance is a little bit lower but that's not that great I mean it's kind of disappointing because it could be hard to kind of test this hypothesis right um so because I mean you you would have to so if you imagine having to test for this for this hypothesis because you cannot base yourself just on the on the value of the stimulus because the response will always be unbiased as long as your priors flat you will have to kind of measure the variance of the response in the of the subject and measuring variance is typically is kind of harder than measuring uh kind of you know average behaviors um so uh one way that people have come up to come up with to uh test uh for for this you know test whether this uh uh this approach to the striving um q combination is uh you know make sense is introducing what is called what can I write it maybe I can write it here an artificial q conflict artificial q conflict so the idea is that this situation here is what we tell the subject that is what's going to happen and we tell the subject I look there's going to be a sound and a light and they're going to come from the sound location and you need to kind of you know use the sound of the light to tell me where the location was but in practice what we do is act is we play the sound and the light from two different locations without telling the subject so it's more like the true situation is more like something like there is some s1 that gives a next one and an s2 that gives the next two okay but we don't tell the subject that so the subject thinks that this is what's happening and so they will perform um Bayesian inference based on this okay so their solution so their their their posterior distribution and their um uh their so their posterior distribution will and also their uh their report will depend will always depend in this way from the two cues because this dependence of the response of the response on the cues is given by the internal inference process of the subject right that only depends on what the subject thinks the structure of the task is okay so this day is the same what changes is that when we compute the the actual distribution of the response as a function of the stimulus now essentially we're talking about what was the average of x1 and what was the average of x2 but now we know that the average of x1 and x2 it's not s and s anymore it's going to be s1 and s2 okay so we're going to get that in this case um mu of s hat is going to be j1 s1 plus j2 s2 over j1 plus j2 okay and what this means in practice is that it's going to be you can you can you can think of this as essentially that we're introducing a bias um we can you can define a concept of a bias say bias one essentially if you think that with respect to stimulus say number one so if say you can in you know concretely you can say that if the sound and the light they come from actually different location you can express this as saying that oh the location of the sound biases your perception of where the light is or vice versa the location of the light biases your perception of where the sound is it's symmetric you can uh you can write it by the way so for instance you can write bias for one uh for say q number one um of say s hat given s1 s2 is going to be you know you can define this quantity which is like say the expected value of what of s hat um given s1 s2 minus the true value of s1 because say for instance we're looking with respect to s1 and this turns out to be if you compute it is just say for instance in this case is is j2 over j1 plus j2 um s1 minus s2 okay just plug the numbers in this is what you get um so what we're saying then is that in this case in the setting of artificial conflict you expect that if the subject behaves according to our our Bayesian model then you should be able to pick up this bias in the kind of in the average behavior of the subject right which is kind of nicer to detect than looking at the variance um and I wanted to uh quickly talk about do I have any time great perfect um I want to say something about an example where of of this of an experiment that that does this basically uh any question I mean maybe this is going to become a little bit clearer now we look at the example but any question till now question give it a second I should should turn on yeah give it a yeah give it a yeah it's uh it takes a second this thing yeah okay I shouldn't you include the pry this fact that you tell the the subject that the sound and the light come from the play from the same place in the prior and so you should change the model hypothesis in this case yeah so thanks thanks that's a good question which also reminded me that there was another there is another thing that we have to to discuss on this so in a way you're including it so we are including it in the sense that we're in the in the sense that the structure the probabilistic structure of the of the forward model that the subject uses for performing the inference depends on what we told them so we are including that uh so it's just not part of the prior like we're actually telling them look the prayer is flat and you know our prayers are actually flat maybe we're being honest about that it's just that we're not telling them there's actually two different stimuli but you see what I mean so it's not included in the prior it's included in the structure of the probability of the forward model um thank you for the question which also reminds me that I forgot to tell you one thing which is important that if you look at how even just in this case here in another color um even in this case where is it yeah this one even in this case here just the just the simple case um so we see that the one advantage of the of this Bayesian setting is that it allows us to treat integration of information that is coming from two different cues exactly in the same way as integration of information from say the past and the present like before because mathematically the expression we had before for integrating prior knowledge or belief about our stimuli was exactly the same expression as we have now for integrating information that is coming at us from two different kind of sensory parts if you want or any other type of two different cues okay and this is maybe this seems a bit trivial because it's you know this is a very simplified model and it looks very simple but it is not trivial at all that you can actually do this you know you set up a model of perception where you can actually uh do this in such a such an easy and natural way so this is a big plus of the of the setting that it really allows us to manipulate very easily this idea of putting together information that come from different sources regardless of whether these sources are you know our prior knowledge or they're just different sensory stimuli okay so I just wanted to make this point yes question I don't know there's a little button and what's about if I okay let's suppose that I have a prior regarding a specific censoring input and then I have a stimulus that is different for example I have a prior regarding my vision and then I have some noise stimulus the output will be always the same so yeah so let me try to say if I understand correctly the question so the problem is that what happens if for instance my model is wrong basically the internal model of the subject is wrong basically you're saying right so if you have a prior there is you know I expect to see you know a bunch of students sitting in the room and instead you know there is a bear sitting in the corner okay that's like what's the what's going on with that um so um yes so one assumption of the of the theory is that so the theory promises optimal inference and optimal you know it's a theory it's a normative theory promises optimal uh results and optimal behavior as long as as your internal model is correct the moment your internal model is wrong you're kind of all bets are off I mean to some extent right there is a um uh so uh it is an important assumption but you can also um kind of uh uh so for instance you know it can be a so for instance in a way even the example we saw with the owl uh it was an example almost of that right because there was a sense in which the internal model of the owl was mis specified with respect to the task because in the task you know sounds could come from any possible directions whereas the owl was kind of really optimized for expecting sounds to come from the center and you can actually show we didn't have the time to do it but you know you can actually show that that is optimal in the sense that when you have you know when when your prior is correct you actually get you know the total amount of error your total amount of error that you make in your in your in your estimate of the of the of the locations for instance in the case of the sound over the course of the experiment would be minimal by introducing the bias associated to having this narrow prior whereas of course if you um if you uh you know if your prior doesn't match what's happening in the world then of course your your behavior is not going to be optimal anymore but you know in some cases like that one we could kind of infer where the non-optimality comes from by doing something like what they did in that in that work by doing uh by kind of inferring the the part to we assume not to be optimal so in that case the prior or in other cases you can even design the experiment in such a way that it is robust to a degree of of what we call modern mis-specification this this type of thing is uh you call it also modern mis-specification um so yeah there are kind of things but this is definitely something that you you have to think about when you're for instance setting up an experiment in this case absolutely yeah thanks other questions one from back uh I have a doubt about this this whole like likelihood thing like this p of x given s because I've even before when we show that these three likelihood lines my increase no no standard deviation lines my increasing noise and this idea of integrating the prior and so like I as the subject I should know what my likelihood standard deviation is not to because like for instance here like if in these two cues example if I see that every time I've been told that's the this one one only source and I start seeing like perceiving noise and sound two different places do I get uh an information that actually there was a modern mis-specification or just that the standard deviation of the likelihood were very large like yeah I mean that's uh that's the question that yeah that's a good question you cannot really answer that question in in general like in practice I mean if you think about it from a point of view of running an experiment I mean of course on one level there is there are the kind of theoretical assumptions of the model where you say oh the subject knows what their sigma is and they're using it to perform optimal inference then on the other hand in practice uh you know you have to see you know exactly if they how to what extent that holds in in in practice but the um uh so for instance in the case where you actually get uh so the example you made where for instance you are the artificial queue conflict and the the queue conflict is so big that the subject can see that you're you're tricking them I mean at the point that the assumptions break down because the the subject is you know the subject sees you know a flash of light over there and uh and the sound coming from over there they're going to start to think that their their internal model of the task is wrong and then I mean then it becomes hard to I mean unless you're you know if you have a specific theory for how they should you know they should learn so you there's also a whole like field of uh you know structure learning so you know studying how people learn the probabilistic structure of you know the world and so you know if you if that's your objective that you can actually study that and see how they update their belief about how the data comes to be but that's much more complicated than what we're talking about here but yeah yeah thank you um okay so let me just very quickly unless there's any questions yeah thanks um okay let me just very quickly talk about this experiment so here okay let me make a drawing first um so this is an example of uh from this paper here a lesson there 2004 um where so the situation is again similar to what to our running example it is also a multisensory kind of localization task uh it's slightly different but um so the idea is that um so this is kind of like the front view so now imagine being the subject and you're looking up at the screen so this is the kind of the screen uh and so you have in front of you this kind of panel which I think is like some sort of like translucent glass or um plastic thing where the experimenters can shine a light that produces some sort of Gaussian blob of light okay so it's like this Gaussian blob of light that could be either very small and sharp and very well localized or big and broad and kind of hard to exactly tell where it is and at the same time you're also uh say playing some uh some some noise you have a system of speakers around the screen such a way that the experimenters can play a sound at any look at any horizontal location okay so it's a similar setting to what we had here um but the difference is that so there's going to be on any trial actually let me leave this here on any trial there's going to be actually two uh such pairs of stimuli one pair is going to be this one which we're going to call the probe stimulus the probe stimulus where indeed the light and the sound are centered around the same location and then there's going to be a conflict stimulus a conflict stimulus stimulus where as the name implies we introduce an artificial queue conflict so in the case of the probe the probe location can be anywhere so this kind of this you know this location here could be you know to the right or to the left any location and in the conflict case um it's the let me yeah so in the conflict case you always have the light on one side and the sound on the other side okay and so they're kind of centered in a way that if the light is off say five degrees to the right then the sound will be off to five degrees to the left and this difference here in uh in location we call it delta that's the amount of conflict okay it goes from zero when they're centered together at the center and then as you as you increase this delta the light goes off to the right you know and basically on any on any trial the experimenters will play both pairs of stimuli so there will be a conflict stimulus and a probe stimulus and the subject will have to report of course as the subject doesn't know which one is a probe and which one is conflict they're going to be mixed up at random maybe on one trial the one is first and on the other trial the other one is first um and they have to report which of the two they thought they perceived to be uh more to the left okay so basically they have to compare the perceived position of this and the perceived position of this okay so the idea is that you know just to give you first an intuition before looking at the data the idea is that when your visual stimulus is very small and sharp by this theory that we have developed here um because you're weighing when in your in your inference you're weighing your two uh your two cues according to their uh relative noise okay so the the you're going to follow more the cue that is least noisy right you're going to give more weight to the to the to the cue that that has less noise so when your light is very sharp and very easy to localize essentially your your percept for the conflict stimulus will track the location of the light okay and so when you compare for instance say in this in this for instance in this setting here where you have say the probe stimulus which is this much to the right and then say you increase a lot the conflict and say your your light stimulus here is over here and the and the sound maybe is here you're going to say that you know even though the sound is far off to the left here you're going to say that the uh the probe stimulus seems to the left of the of the of the conflict stimulus right because the conflict stimulus will be perceived here okay in contrast when you have a very broad uh and blurry uh you know light blob uh you know when this is very large your percept for the conflict stimulus will track the sound position okay just by without changing the sound just by changing the size and the sharpness of the visual stimulus you can essentially the theory says that you should be able to shift which cue the subject is paying attention to basically because they will automatically uh pick the optimal balance which is like just picking the the component that has the least noise and so in this situation for instance um if the the subject is tracking the sound here uh you will have that the subject will report that the probe stimulus is to the right of the conflict stimulus okay everybody everybody with me did here okay so and okay so basically the summary is that that's what happens but okay and you can actually test it quantitatively so that's that's the idea of the paper um so let me just uh show you here uh so for instance in in this plot that you can see here uh is so this one um this this this function here gives you so on the x-axis you have just the position of the probe okay let me see if I can scroll down and show you the okay displacement of probe okay uh on the x-axis and on the y-axis is the proportion of trials that the probe is seen to the left so the idea is that of course as you move the probe to the right you know when the probe is far off to the right you know regardless of the conditions the probe will always be perceived to the left of the of the of the conflict stimulus and then at some point you know when the probe is always to the right is always going to be perceived to the right the interesting stuff happens in the middle okay so in this case this panel here is where you have this four degrees here is the size of the light blob so is where you have a very sharp light blob okay so this is the case in which you expect from the theory that the subject will track the location the so essentially the perceived location of the conflict stimulus this one will be the location of the of the light okay so and the different lines here are the amount of conflict okay so this is five is when you know the position positive position of the conflict is when the light when the light goes off to the right so what we're seeing here is then when the conflict is positive so when the light is off to the right essentially in order to perceive to start perceiving the the probe to be on the right of the conflict stimulus is not enough that the probe is at zero basically it's not enough that the probe is off to the right of the center it has to go further to the right you see the basically this proportion of times that the probe is perceived to the left only drops at around five degrees right you see this is five degrees so basically what this is saying is that in this setting where the probe is there where where the light is very sharp the probe will be perceived to be on the right of the conflict stimulus only when effectively the probe is to the right of the light okay which is basically what I was saying and then vice versa when you have negative values of the of the conflict negative values just means swapping the position of the light and the sound you have the same thing so it's going to start it's going to stop being perceived to the left for essentially as soon as it kind of goes on the other side of the of the of the light and then again according to the theory again also according to what was predicted by the theory qualitatively if you then make your Gaussian blob of light very large in this case is a 64 degrees it's a very very large blob of light essentially it means that you're in a situation where you're the perception so essentially the perception of the sound becomes more reliable than the perception of the of the location of the light and you see that these lines are now inverted basically we have the we have the complete opposite effect this is essentially just showing you with some data the thing that I was describing earlier okay but this is just a qualitative type of prediction um the nice thing is that because we have a quantitative setting to to to think about these things we can actually make a quantitative prediction and again just like in the case of the owls we we're going to do the same thing we can essentially so our theory has some free parameters and we're going to infer some these free parameters um in a in one particular setting and then we're going to use them to make a parameter free prediction uh to see if the the the the theory work so the way we're going to do the the the the way we're going to infer infer the parameters is that so in our case the parameters are essentially the sigmas right we don't know or the j's if you want the j's are free parameters that we don't know about and presumably there's going to be one value for the auditory j and multiple values for the visual j because you're going to assume that there's going to be a different value of the of the sigma for each value for each size of the of the of the light blob and essentially you can the the nice thing is that you can actually just measure this uh these j's in what is called here the unimodal case so which means that in a case where you you only have one sensory modality so for instance you do the the same experiment only when you just essentially you compare to stimuli we just say one sound right sound to the center sound to the right which one was more to the left which one was more to the right there is no conflict there is nothing and you just use this uh this experiment to measure the essentially the sharpness of the response of the subject right and that and that allows you to infer say the j for sound and then you do the same for individually each size of the of the visual blob and you get all your j's this way okay but this experiment the unimodal experiment has nothing to do with integrating cues right it's just one modality at a time and then you say okay according to my basic setting my subject will when I when I then do the the bimodal experiment my subject will combine information between the multiple modalities in an optimal way and it will produce this behavior here that I expect from the theory but now because I know what the j's are I measure them in the other experiment I can actually make a again a parameter free prediction of what the behavior of the subject will be when they combine the the the the information for multiple modalities and so and this is what is shown in this other panel basically so here each of these three panels is a different subject so these are the old days of psychophysics or you could publish a paper with three human subjects so but it's still pretty good um no this is a very nice paper um so the so for instance here you have the the three lines here are um so here on the x-axis you have the the the amount of conflict okay so this again is the same axis is uh is the yeah so is it so on the x-axis you have the amount of conflict so the the degree to which these two things are shifted um and on the y-axis you have psc here stands for point of subjective equivalence but it's essentially it's the the center of this function so it's basically a measure that tells you you know where is the if you want the the midpoint of this function so it's telling you that so here in this case for instance you could see that when the the conflict was positive the center of these functions were also positive right so the the the function was tracking the amount of of conflict okay so you can see that for instance this black line here is the case where the light is very sharp and you can see that this is just kind of reproducing what we saw in the panel above that when the uh when the light is very sharp the position of this of this function tracks uh the the the the amount of conflict so the position of the of the function goes up when you increase the so kind of shifts to the right when you when you increase the amount of conflict and vice versa the blue line corresponds to the case of the very large uh visual light plot and you have the the opposite effect okay fine uh and the red uh the red case is when you have some intermediate value of the size of the blubber but the the main point here is that the so you have the same thing for the for the for the three subjects the main point is that so the squares here and the triangles are the experimental data okay they are the you know the the measured say position of these of these functions but um the punchline is that the so the lines here they are not fit so they look like they're just fit to these points right they just like fits to these points but in fact they're not so these lines here are actually parameter free theoretical predictions of where these dots should be if the subjects were integrating optimally according to the to the Bayesian uh to the Bayesian like uh they were performing Bayesian inference according to the the the the the values of j's that were measured in the other experiment so again this is the same idea as with the owl before like we're kind of inferring our unknown parameters in one case and then making this parameter free prediction in this case which tests just the idea that the subjects are kind of integrating optimally the evidence that they have at their disposal from the two different modalities and in this case it works fantastically well so that's the this kind of like the the summary of this um okay so I think we're probably yeah we're out of time uh this is where I wanted to get today so any questions and then I think there is coffee coffee break upstairs and I'll be here at 4 p.m. for the series of remote dose so please don't be nice to okay thank you thanks for now we'll just we'll just wait so everyone sorry for a little delay so we are still waiting for a few of the participants in person to get into the room which I hope will happen soon they're coming good Congrats so please just be patient so that there's one in in person presentation that was due yesterday but will take place today so if you don't mind we're gonna have this in person talk before and then we start with the list as it's available on the program so after this first talk which according to my information is with Francesco Rinaldi then we start from the top with the Sami Ali Abdelal and so on so forth okay just just two minutes and then we start thank you so do you want to deliver now okay yes I apologize before to all the others but this is the person was expected to talk first so I address this message to him in particular