 So, welkom. This is the behavioral session. I am Jandrgoz Parkidna. I work at the Institute of Pharmacology of the Polish Academy of Sciences in Krakow. I'm the head of the department of molecular neuropharmacology. So, what am I doing here? Major part of our work is using behavioral models to study how drugs, especially psychotropic drugs, affect behavior of animals. And this became popular already in the 30s and has only further developed since. In the last decade or maybe two decades, it became possible to monitor the behavior of animals, but also when test-performing humans to monitor their behavior with a resolution that was unthinkable before. We can collect essentially any amount of data from the experiments and then we ended up with a problem we don't know how to analyze it. And this is the reason I am here and I will have the pleasure to introduce people who are solving this problem of how to extract from a large data set of behavioral data useful information. And we have the distinct pleasure to have as our first speaker Professor Raphał Bogacz, who is the head of the Models and Brain Decision Networks Laboratory at the National Department of Clinical Neurosciences of the University of Oxford. And I will not be taking further time. Please, Raphał. Good morning. So, I would like to first thank the organizers for invitations. We are here to be here. So, I would like to start this session with discussing simple behavior for which the new informatics challenges connected with data analysis have been already a subject of work for several years. In particular, I will focus on choice processes, which are common element of many different experimental paradigms. To understand the choice behavior, computational models have been developed, which describe the decision process. And automatic tools based on these models help to automatically analyze and bring insight from behavioral data. Furthermore, these models can also form a bridge between behavioral data and neurobiology. So, I will start with a brief review of neural basis of decision making. And then I will present the most influential model of choice process called the diffusion model. Next, I will highlight the importance the software tools played in making the diffusion model widely used and accessible, present two applications, two recent applications of this model and if there is some time discussed challenges. So, the neural basis of choice processes have been established in a paradigm in which a monkey is presented with a stimulus consisting of a cloud of moving dots. In the stimulus, fraction of dots moves coherently right on some tries and left on other trials and other dots are moving randomly. And the task of the animal is to make an eye movement in the direction of the majority of the dots. But because of the noise in the stimulus, this task is not trivial and requires looking at the stimulus for some time. Neural activity has been recorded during this task from several brain regions. And for example, this graph shows the activity of a sample neuron in area MT, which is part of the visual cortex. In area MT, neurons are selective for a particular direction of motion and this graph shows activity of neuron preferring motion to the right. And the dark line shows the activity on the trials when the majority of dots is moving to the right while the light line when the majority of the dots is moving to the left. The dark line is above the light line so this neuron provides some information which could be used to solve this task. But this activity pattern is very noisy because the stimulus itself is very noisy. So let us now think how the rest of the brain can make a decision on the base of the activity of this MT neuron. So imagine that these are the empty neurons selected for left-port motion and these are the empty neurons selected for right-port motions. Both populations produce spikes that are generated as votes for these two alternatives. So the brain needs to choose an option which gets more votes or has a higher mean firing rate. But please note that the brain cannot just measure mean firing rate instantaneously. It just listens to the spikes. So it has to listen to the spikes for a period of time, integrate evidence until reach a certain level of confidence that neural correlates of such integration processes have been observed in this task in lateral intraparietal area or LIP. So the LIP is involved in controlling eye movements and the neurons in LIP have preferred directions of eye movements. And this graph shows activity of a sample neuron preferring eye movements to the right. The solid lines shows activity on trials in ki eventually makes eye movement to the right. And you can notice that these lines gradually increase with time, which suggest that this neuron integrate evidence over time so essentially count the votes for right during this decision process. The idea that these neurons integrate evidence is also consistent with the fact that the slope of these lines depend on the amount of information in the stimulus. The highest for the easiest trials when there is a large fraction of dots moving coherently and on these trials there is a lot of information which can be quickly integrated over time. Additional insights on the decision process can be brought by presenting the same data but locked to the moment of response. So first if you look at these traces they look very similar for all conditions which suggest that these neurons integrate information until it reaches certain threshold and when the threshold is reached the choice is being triggered. Secondly, the dashed lines show activity on trials when the monkey makes eye movement to the left so it chooses the option which competes with the option for which the neuron is selected for. And this traces decay with time which suggest that when the neuron loses competition during the decision process they actually decrease their activity. So this suggest that these neurons do not simply count votes for its corresponding option because if this were the case, these neurons would also increase activity but rather these neurons represent the difference or the relative evidence for one option versus another. To account for this data several models have been proposed and one of the simplest model as shown here. So it includes two populations of sensory neurons selected for direction of motions and two populations of these integrator neurons selected for different choices. In this graph arose the node excitatory connection and lines ending with circles denote inhibitory connections. So these neurons integrate the difference between the evidence supporting the two options over time. So let us denote the mean activities of the sensory neurons by mu1 and mu2 and current activities of these integrator neurons by x1 and x2. So we can describe the dynamics in this model mathematically. So the model assumes that at the start of the choice process we initialize both integrators to a level which is halfway from baseline to the decision threshold so that I will denote decision threshold by a and during the choice process the dynamics of these integrators are defined by these differential equations so it changes proportionally to the difference in the mean activity of the inputs and is also disturbed by noise which I denote by epsilon and the choice is made whenever activity of any of the two populations reaches a threshold a. So let me now describe how this model can be reduced to a canonical model of decision making so this graph shows simulation of the model from the previous slide and the activities of two integrator population over time as shown in green and red so you can notice that these two integrators evolve in opposite way because they just get opposite inputs and whenever one of the population reaches the threshold the other must decrease to zero because they start half the way to the threshold therefore to describe this decision process you really don't need to keep track of both of these integrators but it's sufficient to just keep track of one of them and indeed in this diffusion model which is the canonical model for decision making you consider only a single integrator which integrates the difference between evidence for the two option and the choice is made when the value of this integrator evidence hits one or the other threshold and this model is called diffusion because this particle evolves in time as the particle diffusing in the water with current so let me describe this model formally so in the diffusion model we denote by x the integrated input for the option which is correct on the given trial and this integration is driven by the equation so the integrated evidence increases with a certain rate which is called the drift rate denoted by v and there is some noise the diffusion model also assumes that there is a delay between the onset of the stimulus and the time the information reaches the new integrator which is denoted by T0 so the activity of this integrator new is set to the baseline until this time and the integrated value reaches one of the two thresholds so this graph shows simulation of the diffusion model on three trials and in this simulation the drift is positive so on majority of the trials the model reaches the upper boundary which corresponds to the correct choice but on some trials due to noise it may reach the bottom boundary which corresponds to an error so in summary drift, threshold and non-decision time and this version is known as pure or easy diffusion to distinguish it from slightly more complex version with more parameters which are also being used so it's often of great interest to estimate the parameters of this model from behavioral data because they kind of provide the insight on the underlying decision process so for example the drift rate tells you how much information receives per unit of time which can be used in decision making while the threshold parameter informs how carefully participants consider their information before committing to a choice and these parameters can be extracted from behavioral data describing the accuracy which is the fraction of the correct responses and the reaction time across trials so this is a picture of a typical reaction time distribution from human participants and these parameters of the diffusion model can be identified from these behavioral measures because different parameters have different effects on their behavioral measures so let me illustrate these differential effects with this slide so here you can each display illustrates simulation of a diffusion model where you can see trajectories of different trials and then resulting reaction time distribution for the correct and for error trials and each column corresponds to change in a different parameters so let me just go from the right side so here you can see what happens when you increase the non-decision time so this just shifts the distribution of reaction time but doesn't change their shape or the fraction of the correct choices increasing the decision threshold reduces the number of errors the distribution diminishes but reaction time becomes longer and more variable reducing drift also makes reaction time longer and more variable but it actually increases the number of errors so in summary these changes in these different parameters have distinct effects on these different measures and therefore the parameters so the diffusion model could be viewed as a tool which essentially converts row behavior on measures into parameters describing underlying decision process to extract these parameters we need to have a method and many of such methods have been developed let me just tell a few words about them so when you develop such a method you have to ask several questions so the first question is in this objective function you want to optimize so you can for example reduce the difference between behavioral measures and your model behavior you can increase the probability of your behavioral data given the model so you can use some kind of probabilistic method and this different objective function have been compared through parameter recovery so you can simulate data from the models then use these optimization procedures and you can see which procedures give the closest match between your true parameters from which you generated the data and recovered values and also we have to consider what optimization procedure to employ before describing the methods currently available I cannot stop myself from telling you how the parameters were extracted when I started work on models of decision making in 2001 so at this time they were essentially found by hand so a researcher simulated a model for large number of trials and observed the produce behavior for the model and then compared this behavior with the data and gradually adjusted parameters until the two matched so when I started my postdoc I suggested to my mentor John Cohen that maybe I could write a software which would automatize this procedure and find this parameter automatic however John was not initially convinced because he felt that to find this parameter you need to have this human insight on the relationship between these parameters and resulting behavioral measures but he was open to the idea that maybe you can find these parameters automatically and he decided to test it experimentally so he chose particular data set he chose particular model this model was slightly more complicated but it was important he let me develop my automated software tool and he also asked undergraduate student to do his final project on finding the parameters manually so this poor guy spent four months of his life just simulating this model and tuning parameters gradually until the model matches the data and in the end after four months he actually produced better parameters than my automated tool but on the other hand he was able to find parameters for any model which was described by a MATLAB function to match with any behavioral data so the tool which we developed was kind of useful first step however the real change in the applicability of this decision making models was brought by software tools which were very easy to use so first such tool FastDM by Voss and Voss is really easy to use indeed in the parameters of the diffusion model and this software tool allowed diffusion model to be used by people without the background in computer simulations since then many other tools have been developed and my favorite is the hierarchical drift diffusion model from the lab of Michael Frank and it assumes that the parameters of the diffusion model for individual participants come from distributions with certain mean and variances and this toolbox extract this kind of group level parameters and because of this it's particularly useful in situation where there is few transport participants and to illustrate the impact this software tools had I'm showing here the number of citations of the paper introducing diffusion model so nowadays is cited almost daily but actually for many years since its publication had very little impact and I would like to illustrate that the number of citations of this paper was dramatically increasing after the tools have been developed and these tools not only increase the accessibility of this model but also increase the reproducibility of the analysis so initially the diffusion model was used in purely behavioral studies but with time the neuroscientist started to employ it to understand the neural basis of decision making and there are two main approaches of linking this model with neurobiological data the first approach is to look how the parameters of diffusion model change by various manipulations such as electrical stimulation pharmacological manipulations or how the parameters correlate with various features of the brain such as anatomical features disease state or genetic variations and more recently another approach have been developed which tries to understand the dependence on the parameters of neural activity so let me describe two studies which use these two approaches so recently I had pleasure to help fit the diffusion model in an experiment investigating decision making in insects so my colleague Garo Misenbach developed this really cool paradigm allowing to look at the decision processes in the fruit fly so the fruit fly is placed in a tube where the two ends are filled with different concentration of odor which have been previously associated with electric shock and when the fly enters this kind of intermediate decision zone when there is a gradient of odor it has to decide where to go so you can get behavioral measures so you can get an accuracy whether the fly actually went in the direction with a smaller concentration of this odor and you can measure reaction time which is the time from entering to exiting this decision zone furthermore similarly as in the moving dots task you can parametrically manipulate difficulty by changing the relative concentration of odor in the two ends of the tube and so here you can see the behavioral data from the fruit flies which show the same patterns which you usually see in humans so in particular as you make the task more difficult by making the concentrations more similar and the accuracy of the choice decreases and the reaction time increases and in this study Lukas Groszner compared the behavior of control flies in the mutants who had mutation in Fox P genes which are shown in purple and these mutants had the same accuracy but they had increased reaction times on difficult trends so we analyzed this behavior data with the diffusion model so we used slightly more complicated version than the one which I described and one of the parameters which was different between the mutants and controls was the drift rate so how can we understand where this small drift rate comes from so Lukas also managed to investigate the neural basis of decision making in this task and he found that there are neurons which also represent the integrated evidence in the fruit fly brain however here these neurons do not represent the integrated evidence in their firing rate but in the membrane potential and furthermore his data suggest that when the membrane potential of these neurons reaches a threshold then the fruit fly starts to move in the corresponding direction so due to reduced size of the fruit fly brain the whole choice machinery is compressed to a size of a single cell now Lukas also noticed that the mutants had higher concentration of leak channels and to understand the role of these leak channels you can kind of compare the evidence integration during choice process to a butter which is being filled with water where the water corresponds to the evidence and then different leak channels would correspond to essentially hole in the bucket so you can imagine that if you make more holes in the buckets then the rate of accumulation of water in this bucket will increase which is basically consistent with a reduced drift rate observed in these mutants so in summary in this study the diffusion model allowed to link subcellular properties of single neurons with the choice behavior of the entire animal so in this previous study we used a classical way of employing this diffusion model where we checked how the parameters differ between different conditions but there is also a new exciting approach which was developed in this toolbox which allowed to look how the parameters depend on neural activity so let me explain this method on the base of the Treskocin so you can measure some neural activity for a particular part of the brain and you can look if this neural activity correlates across trials with a parameter of the diffusion model such as Treskocin so you would then assume that Treskocin is a linear function of the neural activity on a particular trial and then instead of finding the Treskoc parameter itself you find this coefficients a and b for which the choice behavior on individual trials is best predicted by neural activity and we used this approach in a study investigating the neural mechanism of setting the decision Treskocin in the brain and this was done in collaboration with a group of my colleague Peter Brown Previous research has suggested that decision Treskocin one of the parts of the brain involves in control of the height of the decision Treskocin is the subthalamic nucleus or STM so my colleague Diamond Hertz recorded activity of STM from patients with Parkinson's disease implanted into STM for clinical reasons and in this task the patients were asked to perform the moving dots task and to further manipulate the decision Treskocin on some trials they were asked to be accurate while on other trials they were asked to be fast and then we can now look what are the candidate neural signatures of these changes in the decision Treskocin so we can compare the frequency of different oscillations in the local speed potential recorded from subthalamic nucleus in the speed and accuracy condition and we observe that indeed there is a difference in the low frequency oscillations so then we fitted a diffusion model in which we assume that the Treskocin on a given trial is a linear function and then we fitted a model which instead of having Treskocin parameters had a parameter describing the kind of intercept and slope of this relationship for the two conditions and then on the base of this we can estimate the relationship between the neural activity and the Treskocin which is shown here so our analysis suggests that indeed the Treskocin depends on this low frequency oscillation in the subthalamic nucleus and then when the participants were asked to be accurate but not when they were asked to be fast so this is kind of mechanism which is specific only to trying to be accurate but not to trying to be fast so let me just say a few words about the challenges connected with data analysis so when we use more complex versions of the diffusion model which have additional parameters which can describe other aspects of the choice process it's no longer so easy to identify them uniquely furthermore, diffusion model is only one of the models proposed for decision making, there are other models which fit the data equally well so on the base of the behavioral data alone it's not always possible to identify the underlying decision processes and looking in the brain is really necessary so in this talk we are focused on tasks which are highly practiced but a lot of interest in behavioral research and computational neuroscience is on learning and for learning task the models or the automatic tools are not as developed as for the choice processes and I think that one of the reasons is that there is a larger variety of different models which are suitable for different tasks so the models are not as standardized but I still think that it's really important direction to develop these tools for fitting reinforcement learning models as well and to integrate reinforcement learning with diffusion model so let me summarize the talk so I presented the diffusion model which is a tool which translates behavioral data to parameters describing the underlying choice process and it also allows identifying neural correlates of these processes so I want to say that this work was done together with Jonathan Cohen from Princeton and my collaborators from Oxford and finally I would like to advertise the data sharing platform from our institute in Oxford where you can download various data sets brain atlas and different tools for data analysis thank you Eva Gudovska your talk provoked me a little bit because you talked about the diffusion model and I would like to ask you what do you know about the noise which you incorporate into the model because I think the noise is crucial your model is not the diffusion model it's a continuous time random work known in mathematics so the noise is really interesting so let me just go back to the equation and explain it's a great question so in this standard version of the diffusion model people add noise which is a Gaussian noise with unit variance and one could say that actually this noise could be parameterized in different trials you have different amounts of noise however people in standard diffusion model don't include these additional parameters so if you include a parameter here then it's no longer possible to uniquely identify this noise parameter drift rate and the threshold and the reason for this is that if you scale all these three parameters by a constant the choice behavior will not change so if you scale the threshold drift rate and noise is just like corresponds to stretching this picture so the reaction time distribution the noise to a constant however people sometimes use those in fact in this study with the food flies for simplicity just describe just one of the parameters but if you try to fit a standard diffusion model to this data you cannot fit this data just with changes in drift you need to change drift and threshold but we knew from the neurobiology that changing threshold because they initiate the movements when the spiking threshold is rich and the spiking threshold is the same in the mutants so we looked for other parameters and in fact we found that we can explain this data by changing the drift and the noise and essentially the mutants have smaller drift and smaller noise and that's why the accuracy doesn't change this is the property of the diffusion model that if you rescaled drift and noise by the same amount then the accuracy will not change and then we can look at the traces of the membrane potential as a function of time and indeed we found that the variability of these traces was reduced in the food flies so we could see changes in the noise level in the neural recordings as well so I'm afraid this is the point we have to cut the discussion short although I hope there would be more questions I would like to thank again our speaker Rafał Bogar