 world. So welcome to this talk in the worldwide neuroscience series, kind of under the auspices of Sussex visions. I think this is about the 15th or 16th talk that Sussex visions has organized and it's a real pleasure to introduce for this talk today Mike Manukin from the University of Washington in Seattle. And for those of you who don't know Mike, Mike started out I think studying linguistics and then switched to neuroscience to do his PhD with Jonathan Dem working on the retina over in Michigan and then he did a series of really nice experiments looking at mechanisms of contrast adaptation in particular. And then after working with Jonathan Mike graduated, shall I say, from guinea pigs to primates and went over to University of Washington in Seattle to work with Dennis Dacey and Jay Neitz looking primarily at color mechanisms in primate retina and and also motion processing mechanisms and Mike's had his own lab since about 2015 I think and he's already produced some really really nice papers. For instance looking at mechanisms or at least opposing mechanisms of sensitizing and contrasting adaptation of the primate retina and their mechanisms but today's talk is I think there'll be most mostly about motion processing predicting the future not just predicting the future from the past motion processing in the primate retina. Thanks very much Mike. Thank you so much Leon. I really appreciate the invitation to chat about my research today and and actually much of the ideas I was talking to Leon earlier many of the ideas that have kind of guided what I've been working on in my lab have come from Leon's own work. So today as Leon mentioned I'll be focusing on motion processing and really the recent the recent work in my lab has dealt with motion processing in somewhere or another and today we'll be looking specifically at the question of motion prediction and I'll explain what that means in a couple of slides but first I want to acknowledge my collaborators on this project I was fortunate to work with two really talented undergraduate students Belle Liu and Arthur Hong and with Fred Rieke here at the University of Washington and so the story I'll be telling you today is part of a larger story that we recently posted up to BioArchive so if you're interested in learning more about it I encourage you to to check it out. So what is predictive encoding of motion and so to use to just explain this I'm going to use a baseball analogy but you could also think of it in terms of cricket or tennis. So here's a picture of a baseball game and what's what's happening here is a pitcher has thrown a ball toward the mound and you can see it here hanging in midair and the task that this batter has is to estimate or predict where that ball is going to be at some future time point and then bring his bat into contact with the ball and this is an incredibly challenging task it's a task that I'm quite sure I would not be successful at and one of the challenging parts of this task is that there are these inherent delays that occur in visual processing and in other neural processing that essentially the delay means that the from the time that that light from the ball goes and hits his photoreceptors at the back of his eye there are these delays that are introduced and one of those is a huge delay in phototransduction and so if we say that just getting signals from the the photoreceptors out of the retina takes 50 milliseconds which is a reasonable estimate if that's a 90 mile an hour fastball in that 50 milliseconds it will move over six and a half feet and so the the information about that ball is is lagging quite a bit behind the actual motion of the ball and so we know in order for this to be an interesting sport at all that there the this batter has to be predicting where the ball is going to be at some future time point so that's what I mean by motion prediction and there's some work that's been done previously I'll mention in a second from Leon as well looking at this question in in the retina so this this problem of predicting motion actually comes up in all sorts of natural contexts and this is an example of that so here's a proboscis monkey jumping from the branches of one tree to the branches of a neighboring tree if you think about those branches moving towards the animal very very quickly and this animal has to determine where those branches will be at some future time point so that it can bring its hands and feet into contact with the branches and be successful in that jump so one of the important things you know I'll be talking about predictive encoding in the context of motion but this is really part of a larger problem in neuroscience and really Bill B. Alec has in Naftali Tishpi have been two of the people who have worked on this the most and one of a beautiful quote I like from one of Bill B. Alec's papers is this he says put bluntly non-predictive information is useless to the organism and the reason he says this is because we need to be able to predict future states of the world in order to let information about the past guide our future actions and so something that's not predictive about what's going to happen in the future isn't really useful to us and there have been some really beautiful papers in recent years on on this subject so to work from Stephanie Palmer in particular and Bill B. Alec so just to give you an idea well where we'll be going through the course of this talk we'll ask three different questions as part of the talk today so the first question is do retinal cells and retinal cells in the primate retina encode predictive information about motion and so we'll see that at least four of the pathways major pathways in the primate retina do encode predictive information about motion and I should mention that the the subject for today is recordings solely from the macaque monkey retina which is a nice model for human vision all right so the next question we'll tackle is how how impressed should we be by this predictive encoding and so we'll use an engineering approach to calculate what the upper theoretical limit is to encoding predictive information about the stimuli that I'll I'll show you today and we'll look at how well these four cell types are doing relative to that theoretical limit and we'll we'll see that they're actually pretty darn close to what is physically possible in encoding the stimuli that we'll be using in the study today and then the final question we'll tackle is what mechanisms contribute to predictive coding and as seems to continue to come up in my work the retinal bipolar cells are really important for the phenomenon that I'll be talking about today all right so I mentioned that there's this time lag between getting signals presented at the cone photoreceptor receptors to getting signals out through the ganglion cells the ganglion cells are the output neurons of the retina and the cone photoreceptors at the back of the eye collect the light and turn it into a chemical synaptic transmission and all of the light levels that I'll be talking about today that we're using the experiments are between 15 and 200,000 photoisomerizations per cone per second at the mean light level so that they're photopic backgrounds one way you get signals from the cone photoreceptors to the ganglion cells is through the bipolar cell pathways and bipolar cells release glutamate which at this these synapses is an excitatory neurotransmitter onto these ganglion cells and then each of these these synapses in the outer retina and the inner retina there are inhibitory inner neurons that shape signals flowing through the bipolar cell pathway so there are horizontal cells in the outer retina and amicron cells in the inner retina so what we'll do today is we'll present stimuli at the level of the cones and we'll look at the processing in different parts of the circuit so we'll record responses from the ganglion cells to look at what's going on in the output of the retina we'll record voltage responses in h1 horizontal cells to the same stimuli and then we'll also in voltage clamp measure the excitatory and inhibitory synaptic inputs to these ganglion cells try to understand how these inputs are shaping encoding of these motion stimuli and the the cells that I'll focus on today are these cells here the parasol and the smooth mono stratified ganglion cell and these each come in on and off types so the on types prefer increments in light intensity relative to the mean the off types prefer decrements in light intensity relative to the mean and the important thing to bear in mind about these cells is we know by virtue of their central projections and from beautiful lesion studies that these cells contribute dramatically to motion perception and motion vision in primates so I mentioned I was going to give some background and some previous work in motion prediction in the retina and so there's this beautiful literature that's been done in in salamander and zebrafish retina and this is this literature has been focused primarily on translational motion it's all explain what translational motion is here and just in case your video doesn't work I made this little diagram down here in the bottom left corner and so if this is the space dimension here and this is the time dimension this square has moved from this point in space to another point in space at a later time point and this is just trans the diagram for translational motion it's typically what I think about when I think about motion and so it hopefully you can see this video and hear the the tree branches are translating up and down left and right and and yeah this is the typical way that I think about motion but in this this study today we'll be looking at translational motion but we also want to look at other types of motion that are present in natural vision and so those types of motion are shown here so this is the diagram for diverging motion correlations so here if for example if this square became closer to us it will become larger on our retinas later and later time points so as as objects approach they produce these types of diverging correlations and as objects move away from us they produce these converging types of correlations and you can see that here the tree branches moving towards you those are diverging correlations and moving away those are converging correlations so we'll look at all of these different types of motion and we'll try to understand how the cells are encoding in those those types of motion so the set of stimuli that we used for this study were developed by Jonathan Victor at Cornell and they're a really wonderful set of stimuli for reasons I'll talk about in a minute but essentially you can you can see that we have our diagrams here and just to give you an idea of what the translational motion stimulus looks like so this is I've collapsed one dimension of space because there are bars across that dimension of space and so um if we look at the space versus time dimension here if you pay attention to this this bar down here it's translating it's moving across uh the retina as a function of time and the diverging and converging motion stimuli examples of those are shown here so here you have a um a bright square that diverges into multiple bright squares at some later time point for the diverging motion and there's a really wonderful literature uh looking at these types of stimuli and and the percepts they elicit in in humans and also in fruit flies but we'll for the purposes of this talk and for this paper we wanted to focus on using these stimuli to understand predictive encoding of motion so we'll record the responses of cells to these different classes of stimuli and compare them to an uncorrelated noise condition that lacks that has the same contrast etc but lacks net spatiotemporal correlations and here are some example spike responses from an on smooth monostratified ganglion cell so I'm showing you the spike rate and spikes per second as a function of time and so the one of the key take homes from this slide is that um the cells are responding quite strongly to the diverging and converging motion stimuli and what we'll be able to do is collect the spike output from these cells and then do some analysis to see how the cells are encoding different parts of different information about the stimuli as a function of time but just to give you a background on the stimuli themselves and sort of the question in the context of predictive encoding um they're usually experimentally we use stimuli that are fall on two extremes so the one extreme is a completely random stimulus in which um there are not there are no net correlations in the stimulus the other extreme was a deterministic purely deterministic stimulus and I'll give you an example of that in a second so in the context of a random stimulus shown here if we have information about these stimuli in the past and we try to predict what's going to happen in this time been in the future there's no structure in those stimuli we can use to accurately predict what's going to happen in this time been here all right sorry a technical issue here but if we look at a deterministic stimulus um the the story is different so um we'll just take a second for this deterministic stimulus to show up and so the deterministic example I'm using is just a sinusoid so if we have information about that sinusoid and what's happened in the past um we can pretty well predict that the the stimulus is going to increase here in this time been and if it's a truly deterministic stimulus we can we know exactly what's happened infinitely far in the past or we can predict what's going to happen infinitely far in the future and really stimuli in the natural world fall in between these two extremes and so even with visual motion there are stochastic or random components to that motion and just to give go back to the baseball analogy a deterministic pitcher is a terrible pitcher so if you if the baseball batter can determine exactly where that ball is going to be at any time in the future then that pitcher is not going to be very successful and the pitcher's job is to introduce stochastic components to the flight of that ball that make it difficult to predict where it will be in the future and likewise the stimuli that I've shown you a second ago that were developed by Jonathan Victor have this beautiful combination of deterministic motion components and also stochastic components that is set by the correlation times in the stimuli and we'll look more carefully at the time course of the correlations in a second so let's just jump into trying to answer some of these questions so the first question do retinal ganglion cells in the primate retina encode predictive information about motion and so the tool that we're going to use to answer this question is mutual information and so just to give you an idea of what that is I've made these cartoons of the stimulus and this these subscripts are time bins so the stimulus at time one two three and the response of the cell in the same time bins and what we can ask is how much does the listening the ability to listen to the response of the cell reduce our uncertainty about what the stimulus was and that is exactly what mutual information is and so to give you a concrete example if listening to the response of the cell narrows down this possible set of stimuli by half that that is equivalent to one bit of mutual information and so what I'm showing you here are when the stimulus and the response are perfectly aligned and if we want to measure the mutual information when there's no time shift that's exactly what we do we want to look at what's happening how much the response of the cell at time t equals one tells us about the stimulus at some future time point in other words predictive information about the stimulus we just shift the stimulus to the right and rerun this calculation and that's shown here so that well this is called the time shifted or time lagged mutual information and so the future time lag shown here so this mutual information is a function of time shift future time lag is just shift we shift the stimulus to a future time point and recompute the mutual information and Stephanie Palmer and Bill Bialik have have developed this this technique all right so let's look at some of these mutual information curves again mutual information as a function of time shift and what we'll start with is the information that the stimuli have about themselves and so we're shifting the stimuli relative to each other not to relatives themselves and so for the random stimulus the uncorrelated noise stimulus that we use here in this talk this is what the the mutual information curves look like so at time equals zero when the stimuli are perfectly aligned there's this large amount of mutual information the stimuli have about themselves and then as you go farther out and posit or negative time bins you don't have much in fact you can we know exactly what the time course is of this um mutual information because it corresponds because there are no correlations it corresponds to a single stimulus frame a 16.7 millisecond time window now if we look if we plotted our deterministic stimulus on this mutual information plot it was again truly deterministic it would be flat everywhere so you could go infinitely far in the future and have the same amount of information about the stimulus but the stimuli that we're using have a correlation time that really so so here's the here's the curve for all of our motion stimuli again it's the mutual information is high at t equals zero but as you go farther in the future that decays rapidly and beyond 50 milliseconds in the future the um the mutual information is zero and so the the nice thing about this is we can um measure the same curves between in our cells and the way that they're encoding information about these stimuli and compare them to the available information in the stimuli themselves and that's what we'll be doing here but the one thing we want to do first is we need to account for this time lag from the cones again I pointed out that there are these delays in phototransduction in particular that are pretty um marks and what we want to do is understand in the context of these ganglion cells or horizontal cells etc what um how much predictive information they're encoding and so we need to account for this time lag and the way that we do that is we present our uncorrelated noise stimulus and and I didn't mention it but all of these stimuli we present are interleaved with one another and we do many many many repeats of the stimuli to get the necessary amount of data all right so if we take that uncorrelated noise stimulus and we look at the temporal trajectory preceding a spike so a spike occurred at time t equals zero here we can measure exactly when the response of the cell occurred peaks relative to that time lag from the cones and in this cell and this is an on smooth monostratified cell that is minus 42 milliseconds so it took 42 milliseconds from the time we presented this the stimulus to the cones to when we measure the peak response in our ganglion cells and so essentially what we do is we shift all of our mutual information curves by minus 42 milliseconds for this cell and we do this for each and every cell in the study and we define this this dash line which is at minus 42 milliseconds as t equals zero so anything any encoding that occurred prior to that is past information and anything that occurred after that is predictive or future information when we do the same thing for the curves and which we're looking at the mutual information contained in the stimuli themselves so what i'm going to show you now are some of the first the mutual information curves that the stimulus stimulus information and then we'll look at how a smooth monostratified cell is encoding those same stimuli and so again this is for our uncorrelated noise stimulus there's a very narrow time window in which there's information about the stimulus and these are our different types of motion here and they they're all identical in this study and if we look at the mutual information in the encoded in the response of the cells we can see that for the uncorrelated noise condition here there are two different features that are that are really interesting here first this dash line again is the measured time lag from the cones and the peak of the mutual information for this stimulus occurs right at that time lag and the other interesting feature is that we look at how narrow the information is narrow the time window is for the available information in the stimulus we also see a fairly narrow encoding of mutual information by the cell so roughly aligns with the available information this is also the case for translational motion we see that the available information is across a much wider time window and we see a wider time window in which the encoding occurs and there's another interesting feature in here that i'll get to in a second but first i want to show you the diverging motions mutual information so there again the the available the encoding of the information about these the diverging motion stimuli is wider than we see for the uncorrelated noise stimulus and corresponds to a wider available information but there's another interesting feature here that really popped out to us and that is that the peak of the mutual information curves occurs at a future time lags this is true for the diverging motion and for the translational motion so it's highlighted here so this is not what we saw for the uncorrelated noise condition that occurred right at where we measured the lag from the cones again so this is evidence for predictive encoding of motion and what we'll do a little bit in a little bit is look in much more detail at how much predictive information is being encoded for these different motion types the other reason this is interesting is we don't see this for all of the motion types so for converging motion again that's the tree branches moving away from you we see that the mutual information curve is actually peaking at negative time lag so so it's it's not encoding a lot of predictive information again we'll see later on that this holds true across cells and so again what we want to do is we want to carefully quantify the amount of information encoded about the past versus the information encoded about the future and then use our engineering approach to um compare the encoding of the cell to a an ideal algorithm for encoding those stimuli and so to do that again we're going to define t equals zero um as our past information so wherever we are on the curve at that that time point is the amount the number of bits of past information that we're encoding and this is the technique that was developed in that in Stephanie Palmer's paper and then we'll be able to in use look at the curve at future time lags and be able to compare the encoding of past information to predictive or future information and so we'll we'll ask now that question how close is the encoding of these cells of predictive information to the theoretical limit and um again the tool that we're using is this is called the information bottleneck and I'll give you more details of um and show you some information bottleneck curves in a minute but the idea is that during the encoding process there's a great deal of um compression in the um encoding that occurs and that occurs because just for example the stimulus set that we're using there are 15 to 19 of these bars presented across the retina and on any given frame each of those bars can either be brighter or darker than the in the background so that means there are two to the 15th to two to the 19th different stimulus possible stimulus patterns that can be presented to the tissue whereas um on that same stimulus frame which is 16.7 milliseconds we measured zero to seven spikes in that same time window in our cells and so that you're going from two to the 15th possible stimuli to two to the three response levels in those cells so there's a great deal of information that's being thrown away as part of the encoding process and what the information bottleneck allows you to do is measure what information is encoded and the relative amounts of different information and we'll use it to look at past information versus predictive information so i'll be showing you the curves that look like this so the y-axis is future or predictive information and the x-axis is past information and really because of this compression there are many different several different ways in which um a circuit could allocate its resources to encode information and this is one of them in in which the the circuit doesn't care about predictive information and only encodes past information about the stimulus and in that case the um the values will fall towards the x-axis here another um possible information allocation strategy would be to encode only predictive information and throw away all information about the past and that's actually physically impossible for reasons i'll try to explain now and so if we think about this bar and it's it's going to move across the screen now and we think about the past trajectory of that bar so here i've shown like four different timeframes in the past of where that bar has been and let's say this is t equals zero now if we want to predict where that ball bar is going to be in the future based on its past trajectory you can pretty well predict that it's going to fall somewhere right here based on where it's been in the past now imagine the um condition i just showed you in which you don't encode any of that information about the past you throw all of this away in in the encoding process now it's as if that bar has appeared out of nowhere and there's really no way that you can reliably predict where the bar is going to be in the future because you don't know where it's been in the past so this is just an attempt to illustrate why you have to encode some past information to encode any predictive information so we'll see that for ourselves um when they're encoding predictive information there's a mixture of both predictive and past information that's being encoded and the bottleneck actually allows us to precisely calculate um or estimate how much given if we encode this much information about the past what's the maximal amount of predictive information that we can encode about the stimulus and that is shown here so this is predictive information on the y-axis this axis is the past information and then what i'm showing you is the information bottling that curves for our motion stimuli at three different future time lags so it's 1733 and 50 milliseconds in the future and for the purposes of this talk we'll just focus on the 33 millisecond time bin um and so for example if we we encode one bit of past information then the most predictive information that we can encode is say two-thirds of a bit of um predictive information at 33 milliseconds in the future and then we go all the way up to four bits of past information the most we can encode about the future is two bits of predictive information and what we'll do in the next few slides is look at um where the cells are falling relative to this boundary which is the engineering approach the very best you can possibly do in encoding predictive information so um first i'm going to show you the uncorrelated noise condition and again this is the path the predictive information as a function of past information and as you might have guessed at 33 milliseconds in the future the stimulus has no predictive information available at all and so as we expect all of the cells are falling along the x-axis they're not encoding predictive information about the stimulus however when we look at the translational motion here um this black solid line demarcates the information boundary the bottleneck boundary that tells us what's the maximal amount of predictive information for any given past information encoding and the take home here is that these cells the parasol and smooth mono stratified cells are really sitting quite close to this boundary for what's physically possible in encoding predictive information about this the stimuli likewise if we look at the diverging um motion stimuli and i should have mentioned that the diverging and converging motion stimuli come in positive and negative contrasts and so that's what these positive and negative mean um and so if we we focus in on the open circles for the positive because those are the on parasol cells and smooth cells um we'll notice that they also are sitting quite close to the boundary of the bottleneck so they're doing quite well and likewise for the the off cells for the negative contrast they're sitting quite close to the boundary for available um encoding of predictive information now if we look at the converging motion stimuli we there are some clues from the previous curve that we looked at that this is not going to be the case for converging motion and sure enough when we look at these um these curves we see that the encoding of the cell is really collapsed towards the x-axis they're not encoding much predictive information about the converging motion stimuli and it's very far from anything that would be considered an optimal encoding so again the take home from this set of slides is that the predictive encoding is is impressive and it's also seems to be restricted to at least for these cell types for translational motion and diverging motion correlations so the last part of the talk I want to focus on the mechanism so what parts of the circuit and what neural mechanisms are contributing to this predictive encoding of motion so the way that we're going to do this is we're going to record different aspects of the circuitry so we'll record voltage responses in h1 horizontal cells as I mentioned and here are some voltage responses this is the membrane voltage is a function of time for some of the stimuli and what we'll do is we'll use the same technique that we used in the spike output to measure mutual information encoding in the membrane voltage of the cell relative to the stimulus we'll also assay the inhibitory synaptic inputs to our ganglion cells and the excitatory synaptic inputs to our ganglion cells to try to understand how these components of the circuit are contributing and so if we look at the information curves in the h1 horizontal cell here again this is the information on the y-axis is a function of time lag on the x-axis and one of the interesting features here is the blue curve is uncorrelated noise condition and the translational motion produces in teal here produces higher information rates on average um than uncorrelated noise condition but our diverging and converging motion stimuli don't and so they're really sitting very close to the peak information that we see for uncorrelated noise and so the h1 horizontal cells don't appear to be particularly sensitive to those types of motion correlations if we look at the inhibitory synaptic inputs to our cells we see that again the translational motion produces a large spike in mutual information but the diverging and converging motion are pretty unimpressive in terms of their magnitude of mutual information for those stimuli now we see a very different pattern when we look at the excitatory synaptic inputs and this again these are this is originating in the glutamate release from diffused bipolar cells what we see is actually something very similar to what we saw in the spike output of our ganglion cells so this blue curve again is the uncorrelated noise condition and all of our motion types are peaking at higher mutual information rates than our uncorrelated noise condition and also in terms of our question here about predictive encoding you can see that there's this evidence for predictive encoding for both translational motion and diverging motion in these cells and we look at this more closely in the actual paper but you can see that the curves are peaking much earlier and there's quite a bit of information here at future time lags all right so what's going on here what do we think could be driving this predictive encoding of motion and one of the things that we we considered is that this there's a there's some mechanisms that we identified earlier in some of our earlier work in Fred's lab and in my own lab that contribute to generalized motion sensitivity in these cell types and we think that they also can contribute to predictive encoding of motion in these cells and so let me just give you an example of of those those mechanisms and so basically it's it's electrical coupling between the bipolar cells and then their output nonlinearities the glutamate release nonlinearities that we think are contributing so if we present a stimulus over this bipolar cell here it will cause this bipolar cell to depolarize and because of electrical coupling between the bipolar terminals a portion of that current in that bipolar cell will spread laterally to its neighbor here now this this spread of current actually primes this bipolar cell terminal so that if in a short time window afterwards there's a stimulation of this bipolar cell it actually pushes it farther up on it increases the voltage in the terminal and what that so what i'm showing you here is the voltage in the terminal as a function of glutamate release and so if we push the voltage of that terminal farther up here what will happen is that maps to a much higher output in terms of actual glutamate release and we showed previously that for several types of motion in both mice and in primate retina that this mechanism contributes to generalized motion sensitivity and we think that this also contributes to the phenomenon we're seeing here for encoding of predictive information first i want to point out though before i go into that is that this mechanism will not only be engaged by our translational motion that i showed you earlier but will also be engaged by diverging motion correlation so again diverging motion would first stimulate this bipolar cell caused the lateral spread and the priming of its neighbor and then in a short time window for diverging motion you get that native that second bipolar cell getting being stimulated and that priming can have an effect on the glutamate release so to explain why why we think that this mechanism contributes to predictive encoding let me explain the uncorrelated stimulus condition so again as i mentioned our our motion stimuli have both stochastic or random components and deterministic components and the stochastic components are ones that will not contain predictive information about future motion and so what will happen if we have random stimulation of the bipolar network so for example again you get if you stimulate this bipolar cell and you get the lateral spread and priming of this neighboring bipolar cell for an uncorrelated noise stimulus or any kind of stochastic stimulus this bipolar cell won't necessarily be stimulated in a short time window following its priming by its neighbor and so this cell is it's you know it's primed and now what right it's it's not able to take advantage of that priming and so what will happen is in terms of the the output nonlinearity here the because it's primed and then not stimulated it will end end up somewhere here on its voltage curve and mapped to a fairly low level of glutamate release and again we showed this previously for uncorrelated stimuli in our previous work but we think the same thing is true for the stochastic components of our motion stimuli the ones that don't contain predictive information so essentially the take home for this slide is that non-predictive information will have a tendency to fall down here and on the output curve and not get passed along downstream to the ganglion cells whereas for correlated stimuli as I just mentioned you do get to take advantage you prime this bipolar cell you get to take advantage of that priming and push the cell there the terminal higher on the voltage curve and then produce an increased glutamate release and so correlated components of the stimulus that contain that predictive information will have a tendency to get passed on downstream to the ganglion cell circuitry now we wanted to to look at these potential mechanisms more carefully and we did that in the context of a computational model and so what we'll do is we'll we have a computational model we measure components of that model directly from different in vivo or in in vitro recordings from the circuit but essentially the idea is that you have a a similar stimulus that's presented to these model bipolar cell subunits and then they are passed through an output non-linearity and then pooled at the level of the model ganglion cell just as we see in our own in the actual circuit and we'll compare the output of this model that lacks any coupling between the subunits to one in which the subunits are coupled and we'll compare encoding of predictive information between those two models and we'll also look at this in the context of this output non-linearity again I mentioned that that non-linearity is really critical for generalized motion sensitivity and we think for predictive encoding of motion because it rejects uncorrelated and non-predictive components of the stimulus and so what we'll do is we'll vary we'll change the shape of that output non-linearity by changing the threshold at which the piecewise non-linearity begins to respond so here's a threshold of zero it's perfectly linear and we can compare that to a high threshold in which you have to reach a quite high threshold in order to to elicit an output so what I'm showing you here are some curves where we've the model has produced mutual information curves as a function of the threshold for the non-linearity and the black curve is the total predictive information and the teal curve is the past information and what we'll do in the next slide is compare the predictive information the ratio of predictive information to past information encoded this is for the uncoupled subunit model and we'll compare this uncoupled model to the coupled subunit model and that's shown here so this is the ratio of predictive to past information as a function of threshold again and the take-home here is that the coupled subunit model performs superiorly at encoding predictive information relative to the uncoupled model and this difference becomes even more dramatic as you increase the threshold of the output non-linearity the same is true for our diverging motion correlation so a positive thresholds here that the coupled subunit model encodes quite a bit more predictive information relative to past information about the stimulus and this is also true for our converging correlations so what's we asked three major questions as part of this talk and the first question was do retinal cells ganglion cells in the primate retina encode predictive information and we found that yes the parasol and smooth mono stratified cells encode predictive information and that is specific for translational motion and diverging motion correlation so the tree branch is moving towards us and then we asked how impressed should we be by this encoding of predictive information and we use an engineering approach to calculate the upper limit the upper bounds on encoding of predictive information and found that these cells are very close to that upper boundary for encoding predictive information so I personally am quite impressed with how well they're doing it so the final part of the talk we asked the question what mechanisms are contributing to this predictive encoding and we identified the source of the encoding as the diffuse bipolar cell network and we found that both electrical coupling and the nature of nonlinear synaptic transmission are contributing to predictive encoding of motion and rejecting the uncorrelated components of the inputs so finally I'd like to thank again my collaborators on this project Bell Arthur and Fred and my funding sources and also I'd like to thank you for your attention and for coming to the talk great thanks so much Mike thanks very very much wonderfully clear talk trying to explain some well new concepts for me at least in terms of trying to understand information flow through the retina so guys I want to encourage you to put any questions that you might have into the chat and we can immediately start with a question from Anna Vlasit so Anna was wanting to know a bit more about the stimulus Mike can you see can you see the questions in the chat there I don't I just have a okay well I'll repeat them anyway but if if you open the chat in the zoom you might see it but I'll repeat it anyway so Anna's asking how big the individual pixels are in your stimulus and uh relative to the size of the bipolar cell receptive fields in the macaque very good question um the the bar width was 50 microns that we used um and that so the the diffuse bipolar so so these are recordings from macular to mid peripheral macaque retina and the diffuse bipolar cell dendritic trees very quite a bit over that range but in the peripheral area there 35 to 50 microns would probably be the the range of the diffuse bipolar cell dendritic trees um so it's fairly close to the bipolar cell receptive fields I would say Mike and I had another question and I think my question relates to her question but I'll put it her way first so Anna's asking what's the spatial and temporal limits of this predictive coding excellent question so the um spatial limits I don't know how to address that I the that's tricky because the this the correlations in the stimulus are present across space just the way that those stimuli are generated um the temporal limits we only probed um this for at least for this project we probed the um temporal components where we we define the correlation times in the stimulus to be to go away after plus or minus 50 milliseconds um so we didn't do anything where we changed the update rate of the the project the projector or anything like that to look more carefully at the temporal limits but the really the take home what we wanted to do is focus first on defining it and looking at it consistently across one set range and in the that context those cells are really encoding nearly all of the available information in the translational and diverging motion stimuli um in those those time ranges so even if you go and you can look at the paper even as you go out to 50 milliseconds in the future the the cells are really pushing the limit there too in terms of their encoding of predictive information so I mean what I was thinking about what I was thinking about your model Mike was membrane time constants whether you that's part of the model whether you just kind of simplify and ignore those so far we've um this has been challenging enough so far so we just simplified it to start but that's an excellent question and definitely something to look at in the future yes I mean the diffuse bipolar cells have lots of teeny weeny I mean all the synaptic compartments are pretty small aren't they but um but the resistance between compartments um anyway yeah we'd have to think more about that but anyway okay lots of questions standing up Mike right so Jeff Jeff is asking um Ray Turner and Ricky's recent paper around surrounding inhibition your effects should occur at smaller spatial scales than surrounding inhibition correct is that right have you tested for a role of the surround um that's an excellent question Jeff and that's actually one of the things that I've been doing recently in this for this project we didn't test for the role of the surround um but that's something that we're looking at right now okay and then Sylvia Schroeder asks I think I don't quite understand the measure of past versus future information if the stimulus is a non-zero auto correlation how can the neural response encode past info without also encoding future info um yeah I I guess I didn't explain that very well they they are they do have to encode past information in order to encode future predictive information and so that's right there is a non-zero correlation auto correlation and correlation time in the stimuli and um yeah we we define the past information based on Stephanie Palmer's work at t equals zero with that time lag from the photoreceptors and then calculated future information relative to that but you're right um if you don't encode information about the past it's impossible to encode predictive information yeah and then a question from democratis karmin lis are the values of coupling strength required to see the difference between coupled and uncoupled models what you'd expect from the physiology of gap junctions yeah that's an excellent question so um in our previous work we estimated that the approximate coupling strength or spread lateral spread from one bipolar cell to the next is between five and ten percent of the current from one terminal to the next um and that seems like a reasonable um and you know it's fairly directly measured and that seems like a reasonable physiological um possibility and really in in that range and we've played I played with a little higher ranges of coupling but in that even that lower end of the range um you still see this effect of of the predictive encoding um hi Jeff we've been zoom bombed by Jeff hey man so Brian Jones is asking uh can you elaborate a bit Mike on how uh frequency hurt cells might contribute and are their uh contribution horizontal cells horizontal cells might contribute and are their contributions dropped out with longer time constants um yeah so one of the things that is is present in the paper is that there is a little bit of predictive encoding in the horizontal cells um and and I don't know exactly how to think about their contribution but it's something is so I guess the the take home is that there is something there particularly for for translational motion they seem to be very sensitive to that and there is some predictive information that's being encoded it doesn't make it to the theory close to the theoretical limit as we see in the bipolar cells and so um that's a great question to look at but in terms of the talk or in in terms of this project we saw that somewhere between the input to the horizontal cells and their their output there is this emergence of nearly optimal predictive encoding so we wanted to really focus on that for the purposes of of this project but yeah it's it's a good question um so Mike you tell us a testing the model further and this is a bastard question because gap junctions and kind of modulating I guess that's what you want to do ideally um um but I know they're difficult to kind of you know pharmacologically target but um as I understand they are modulated by for instance dopamine probably in the outer arena as well as in both um but I just wonder if that's an interesting direction that you're thinking about going in you know how might modulation the gap junctions affect your your measures of information kind of experimentally as well as within the models yeah that's it's a tricky one and and one of the this has come up repeatedly in some of the recent work we've done because these these gap junctions seem to keep coming up as being important and it's tricky as you mentioned because pharmacological manipulations are pretty dirty in terms of their effects and um for example if we wanted to block them with mechalphenamic acid or something like that there's a very tiny at least in my hands there's a very narrow time window in which you have the effects and you don't like destroy all light responses to tissue and just the amount of data I didn't really emphasize it but the amount of data that we had to collect from each and every cell in order to do this analysis adequately um was tremendous and so so that's why we sort of focused on trying to do this in terms of the model yeah but yeah it's it's something to think about sorry about can you manipulate dopamine levels for instance and even if even if you can't identify the kind of molecular mechanism at least ask questions about how the circuit state of the retina as affected by dopamine for instance might modulate um this aspect of the you know um yeah yeah it's it's a good good idea often noodle that one um because it seems like some it would be cool to to mess with the gap junctions in that way yeah I mean yeah just the kind of as a proxy for kind of adaptive state uh and how how that all impacts on the on this potential mechanism yeah um so let me see what we've we got yes golly loads of questions this is great so Matthew Yedudenko at my casks very very time reported the motion anticipation critically depends on the contrast gain control and fails for low contrast stimuli below uh 33 percent webber such stimuli cannot induce gain control have you observed similar effects so yeah does it break down below a certain contrast no it's um and I can show you so this is present in the paper and if you want I can share my screen and show you a plot from the paper to go ahead yeah um so I actually came prepared today for that question um let's see here so it we actually see the opposite so um that the work from Michael Barry and Marcus Meister did show that um they're looking really at as a as a moving bar encroaches on the receptive field you have this gain control mechanism that shuts down the response and so um the response of the cell leads the position of the bar on the photoreceptor array um and so essentially what we see here so again this is uh three different contrasts where we've we've tested um the predictive encoding and so this is 25 contrast 50 percent and 100 percent we can see that even at low contrast the um the predictive encoding is really quite optimal um quite close to optimal and we go in the paper for reasons that that is the case but essentially um it is we can explain it in terms of that electoral coupling non-linearity sort of mechanism that um the that these data the way that I interpret them and I know this is somewhat controversial is that the these data indicate that these circuits are prioritizing predictive information so even at low contrast that is actually the information that's encoded first and at high contrast then you can start to encode some of the past information that lacks the correlations um now the question of gain control mechanisms Fred and I have actually chatted about that um and whether gain control mechanisms could contribute also to this um and we're we'll look at that but in terms of the contrast gain control that Meister and Barry were showing where that's so critical that clearly not the case here okay okay so I think we're it's about an hour now um now um we can I think we'll officially end now but people who want to carry on chatting are invited to enter into the zoom chat and Maxima are you going to put up the uh the meeting ID for the zoom chat in the chat everybody so if you want to join do it now we'll close the YouTube streaming great that's great so anybody who wants to click on that zoom link to join the kind of more relaxed chat afterwards but before we leave I just want to thank Mike very very much for a really interesting talk full of new new ideas for me at least and um beautifully explained as well thanks very much Mike thank you thank you so much I appreciate I appreciate the opportunity so thank you okay great great Maxime do what do I do do I do anything