 που είμαστε ουσιασμένοι. Γεια σας, καλύτερα και ευχαριστώ για έναν άλλο σημεριό της Σεμίναρς Σεμίναρς, όπως πάνω στην οικογένειο της Worldwide Neuro-Initiative. Είμαι ο Γιώργος Καφετζής, ένας οικογένειο της Σεμίναρς Καφετζής, και τώρα, ένας ΠΕΑΤΑΤΑΤΑΤΑΤΑΝ. Και αυτός είναι το σημεριό σας για σήμερα. Θα ήθελα να ξεκινήσω, πιστεύω, για να ευχαριστώ για την Ελλάδα Βόγγελς και Πάνω Σποζέλος, για να παίρνουμε αυτή την οικογένειο της Σεμίναρς, αυτή η σύμφωνα της Σεμίναρς Σεμίναρς, στους Εγγρήνες και πολύ περισσότερες Σεμίναρες. Αυτό που είχα πει, φυσικά, να πάμε back to the reason we all gathered here for today, και να παίρνουμε το σημεριό μας από οικογένειο της Σεμίναρς Σεμίναρς, Πρόεδρο Φραίδρυγκέν. Ευχαριστούμε, Γιώργο, για τη σύμφωνα της Βόγγγελς και also for the opportunity to tell you about some of our work today. Οπότε, πρέπει να έχετε όλα my slides now. Λοιπόν, as Γιώργς είχα πει, θα μιλήσω για το σύμφωνα της Βόγγγελς, και στους πρόεδρος είχα πει, πρόεδρος είχα πει, πρόεδρος είχα πει, πρόεδρος είχα πει. Για να ήθελα, φυσικά, το σύμφωνα της Βόγγγελς. Και γιατί που βλέπουμε, στους πρόεδρος, είχα πει πρόεδρος για το σύμφωνα της Βόγγγελς. Προεδροί, όπως πήρα και έχουμε δεδρο Knowledge, όπως είχα πει πρόεδρος για θέα, πρόεδρος για να είναι για το πρόεδρο και να είναι για το πρόεδρο Vous. Many other things in nature, but there are a few things in that we have to understand. για να καταλαβαίνουμε, όμως, φυσικά, τα πράγματα που really care about how the systems work for. Λοιπόν, ας ξεκινήσουμε μόνο μόνο γιατί αυτό είναι ένα δύσκολο πρόβλημα. Και οπότε η καταλαβαίνηση που έχουμε είναι από την αρκετή στιμβουλία. Στην περίπτυξη, τις σκοτές, τις γραμμές και το νοτισμό, το πρόβλημα που βρίσκουμε είναι πολύ πιο δύσκολο, και αυτός αφήνει πίσω, γιατί θέλουμε να καταλαβαίνουμε κάτι like how a natural movie, αυτός, είναι εγγραφότητα by a cell that's paying attention to an area of the movie, like the circle here. And so the issue is how does the spatial temporal and chromatic pattern of inputs within the circle get encoded by the spike response that you see on the right. And if you see the answer right away, you can post that in YouTube and we can dispense with the rest of the talk, but I think you probably appreciate that this problem is difficult and it's difficult in large part because of the complexity of the input and what do I mean by that? I mean that particular natural inputs have highly correlated structures. So you might propose that the cell that we're looking at the responses of here encodes the shape of a flower petal and so it's going to fire strongly whenever a flower petal comes into the field of view. But it might instead respond to purple and purple and flower petal shape are highly correlated here. It's very difficult to disambiguate that type of correlation in the encoding. And that's one of the issues that's made studying natural inputs difficult is we don't have the ability to causally relate properties of the input to properties of the response. And related to that is we don't actually understand the statistics of natural inputs and sufficient detail to generate truly naturalistic stimuli with control that are based on the control parameters. This is further complicated by the fact that these artificial stimuli are often designed to highlight specific circuit mechanisms and engage those mechanisms in particular. Whereas complex inputs like this are going to engage multiple nonlinear mechanisms that are interact and we really don't understand the interaction between those mechanisms because most of our understanding has come from approaches that highlight single mechanisms. Or of course not the only people to think about this as being an important problem. And so what do I think we can contribute there? One of the things that I think has not worked so far is that purely data driven approaches record long sections of responses to natural inputs and use sophisticated statistical approaches to uncover how the system is working. Those approaches generally haven't worked super well. So we're going to take a somewhat orthogonal approach to that and try to manipulate the stimulus in identifiable clear ways that hopefully give us more insight into how the system is working. I should also say that this is an important problem for several reasons. One is just our general understanding of sensory function should incorporate things like how these multiple nonlinear mechanisms work collectively to encode complex inputs. It's important for understanding a division of labor between the retina and the cortex. We really need to understand how retinal encoding is working to understand what additional properties are occurring in cortical circuits. And I think it's important from the point of view of designing prosthetics. We need good models for the encoding of complex inputs if we're going to really exploit the full power that appears to be on the horizon in terms of retinal prosthetics. OK, so as I said, this is a general issue that is true not just of vision but also true of other sensory systems. There's some clear advantages of studying this in the context of the retina. This is a beautiful cross-sectional view of the retina from Rachel Wong's lab in which the different major retinal cell types have been labeled genetically. So the photoreceptors are here on top, the cones. Signals that originate in the cones are then relayed across the retina through these green bipolar cells to the ganglion cells, the output cells at the bottom. And that straight-through pathway is, of course, modulated by lateral interactions provided by the horizontal cells and anachronous cells. So one of the clear advantages of studying retinal circuitry is that from decades of beautiful anatomy, we have a near-complete list of the components of the retinal circuitry and an emerging list of the connections between those components. So we can study that known circuitry, for example, the subtypes of bipolar cells here. We can study that known circuitry with behaviorally relevant stimuli. So we can kind of run from kind of biophysical approaches that really tell us about how individual cells are working out to how those cells are working in the context of real light inputs. And together, these really constrained models for how neural coding will work and we'll try to make use of some of that background information as we develop stimuli to try to get at this issue of how natural stimuli work. Okay, so what approaches are we going to take to this? All the work that I'll tell you about is from non-human primate retina. These are retinas that we obtain through a tissue distribution program, the primate center here at the University of Washington. And I'll kind of give a shout-out to Chris English, who is a technician in that program, who's been just wonderful in helping us really make the work that I'm talking about possible. So what we do is we take a small piece of retina, RP Attach, we put it photoreceptor side down on a glass cover slip that forms the bottom of our recording chamber. We then project light stimuli such as natural images onto the photoreceptors while we record response to the different cells within the circuitry. And I'm going to use, I'm going to take a little bit of a tangent here. So as George mentioned, I was a postdoc with Dennis Baylor and, as many of you know, Dennis passed away about a month ago. And it was, for me and many others, working with Dennis was highly impactful, really set my career off in a direction that would not have gone without that time that I had with him. And that's true for many, many people in this field. Dennis used to start talking about absolute visual sensitivity with a statement something like, if you take a sugar cube and you drop it an inch and you take the energy you obtained from that and convert it to blue-green photons, you can deliver the intense sensation of light to every human being who's ever lived. Which I thought was just a nice way to, it was a very Baylor statement. So I'm going to propose that we should use sugar cube inches as a new unit of light intensity. You know, we've got trolons, we have foot candles, we have some somewhat arbitrary unit, so sugar cube inches should be in there. Okay, so kind of back to the non-tangent part. We're going to focus on two cell types, on and off parasol cells, and we will identify those by the responses to a light step initially. So the on parasols fire a transient burst of action potentials at the onset of a light step, the off parasols fire a transient burst at the offset of a step. At the end of an experiment, we can confirm the identity of the cell by filling it with a dye and imaging the dendrites. So here we're looking down on the dendrites of the cell. We've focused another 150 microns or so in, we find the further receptors. Of course, the dendrites are collecting those further receptor inputs that are getting conveyed across the circuit. Okay, so we're going to record from these cells using this recording configuration and try to understand how these cells are integrating spatial information from the stimuli that we deliver. So any discussion of how spatial integration works, how to start with the classic center-surround receptive field. So we know from work from Barlow and Kupler back in the 1950s that cells in the retina showed this center-surround receptive field organization. And so what I mean by that is that there's an area of the input, which is this solid circle, and when your cells integrate their input, and that's opposed by the input that's collected from a larger area, such as the dash circle. The center is comprised of the bipolar cells that convey inputs within the center through the circuitry to the ganglion cell. And the notion of the center-surround receptive field is that here we have the center looking at the response over a position and the surround, the dash line, and the combination of those two gives us the difference of Gaussians center-surround receptive field. That receptive field organization continues to form the core of many modern predictive models for ganglion cell responses with linear non-linear models, generalized linear models, start with a linear spatial temporal receptive field, and then follow that with a post-filtering non-linearity, kind of single non-linearity after integrating signals across space. So the canonical assumptions that this model is based on is that there's a linear spatial integration in the receptive field center and surround, and that there's a linear combination of the center and surround prior to this late kind of post-integration, post-filtering non-linearity. That in turn kind of supposes that all of the circuitry within the retina is operating linearly to convey signals from the fraud receptors to the ganglion cells. And we know from lots of other work but that's not the case. Okay, so from a long history of work, not only do we know that there are important non-linearity within the retinal circuitry, but we also know that this center-surround receptive field, even with a post-filtering non-linearity is insufficient to account for responses and many stimuli that includes natural inputs. And I've listed a subset of the work that's sort of made that point here. Okay, so that brings us kind of the core of what I'm gonna talk about today. It's really the question of can we do better than that? So this is another representation of the classic center-surround receptive field where we take a weighted sum of the pixel values. So i is gonna be the value in the pixel small i. We weight those inputs, sum those, and then hit that with some non-linear function f. So that's not gonna do it. Many of those classic studies support instead a picture in which there are subunits within the receptive field and those subunits are composed of bipolar cells. So each bipolar cell collects its inputs. Those individual inputs are again hit with a non-linearity but that non-linearity occurs prior to integration across space. Okay, so now we have a non-linear function that's applied to the bipolar signals. Those are then weighted and then those are hit with another non-linearity at the level of the ganglion cell. Now this model, so this is certainly a more accurate model based on a number of studies. However, this is a pretty complicated model in the sense that we need to know the location, size and shape of each of the individual bipolar cell subunits and one might worry that that's a little bit, that's a lot to ask of a cell that's reading out signals from the retina to know where all the subunits are in order to interpret the signal that's coming from this ganglion cell. So we're gonna look at something, we're gonna see if there's a way that we can simplify that. So we're gonna be kind of inspired by this architecture of a model, but we're gonna ask whether we can be freed a bit from really needing to know the location of the individual subunits. So I'm gonna call, instead of calling these subunits, I'm gonna call things subregions from now on so that you don't make a direct correspondence between the region and a bipolar cell subunit, a real anatomical subunit. So our goal is gonna be to identify the simplest model that we can that captures the responses of parasol cells to natural stimuli inspired by this general architecture of the model, but not constrained by the number size location of the individual subunits, bipolar cell subunits. Okay, so we'll be asking questions like how many regions do we need? What do we need to know about the location and size and can we, once we've answered those questions, can we identify something about how those different subregion signals are combined to generate the parasol responses? We're gonna approach these questions empirically and the core of this is gonna be to generate what we'll call linear equivalent stimuli and illustrated those here and shown over in the movie on the right as well. The linear equivalent stimuli consists of taking a frame of a natural movie in some region of that frame such as this circular region here. We then instantiate the assumption of linear integration by integrating the inputs across this region and then using the result of that to adjust the contrast of a spatial uniform region that spans the same region of space. So this linear equivalent disc in this case directly instantiates the assumption of linear integration over space. In other words, if the cell is integrating linearly over this region of space, responses to the linear equivalent disc in the original movie should be identical. We do that for every frame of the movie and then we deliver these two movies and ask whether we indeed see similar responses or not. So again, if the cell is integrating linearly we should see similar responses. If it's not integrating linearly then this should fail and we should see different responses, okay? And this just shows that again so this is a different natural movie with the original movie on top and the linear equivalent movie on the bottom. Let's say all these are movies from the DOVs database. So these are, we're using movies of static natural images. The images came from the original Manhattan database and then we're using eye movement trajectories of humans that are viewing those images and that's how we are moving the image across the surface of the retina in the context of our experiments. Okay, so let's look at how this plays out one example. So this is one particular natural trajectory. Here's the original movie over frames, the linear equivalent and these movies are six seconds long. Here's the response to the original movie. Here's the response to this linear equivalent disc movie. Those look pretty similar. So the next step is to quantify that similarity and to do that we're gonna use a spike distance metric that was introduced by Victor and Purpura. And the basis of the spike distance metric is to we have two recorded spike trains and the spike distance metric works by converting one spike train into the other through two different operations. One is deleting and adding spikes each of which has a distance of one and the other is by shifting spikes in time which has a distance that's given by some cost per unit time of shifting. The conversion and the way that algorithm works is to find the smallest distance associated with converting one spike train into the other. A nice thing about that is it gives us a way it's not the only way that we can compare to spike train but it's a nice metric to use to compare the similarity or dissimilarity of two spike trains. We're gonna bound the spike distance by two different bounds. There's a lower bound which comes from the distance associated with repeats of the same stimulus. So the distance of two different responses to the precisely the same stimulus that gives us a measure of the noise in the system that's causing spike responses to differ from one response to another. And then we have an upper bound which is when we randomize the spike trains. So in other words, if we have the same number of spikes but they're randomly distributed over time then that gives us an upper bound. And we'll define the explainable spike distance then as being the difference between our predicted and measured response corrected for the lower bound divided by the upper bound. This gives us a measure on a scale of zero to one where zero would be our predicted or measured spike train to stimulus A is very different from that to stimulus B and this kind of equivalent to a randomly dispersed set of spikes and an explainable spike distance of one would be we're perfectly replicating the structure associated with the spike train. So now coming back to this example in this case, the explainable spike distance is 88%. So the linear equivalent disc movie here does a nice job of replicating the response to the original stimulus. And this is I'm showing you one example spike response here. This is measured across many different repeats to the same set of stimuli. So now movie things look pretty simple. We can replace the spatial structure on the receptive field center with a linear equivalent disc. Everything looks good for kind of a classic center surround type receptive field at least with respect to the center alone. Here's another movie in which it's quite different. So this is another image from the Manhattan database and we've tried to choose images that are kind of at the extreme ends of the spectrum in terms of having lots of spatial structure or a little spatial structure. In this case, here's the response to the original movie. The next row is response to the linear equivalent disc movie. Those responses look quite different. And so now what we're gonna do is start to subdivide the center into subregions. Each of these subregions will be treated identically in terms of generating a linear equivalent stimulus in say this upper right quadrant, this lower right quadrant and so on. So we're gonna get a movie that has four different linear equivalent regions. Each of those regions then has the capability of being processed by a separate non-linearity that's present in the circuitry and that's the critical thing. So once we've integrated the structure, we've washed out all the spatial input into a single number and that number that we have no capability of independently applying a non-linearity to subregions of the original movie. If we divide this up into quadrants, there can be a non, we've integrated the structure in the sub-right quadrant but that structure can then, the response generated by that structure can then be hit by its own non-linearity. That can be different than the non-linearity that we hit the structure in the lower left quadrant, for example. So this is very much like bipolar cell subunits but the regions that we've defined here don't have any obvious correlation or correspondence to the subunits. In particular, we don't really expect subunits to look like slices of pizza. We've divided this somewhat arbitrarily in a way that we really span the full receptive field center and I'll come back to that issue later and we'll test a little bit from the issues associated with dividing the center in this way. But I think as you can see, this is a nice simple way that we can divide the center without having to have knowledge of what the individual true anatomical subunits looks like. As we subdivide the center, this explainable spike distance goes up. So it starts around 50%. When we've fully integrated the center to a single linear equivalent disk then that goes up as we add subunit. Corresponding to the requirement that we have individual spatial regions that can be hit by separate non-linearities. Okay, that's true for that movie. Here I'm going to show you through across a number of different movies and a number of different cells. These are on parasols on top, off parasols on the bottom. And here we see first of all that there are some differences. So sorry, this is explainable spike distance plotted against the number of regions that we divide the center into. There's some differences across different images. These are the two that I showed you a second ago. You see something similar in the off paracels. But across movies and for both on and off paracels cells, we see that there's an improvement of our ability to recapitulate structure in the original movie as we have subregions. And something like eight to nine subregions is sufficient to kind of saturate that improvement. We're also able to recapitulate almost all of the structure in the original movie by using that number of subregions. Okay, so kind of conclusion of that piece of analysis is that something like eight subregions is sufficient to capture the sensitivity to spatial structure in the receptive field center. We do something very similar for the surround. So here what we're doing is putting a natural movie in the center and then either having a fully integrated surround or chopping the surround up into subregions. And again, we're gonna plot the explainable spike distance against the number of subregions now in the surround. We again see that there's some improvement as we add subregions in the surround. That again is dependent on the image that we use but when average across cells and across images we see a systematic improvement. Again, that saturates somewhere around eight subregions. And there's nothing magical about eight. Eight was kind of a compromise that we reached that allows us to achieve kind of optimal performance across different movies. And as I've shown you, there's some movies for which we don't need eight for which a single subregion will be sufficient. So the bottom line of that then is that dividing the center and the surround into eight somewhat arbitrarily placed subregions seems to recapitulate responses to the original natural movie. So let me show you what that means in terms of a movie. So here's the original natural movie. This is centered on the receptive field center. If we chop or reduce that movie to the movie seen on the right, what we find is that the movie on the right generate nearly identical responses as the movie on the left. So in other words, all of the essential spatial information from the movie on the left is captured by the movie on the right. Okay. So we'll come back to why I think that's important in a few minutes, but the next thing we wanted to do is really test is that statement that I just made true? Is it in fact true that the movie on the right here captures all the relevant spatial information from the natural movie on the left? And the way Julian did this was depict this kind of schematically. So imagine that there's a space spanned by all of the couple hundred cones that provide input to a parasol cell. And in this space, we've identified the relevant coding direction of this gray line. So that's the direction or the projection of the natural input onto our 16 dimensional reduced movie. Okay, so the statement is that all that the cell cares about is where you are along this gray line. And that completely captures the cells response. That's the thing that we'd like to test. So if that's true and we add structure that's orthogonal to this gray line, that should be irrelevant from the point of view of the cells response. We can add kind of whatever structure we want along this orthogonal direction. Okay. So Julian tested this by generating movies which have the identical location along the gray line but very different spatial structure. And we can then test whether those movies indeed elicit similar responses. Again, this is going to be clearer if I just show you a movie of it. So here's the original movie on the left. The reduced representation in the middle and what we're going to call the spatial metamer movie on the right. And I think you can appreciate right away that really the only thing that these two movies share is this same reduced representation in the middle. The way the movie on the right was constructed was for every spatial region. Julian went to the image database and found another piece of an image that had the same projection in a reduced representation. Did that independently for every different spatial region and for every frame of the movie. And that's why this looks like such a hodgepodge of different images is there are no correlations between the spatial structure in one wedge and the other wedge here. But I think you appreciate that what we've done by doing this is added a lot of structure to the image that doesn't influence the reduced representation that you see in the middle. But we've added kind of as much structure as we can while retaining this same reduced representation. Okay, so then the question, of course, is does the original movie, do these three movies generate similar responses? And that's what's shown here. So here's a response to the original movie, response to the reduced movie below that and then the response to that spatial metamer. Here, two different instances shown here. And I think you appreciate just by looking at the responses that the metamers indeed capture much of the spatial structure of the input. And as I can show you more if you're interested that we've quantified this again using explainable spike distance and by that metric, this looks quite good. So the conclusion from that being then that we can indeed add a lot of structure to these movies as long as that structure doesn't influence where we lie along this coding direction, it seemed to have little impact on the responses. And so that's a nice test that in fact this reduced representation captures the key aspects that are controlling the parasol responses. Okay, so let me come back to this question then of how this could work given that we fairly arbitrarily divided the center up into subregions. You know, we have these pizza slice things. What are those? Those don't correspond in any obvious way to a real anatomical subregion. So okay, so pizza slices don't correspond to anatomical subunits and kind of related that we don't know the location of the real anatomical subunits yet we have these hard boundaries between the regions that we've demarked. And so we wanted to test that a bit more and I'll show you just one of several tests that we've done along these lines. So the first thing that Julian did was he took his original natural movie and he swapped different regions. So he randomized where the regions are with respect to each other and then generated a new movie. So now all the regions are shuffled with respect to each other. And the question was, did that elicit similar responses? So in other words, are the responses, for example, within the center invariant to where these different pizza slices lie within the center? Can I swap things around in the center and still get a similar response? So these movies look something like this. There's a original movie on the left, movie in which the different regions have been swapped on the right. Again, the question being, do we see similar responses to those two? So here's the results of that. This is, here's an example of responses on the top. So the original movie, two different shuffles of that movie. See the spatial structure of the responses looks pretty similar. If we quantify that in terms of explainable spike distance, those explainable spike distance across a number of different movies, we see that that explainable spike distance is quite high, it's around 95%. So there is a systematic difference between the response of the shuffled movie and the original movie, but that systematic difference is quite small. So most of the structure that's relevant for driving responses is retained even when we shuffle around the subunits. I think this is a piece of the answer as to why shopping the center up into these somewhat arbitrary regions works is that the size, shape, and the location of those subregions are not essential for producing the responses. And that's not to say that you couldn't design stimuli for which the size, shape, and location would be important. What it says is in the context of natural inputs, size, shape, and location are not critical. I think that's really a statement about the statistical piece of the natural image that's driving the responses is well captured even when you shuffle the subunits around rather than a statement about kind of across all possible stimuli how the receptive field works. But seeing the receptive field is constrained by the fact that it only sees a certain set of statistics. And as long as the structure of the receptive field is invariant or isn't being driven by statistics that really highlight the shape or location of the subregions, then we may be able to shuffle them around. And that's a key simplification in terms of our ability to generate these models. And I think it's also a key simplification in terms of cells that might be responding and interpreting the responses coming out of the retina. Come back to that very brief plane in a second. Okay, so hopefully at this point you have some sense of maybe we have identified this effective ability to reduce the dimensionality of a natural input into this lower dimensional space. And I wanna say just a couple of things in closing about why this might be useful. And one of those is to characterize kind of the key computations that shape retinal encoding. And what that's really gonna focus on is how do the activity in these different regions come together to dictate the ganglice or response. In other words, we've taken a natural input, we've projected it down to the 16 dimensional space. How do we get from the 16 dimensional space down to a one dimensional spike output? And we know from a good deal of past work that surrounds appear to be added to bipolar cells prior to the output non-linearity of the bipolar cell. So that suggests the model in which center and surround come together. Each of those is then hit by non-linearity and those get integrated to form the ganglice or spike output. And I'm not gonna take you through the kind of complicated pieces of the slide on the right, except to say that Julian tested a whole bunch of different architectures that consist of different orders of combination of center and surround and non-linearity. And what that supported is that this is a, this seems to be a pretty accurate picture of for natural inputs of how to get from a center region and a surround region, get those combined, hit those with a non-linearity and then combine those to kind of the 1D spike output. So that model with this architecture appeared to capture the responses most accurately. Okay, so that suggests that center and surround appear to be combined prior to the output non-linearity of the sub-units. And I want to say briefly why this might be important. This is work from Max Turner that's published. I'll just kind of summarize it very quickly. But the idea of this, of Max's work here was that that might mean that surround activation can control the degree of non-linearity of the receptive field center. So in other words, if activation of the surround if we're looking at the bipolar output sent out to here, we're looking at synaptic release versus pre-synaptic voltage. We have some non-linear synaptic input output relation. If activation of the surround hyperpolarizes the bipolar cell, it could move it to a region of very little sensitivity. If it depolarizes it, it could move it to a region in which the synaptic output is basically linearly related to the pre-synaptic voltage. So the synaptic would be in a kind of linear operating regime. And if it's sitting kind of at its normal resting point, you might notice that the synapse is highly non-linear. So in other words, surround activation by pushing this synapse up and down its input output curve could control the effective non-linearity of the bipolar output synapse. That non-linearity is critical for whether spatial integration is linear or non-linear. If the bipolar cells are all sitting up here, then spatial integration is effectively linear. If they're sitting down here, it's non-linear. Okay, so that was a prediction. A way to test that prediction is to take a piece of a natural image and we're gonna compare the encoding of that natural input, that natural image to the encoding of a linear equivalent disc. So by comparing these two, we can monitor the degree of spatial non-linearity and the response to activation of the receptive field center. We're gonna do that while systematically changing the activity of the receptive field surround. And so the prediction would be then if we, this is in an off parasol cell. So if we use a bright surround, we push the bipolar cells up to a place where there's synaptic input output relation is fairly linear, and we should see similar responses to the disc and the image, like the green point up here. But it's sitting at rest without surround activation. We might see a larger response to the image than the disc, the bluish point here. And if we have a hyperpolarizing surround that pulls it down here, then we might basically suppress responses altogether. And what Max saw was aligned with that quite nicely. So here are responses to the image and the disc. These are flashed on for a couple of hundred milliseconds. The much stronger response to the image than the disc. So if we plot those against each other, that's this point. If we then start to systematically manipulate the surround, if we have bright surround, the response to the image and the disc are quite similar. Those are the points up here. If we have a dark surround, we suppress the responses strongly, the points down here. So it's the difference between the image and the disc response, which a measure of the degree to which spacial integration in the receptive field center depends strongly on activation of the surround. And that's due to this architecture that I showed you here where the center and surround come together prior to the output non-linearity. So the surround then has access to the center signal before it gets hit by the non-linearity. As a consequence, it can control the degree to which the center signal is non-linearly processed or non-linearly affected by the bipolar output signals. Okay, so one other thing I'll just mention really quickly is that this reduced representation also gives us, it gives us some ability to identify different stimuli which are degenerate, certainly from the point of view of individual cells and we hope from the point of view of populations. So these degeneracies, in other words, stimuli that elicit the same response come first because we have the same activation of subregions and I showed you this from the point of view of metamers very directly. They can also come from the point of view of stimuli that are integrated in the same manner by the function that takes us from a point in our 16D space, a point along this coding direction to the ganglion cell spike output. And without kind of belaboring that too much, what we hope we can do with this is begin to consider populations of cells and identify stimuli that might be, for example, illicit similar responses from parasol populations but very different responses from midget populations and this may give us some traction on identifying stimuli that help us separate responses from midgets and parasols and begin to use those to think about perceptual phenomena that are listed by midgets and parasols separately or higher level responses that are specifically dependent on midget or parasol responses. Okay, so let me summarize what I've said then quickly. We wanted to develop an approach that allowed us to identify how spatial structure and natural input was integrated by the ganglion cells and we found that we could do this to the classic centers around receptive fields or to provide an incomplete description of that process. We found that we could instead, using an empirical approach, identify a reduced dimensional space, eight dimensions in the center, eight dimensions in the surround that captured the key structure of natural inputs. We think that's interesting and important because all of the relevant spatial structure is in this lower dimensional space and that gives us the capability of describing that structure and hopefully identifying the relevant correlations for example, in natural inputs that are really driving responses in the cells. And another application is that we may be able to use this to identify degenerate stimuli and we can certainly do that in the case of single cells and we hope that we can do that in the case of populations of cells. Okay, finally, thank the people who are involved in the work and this is largely work from Julian Friedland who is a student in the lab right now. This is really initiated by Max Turner who is a student in the lab who is now a postdoc down at Stanford. Okay, so with that I'll stop and then definitely welcome any questions you have. Thank you for your time. Thank you very much Fred for this wonderful talk. There are already questions appearing in the chat and I will make the Zoom room address available in a second. So as you don't have the YouTube tab open I would like to let you know that there are people that greeted you in the beginning and people that are thanking you now in the end. There are a lot of questions. So for times economy, I will start with the first ones that appeared. So first one is from Marla Feller. Are the recordings in foveal or peripheral regions? Either, yeah, we would love to do this in the fovea. But these are all in the periphery. So these are all between 20 and 45 degrees eccentricity. There are some technical challenges associated with getting more central. And so we're definitely working our way in but it's more technically challenging to get towards the fovea. Right, so a kind of related question is from Tyler Godot and I'm quoting I think an earlier slide so the parasol cell at 28 degrees, how does the size of those roughly eight subregions compared to the size of photoreceptors at that eccentricity? Yeah, good, no good question. So the regions that we're using have, let's say 30 to 40 cones within that region. The anatomical subunits, the bipolar cells receive converging input from about eight to 10 cones. So our regions are about four times larger than the bipolar cells. One question of mine related to what you just described and from what I see in the chat, something similar Ana Vlacic and Marla want to add. But like my question is given that you rely on these saccades to refresh the images that you play and like we don't have foveal but para foveal or peripheral regions. Do you think time would play a role in how many subregions you need or maybe the role of time could kind of compensate for different subregions? Yeah, no, that's a great question. And we, and I should have said this earlier. So thanks for the question because it helps me clarify something that I just sort of forgot to say. We have, I mentioned earlier that is that those sort of data driven approaches have struggled to find clarity in an answer to how natural inputs are encoded. And one of the things that we have tried to do our dog is waking up and so he wants he wants some attention. One of the things that we try to do is simplify the problem. And we've done that in part by focusing on single cells and part by centering our stimuli on the receptive field and so on. But we've also done that in part by really kind of punting on time so we are not doing anything to simplify the temple structure of the stimuli. That's just present in whatever the eye movements were. And we've simplified the spatial structure and really focused on that spatial integration part. We would love to say more about time. I think that's kind of the next, the next thing that we'll start to play with but for now those subregions that we define the time course of the input and those subregions is dictated entirely by the eye movement in the original movie. And whether there are other, for example, adaptational mechanisms that are operating on a smaller spatial scale that we might be able to reveal if we start to manipulate things over time. And one of the things that Julian has done a little bit of is to interchange the reduced representation and the original movie on each temple frame. So you're basically flipping between a natural movie frame and the reduced representation and then a natural movie frame and the reduced representation. Is it really true that that reduced representation is still holds when it's being interchanged with the original movie over time? And the answer to that is it doesn't hold fully and that's probably revealing some adaptational mechanisms that are representing kind of an interaction of space and time in an interesting way. And we have that's kind of on the list of things to understand better. Right, yeah. So in case Marla or Anna this doesn't answer completely your question please join us in this room room. I have already posted the link. And given the context like I will suffer the questions that appear you mentioned that time will be one of the next things. What about color? So Tom naturally asks have you tried to color your pizza slices? Yeah, no color. But yeah, especially if we get more central that will be particularly important. The next one appearing in the chat is from John Ball. What is the variability in sub-region responses to repeated presentations of the same videoclip? Like have you tried to repeat the same movie? Yeah, we definitely we use kind of generally we use about 10 repeats of the same movie. And of course I had to answer kind of quantitatively how much the variability is, but it's quite small. And if I lined up two different repeats to the same movie, I showed you for example in the Metamer movie I showed you the original movie and then the reduced representation and then the Metamer responses to repeats are somewhat more similar to each other than those examples that I showed you not dramatically more similar but somewhat more similar. And along these lines before I continue with the questions that appear in the chat these movies are like about 10 seconds long and like from one slide that you showed like the original movie Suffle A and Suffle B it looks like for Suffle B the response to the end of the stimulus gets faster and faster. Is there like some history dependence in the responses that you observed? Yeah, and it's kind of related to whether there are adaptational mechanisms that are shaping, right? And again, that's something we so likely yes but that's not something that we've retained all that temporal structure and not tried to manipulate it but I think it is that's an important question and it's one that we need to get to just so as not to get just overwhelmed by the complexity of the problem it's not the first thing that we went after. So the next one is from Tim Gollis actually right now there are two more questions appearing in the chat so I would like to ask our audience to join us in this room because I will be terminating the live broadcast after we are done with these two questions. So I'm posting the link again there and as I said, the next question is from Tim Gollis. Thanks for the great talk this around would contribute both before and after the local non-linearities. Is there an advantage in the model to consider two separate surround contributions? Yeah, good question. We haven't, we've not thought about that. So part of that is whether we really have the leverage in the way we did the experiments to differentiate that. I suspect that we don't have the data that would really allow us to pull apart another role of the surround that's post non-linearity but we can design experiments that get at that in more detail. So we really just considered kind of the pre-non-linearity versus post non-linearity and not a combination of the two but it would be certainly would be interesting to do that. One of the things that was true throughout the experiments is that and this isn't shocking but the activity of the surround in the parasols is not a huge impact on their responses. It's there, we can measure it but the center is really what's kind of running the show for the most part. And so I think these issues about the impact of the surround might be better served by looking at other cell type for which the surround is a stronger stronger influence. Thank you very much for that and the last question as people have already joined us here in the room is from Weili. Does the near optimal number of the subregions correlate with the size of the receptive field of the parasol cell? We haven't seen any correlation between that but again, as we haven't gotten really central we're only looking maybe at one and a half to two-fold changes in center size across the range of data that we've looked at. So really the right way to look at that would be to get more central when the receptive fields are really shrinking down. Right. Thank you very much Fred for this fantastic talk and thank you very much to the audience that joined us for another of our seminars. I will be stopping the live broadcast right now and we will continue in an off-light mode. Thank you all. Great. So we are officially