 and we are live. Hello everybody and welcome to another session of our Sussex Vision Seminar Series, always within the Worldwide Neuro Initiative. I'm George Cafegis, a former master's student in Thomas Euler's lab and currently a PhD student with Tom Badden. And as your host for today, I would like once again to begin by thanking Tim Vogels and Panos Bozellos for putting forward this very initiative towards a greener and much more accessible seminar world. And of course, having said that, allow me to get back to the reason we all gathered here for today and introduce our guest from Duke University, Professor Greg Phil. Greg received his PhD in physiology and biophysics from the University of Washington where he worked with Fred Rieke and investigated the limits to absolute visual sensitivity. He then moved to San Diego and the Salk Institute for Biological Studies where he worked as a postdoctoral fellow with EJ Cicilniski and following highly productive years with both. He spent a couple of years at University of South California as an assistant professor before joining Duke University in 2015 where he has been located ever since. Recipient of many personal awards for his influential research and with interest revolving around populations of retinal ganglion cells and different encoding strategies. It is with great pleasure that I'm living this stage for him for a talk entitled Efficient Coding and Receptive Field Coordination in Diretina. So without any further ado from my side, please all welcome Professor Phil. Greg, the stage is all yours. Thank you, George. Let me get my slideshow set up here. Can you all see that? Yes, crystal clear. Awesome. So thank you for the introduction. It's a real pleasure to be here today. I just really wanna thank George and Tom for the invitation to speak. This series has been an immense success in my opinion and I'm incredibly pleased and honored to be able to participate in it today. So my lab is primarily interested in understanding how populations of neurons in the retina encode the world around us and how they transmit that information to the brain. And I really wanted to emphasize here populations of neurons because I think populations encompass one of the real challenges that we face in neuroscience. How are neural populations organized and what are the underlying principles, if any, that govern that organization? So I think the retina is a terrific system within which to begin to understand or discover the answers to those questions. And I hope as we go through the rest of my talk, I'll convince you too that it's a good system for answering those questions. So I'll start just with a quick overview of the visual system to orient folks. Light enters the eye and is focused at the thin sheet of neural tissue at the back of the eye called the retina here highlighted in red. That light gets converted by the retina into a set of electrical impulses or spikes that are then transmitted back to the rest of the brain. And it's from these electrical signals that originate in the retina that the rest of our brain constructs all of our visual perception. Now in broad strokes, we know quite a lot about the organization and the diversity of cell types that comprise the retina. For example, the photoreceptors detect light, they pass those signals on to bipolar cells which then eventually pass those signals on to retinal ganglion cells the axons of which form the optic nerve and these are the cells that transmit signals to the brain. These are the cells I'll be talking about mostly today. There's also a couple of types of inhibitory interneurons, horizontal cells and amicron cells that tune the spatial and temporal properties of information processing on the retina. However, the retina also exhibits a remarkable diversity of cell types and an intricacy of connectivity that is not well represented in this picture that starts to be revealed in this one. This shows how each of these classes of cells horizontal, bipolar, amicron and ganglion cells is actually composed of many distinct cell types. And while this slide is somewhat out of date at this point, it gets the point across in that there are a couple of kinds of horizontal cells, approximately 15 different types of bipolar cells, not all of which are diagrammed here. About 40 or 50 types of amicron cells and about 40 distinct types of ganglion cells. So altogether, there's something approaching 100 cell types in the mammalian retina that fall into these broad classes. Now you might very reasonably be wondering how we as a field of retinal scientists can be quite certain that a cell like this is truly distinct from a cell like that one. After all, they have some features that seem kind of similar while they have other features that are perhaps a bit distinct. How do we really know that they're distinct cell types and not just two points along a continuum? Well, there are many lines of evidence for that ranging from the fact that these cells have distinct functions and distinct gene expression profiles, but I really want to zero in on what I think is the most compelling piece of evidence that these cells are distinct types. And the reason why I want to focus on this is because I think it reveals an underlying organizational principle to the retina. So this principle is known as mosaics. Many of you will be familiar with that term and if you're not, I'll explain it now. So zooming in on one of these bipolar cell types, we see that the cell bodies of these, all of these particular bipolar cells are regularly spaced and the dendritic field has its own unique territory that tiles space, kind of like tiles in a piece of artwork or a mosaic. And so that's why we call these mosaics. And for this particular cell type, the axons also form a mosaic that beautifully tiles space. Now this organization is not just present in this one cell type of the mouse retina, but it's incredibly common across most, perhaps even all cell types in the mammalian retina. And this organization was first discovered in the early 80s by Heinz Bessel and his colleagues. Here's an image from that work where there are two types of ganglion cells in the cat that have similar functions. One encodes decrements of light, the other encodes increments of light, hence the off and on prefixes. And you can see that the dendritic fields of these cells beautifully tile space and the soma are nicely spaced apart from one another. These, this anatomical organization gives rise to a particular functional organization because the cones in this picture above the dendritic field of the ganglion cell are the photoreceptors that will ultimately provide input to this ganglion cell. And since space in the actual world is projected onto the physical space of the retina, this corresponds to the receptive field of these neurons. And so the anatomical organization of these cells tiling space leads to their receptive fields also tiling space. And in one of my favorite papers from Steve DeVries and Dennis Baylor where they recorded from rabbit retina on a 64 channel electrode array, they identified many different ganglion cell types with different functional or physiological response properties. And each one of these different ganglion cell types exhibit this mosaic like organization. Just to be clear what's being plotted here, each one of these receptive fields that we took a cut through it along one single axis would look like this classic difference of Gaussian's function where the cell is sensitive to say increments of light in the center and let's say decrements of light surround the ellipse here as a particular contour on that receptive field. So if you think of the receptive field as kind of like a mountain, that ellipse is representing one elevation on that mountain. And the neighboring receptive field would look something like this. So it's not that the receptive fields don't overlap which could impression could be generated by this image. It's just that they don't overlap at the one standard deviation contour at lower elevations, however, they clearly do overlap. And this functional organization was nicely pointed out in this paper that when you sum all of these receptive fields together when they have the spacing, you get a nice flat uniform sensitivity surface which seems like a very good design principle if you're building a retina, you'd like to be uniformly sensitive across visual space to any particular visual feature that you might be interested in. These are similar kinds of data recorded from the macaque monkey retina. These are various measurements performed by me and other folks in Egypt Shchitulnisky's lab and I was a postdoc there. And you can see that each of these cell types convincingly forms a mosaic. Occasionally we have mosaics where we struggle to sample all of the cells. I think there are really ganglion cells in these locations. Our measurement technology just failed to identify them. When we get patches of all of the cells, you see a nice mosaic organization. Okay, so I think this perspective on the retina leads to kind of a nice aesthetically pleasing way to think about visual processing and the organization of many diverse cell types in the retina which is that the retina is really composed of something like 100 mosaics all packed in with each other. And each one of these mosaics is representing and processing a distinct set of features about the visual world, whether particular locations are brighter than the background or darker, whether they're red or blue, whether there's motion to the left or the right, whether there's oriented structure that's horizontally oriented or vertically oriented so on and so forth. Each one of these mosaics represents a different cell type and a different computation. And I think this perspective leads to the following question. Are these mosaics coordinated with other mosaics? So what do I mean by coordination? Well, I mean, is there some kind of spatial relationship between the mosaics? And in general, I think there's three possibilities. One possibility is that the mosaics tend to be aligned with one another. So you could think of two honeycombs stacked on top of one another where you align the holes of one honeycomb with the holes of the other. That would be a representation of alignment. Another possibility is the mosaics could be anti-aligned. So imagine you take those two honeycombs and shift one so that the vertices are at the holes of the other. That would be anti-alignment. And another possibility is that they could just be statistically independent. So that, say, knowing the locations of the cells in this mosaic provides no information at all about the locations of cells in that mosaic. And this image here shows these two mosaics put on top of one another. It certainly doesn't look like they're perfectly aligned, but it's hard to reject the idea that there might be some tendency for the cells to be closer than expected by chance, at least by just visual inspection here. So there's these three possibilities, alignment, anti-alignment, and statistical independence. And I just want to motivate why you might even care about this question or why the answer might matter. So you can imagine that if you're the brain and you're trying to reconstruct the world from the signals coming in from the retina, there might be an advantage to having the mosaics aligned with one another because then essentially a mosaic is a grid of detectors. And if you have two grids of detectors, you might want to have those detectors in spatial registration with one another. That may make it easier to understand or decode what's occurring in the visual world given the signals coming into the brain from the retina. It also could be useful to have the mosaics anti-aligned, so shifted with respect to one another that could potentially give you higher spatial resolution information. So if you imagine you've got two receptive fields and they're overlapping, there's a small region in the overlap that if you could pay attention to the signals there, could give you higher resolution than you could achieve by each individual mosaic. Finally, statistical independence may be advantageous simply because it balances those two possibilities, balances the advantages of alignment with the advantages of anti-alignment or that it maybe is just straightforward to achieve developmentally. So we're not the first people to have thought about this question. It's been tackled from an anatomical standpoint by a number of investigators at a number of different times. So actually Heinz-Vessela had an insight to think about this all the way back some of the earliest papers on mosaic organization and used some statistical methods to try to understand if multiple mosaics were independent or coordinated in some way. And the conclusion from that was that the mosaics were independent. Subsequent years later, work from Dick Maslin's lab looked at a variety of different cell types in the rabbit retina. Again, all of the level of anatomy and looking at soma locations and concluded that in general, the anatomical mosaics were statistically independent of one another. And there was one study by Dave Marsheck's group in the late 90s that looked at the physical locations of scones and the bipolar cells that are connected to scones and concluded that there was a tendency for these two mosaics to be aligned. But I would say in general when this has been looked at the conclusion has been that mosaics are independent of one another. The main caveat of all of this, and I forgive, I hope the anatomists will forgive me for this statement, but at the end of the day, I think it's the receptive fields that matter and not the soma locations. And so, and the reason for that is it's really the receptive fields that are denoting or connoting what physical locations in space these cells are transmitting information to the brain. And the soma locations are gonna be related to that, but they're not going to be precisely that. So we decided to try to look at this question from the perspective of function, from the perspective of the receptive fields. And again, we got multiple mosaics here. These are receptive field mosaics. We have one that's an on type and one that's an off type. Are these mosaics, do they have any tendency at the level of their function to be aligned, anti-aligned or independent? So I just wanna jump in here and give credit to the people who have been really done the work and have been my close collaborators in doing this. So Suva Roy, a postdoctoral fellow in the lab, that really did all of the experimental work that I'll talk about. Naye Young-Jun, a PhD student between myself and John Pearson did all of the theory that I'll be talking about over the next 45 minutes. And John Pearson, my collaborator here at Duke has been absolutely essential to all of the work that I'm gonna tell you about today. And it's really been a phenomenal partner in collaboration on all of these projects. So we're gonna start with trying to answer the question, a slightly different question than one I posed a minute ago. Not are the mosaics coordinated but perhaps how should mosaics be coordinated? And we're gonna start by thinking a little bit about efficient coding theory which has origins in the writings of Horace Barlow and Atneve and is developed into a more rigorous mathematical formulation by a variety of groups, Van Haderen, Attic and Redlick, Aeroson and Shelley in particular. And these mathematical formulations predict quite a lot about how early sensory systems operate. And in particular, they predict a lot about how the retina is organized and operates. For example, efficient coding theory predicts centers around receptive fields which are one of the first functional principles or characteristics that was observed among neurons in the retina. It also predicts the presence of on and off pathways. So cells that detect increments of various features in the visual world and cells that detect the decrements of their various visual features. And finally, it predicts much about light adaptation and what happens is you switch from broad vision to cone vision. So the approach we're gonna take is to optimize a model of retinal processing according to efficient coding theory and determine what if anything that model predicts about how mosaics ought to be coordinated. And this is the conceptual model that we're going to optimize. So that kind of the simplest model you could see of a retinal processing. Just wanna point you to the fact that this model was originally developed by John Carplin and Erikson and Shelley and we're basically following their lead on this model. So we take a library of natural images and pump it through a set of filters and a set of non-linearities that translate the output of these filters into spike rates. And what we're going to do is optimize this bank of linear filters and the associated non-linearities. So each filter has a non-linearity associated with it. We're gonna optimize those so that we maximize the amount of mutual information contained in these spike rates about our library of natural images. And we're gonna do that optimization with a couple of constraints. So we have a couple of sources of noise. We have noise at the level of the inputs and we have some noise at the level of the outputs. And we have what's effectively a metabolic constraint. So the cells can't spike at super high spike rates. They can basically, on average, just generate one spike per image. So we optimize this model according to these constraints so that just again, we maximize the mutual information between these spike rates and our library of natural images. And I wanna build some intuition for what happens as we do that. So here are two of these linear filters, W1 and W2 and below them the non-linearities associated. So it would be like this one and that one. And this is how they're initialized before we've done any of the training. And what I'm gonna show you is a movie of what happens to these filters as this model is optimized to encode as much information as possible about the natural scenes. Playing that movie now, you can see that what started out is just a patch of noisy pixels starts to have some organization. And in particular, you get a very nice cluster of pixels here that's encoding a region of space that's darker than average surrounded by something that's lighter than average, whoops. And over on the right hand side, you see a cell forming that has the opposite polarity. It likes increments of intensity in the middle and decrements in this round. So these are just two of the cells that we optimize in this procedure, but when Na Young built this model, she put a hundred or so cells in here. And you see that every single one of them looks like either an off cell or an on cell with some associated non-linearity. And what I'm gonna show you in the next slide is how all of the on cells, how all of these units developed together or were optimized together in this model and how all of the off units were optimized. So the on units are over here on the left and the off units are over here on the right. And you see that they form these quite nice mosaics. Each one of these rings corresponds to some contour on one of these receptive fields. So I'll just play this one more time because I never get tired of looking at this. It's fun to see how these cells kind of try to squeeze in and sometimes they succeed and sometimes they fail. But at the end of the day, you get two very nice mosaics. So are these mosaics aligned, anti-aligned or independent? And to answer that question, both for these data or these simulations and for our real data, we needed to utilize some maybe non-standard analysis approaches. So I wanna explain that analysis approach to you now. Okay, so imagine we've got a grid of two sets of points, the green points and the purple points. And we wanna know if there's any spatial relationship across these sets of points. The method that we use to try to do this is to postulate that there's some repulsive force between the purple points and the green points. And they don't wanna be on top of one another. And if you assume some repulsive force, you can calculate a potential energy for the system. So when a green point and a red point or a green point, a purple point are sitting right on top of one another, you think back to electromagnetism, that state of the system will have a very high potential energy. But if they're far apart, the potential energy will be relatively low. So we can calculate the total potential energy across all of these points by say taking this purple point and calculating the energy it has or the force exerted on it by all of the green points around. And we just do that according to this equation so that the energy falls off as one over the distance squared between the purple points and the green points. At the end of calculating that potential energy, you have one number. It's the potential energy of this system. And then we can take one mosaic and just shift it with respect to the other mosaic and recalculate that potential energy. And we do that over and over again for many different shifts between the two mosaics. So I'm sure this sounds like a bit of an obscure analysis to do. So let me try to generate some intuition for how the analysis works. So imagine we have two mosaics that are aligned. There's a red mosaic here and a blue mosaic. And they're almost perfectly aligned with one another, like those two honeycombs that we've stacked on top of one another. We can calculate the potential energy of the system for zero shift and that potential energy will be quite high because nearly every blue point is sitting on top or very near to a red point. And so because those points want to get away from each other in this analysis, the potential energy is high. And then so that's this middle point here, high potential energy. As we shift one mosaic with respect to the other, that potential energy tends to fall off. And so this is what that energy map looks like as a function of shifting one mosaic to the left or right or up or down. And of course, if you shift the blue mosaic all the way over so that you get one period shift, the potential energy will come back up, which is why you've got these other flanking peaks in the potential energy. If the two mosaics are anti-aligned, you get the inverse relationship where the potential energy is low at zero shift and tends to rise as you go in other directions. And if the two mosaics are statistically independent so that sometimes points are near each other and sometimes points are far away from one another, then there's no conserved topographical relationship in this energy map. So we take these energy maps and we simply compute their radial average, which looks like this. So for a line of the radial average energy map tends to fall with shift or anti-aligned it tends to rise and for independent tends to be approximately flat. So we apply this analysis method to our optimized mosaics that we generated by our efficient coding model. And we see that the potential energy between the two tends to rise. You can actually see that they're anti-aligned. So if you take this cell and this cell, you can see that this cell lies right in the middle of them. And for example, this cell lies right here in the middle of these. So efficient coding predicts that the optimized mosaics in this model opt to be anti-aligned. And that is a prediction that we can now try to test with real retinal mosaics, are they anti-aligned? So to do this, we use a large-scale multi-electrode array to record from segments of isolated retina. Most of the data I'll show you today is rat data. We place a small segment of retina down, ganglion cell side down on the electrode array. And we present visual stimuli that are focused onto the photoreceptors. We then measure the ensuing spiking activity of those ganglion cells in response to the visual stimuli. And process the data to understand what the receptive fields are of these neurons that we're recording over the electrode array. I just want to note this electrode array was developed and implemented in the mammalian retina by Alan Littke and Yidzicicillinuske in a long-term collaboration that they had and they continue to have when Yidzic started at the salt can is now at Stanford. So here's how we measure the receptive fields of the neurons over the array. We present a checkerboard noise movie. So each pixel of this movie is being updated independently in space and time of every other pixel. And we simply accumulate the spikes for many given cell over the electrode array. For every spike, we take a brief clip of this movie that preceded the spike. We simply average those movie clips together. So if we've got 1,000 spikes from a given neuron, we'll have 1,000 movie clips. We average those clips and you get something that looks like this. So this particular neuron liked a brightening in that region of space. We call that the spatial receptive field. And we can summarize that with this little two-dimensional ellipse fit to this spatial profile. We can also look at how the stimulus evolved in time prior to the spikes. So this is the time of the spike here. And the stimulus got a bit dimmer and then rapidly brighter immediately before the time of the spike. So let me just show you that in the movie, bit dimmer, rapidly brighter, and then that's the time of the spike. All right. In many of the plots I'll show you, we've switched over to a color map here where blue indicates the cell likes increments of light and red indicates the cell likes increments of light at these different spatial locations. So this is the output of a typical experiment. This is the outline of the electrode array. So about two by one millimeters in the rat retina that corresponds to about 15 to by 30 degrees of visual angle. And we'll typically have something like 350 to 500 ganglion cells over the electrode array. In this particular experiment, we had 431. And we functionally classify these neurons. I should just say that each one of these ellipses represents the receptive field of one of the ganglion cells recorded on the array. And we functionally classify those according to their light response properties. And I'm not gonna go into the details of that because we've published that procedure in several recent papers. So you can go look it up there if you wanna understand how we actually do that. But this is the result of it. We'll end up with a bunch of mosaics, a bunch of different cell types. And I'm showing the four cell types here that where we typically have fairly complete mosaics across these four types. And we call these types brisk sustained. We've got one on and one off type and brisk transient again, one on and one off type. And I've switched from showing the receptive fields as just a simple Gaussian function or an ellipse in this slide, right? All of the receptive fields are ellipses. In this slide, they have these somewhat funny organic looking shapes. That's because the cells are often not particularly Gaussian in their shape. They frequently have is more lack of a better term organic shapes to them. And I think a nicer view of the mosaics is to see these contours rather than just a Gaussian fit. So in much of what you'll be seeing today and the rest of the talk, we'll be representing the mosaics by this contour and showing you the center of mass, which is what we'll use in this energy analysis that I described a few minutes ago. Okay, so just to convince you that these are four different cell types. These are the temporal receptive fields for these on brisk sustained and on brisk transient types. You can see that the variability within a cell type is small compared to the variability across cell types. This is the same information for the two off types. This is the spike train auto correlation functions for the two on and two off types shown here and here. You can see that they're quite distinct. And these are the contrast response functions for the two on and two off types. You can see that they're quite distinct. So despite the fact that these receptive fields are all about the same size, they're functional properties in terms of temporal integration, contrast response functions, spiking dynamics are very distinct and give us a high degree of competence that these are truly four distinct cell types. Okay, so I've explained to you the nature of our data collection and how we record from many ganglion cells off the electrode array, how we measure their receptive fields. Now we're in a position to actually analyze whether these two populations of receptive fields have any kind of spatial coordination. So using the energy analysis that I described a minute ago where we postulate this repulsive force and then calculate the potential energy and then shift the mosaics with respect to one another and recalculate the potential energy for each shift. This is what the 2D energy map looks like with this particular pair of mosaics. And you can see a bit of an energy well here, low energy near the middle. And if we calculate the average radial energy for this, we see a function that's rising, suggesting that these two mosaics have some tendency to be anti-aligned. If we do this across five different retinas, different on and off pairs, we consistently see this trend where the radial energy tends to rise with shifting one mosaic respect to the other. Again, indicating that these mosaics have a tendency to be anti-aligned before. This analysis is performed on the on and off brisk transient cells in the rat retina. These are the on and off brisk sustained cells. So these cells like slower changes in the visual input. Here's the 2D energy map for this particular pair. You see a bit of an energy well in the middle. If we compute the radial energy function based on this energy map, again, you see it's rising. Again, indicating that there is a tendency for anti-alignment between these two mosaics. If we look at multiple retinas, we see that this tendency is fairly consistent. E.J. Chichulniski was also kind enough to provide us some data from his lab. So we looked at primate parasol cells. These are on and off parasol cells from the lucky rat nut. You see a fairly nice energy well here in the middle for this particular mosaic. If we look at this mosaic plus these other two, we consistently see these rising energy functions. So complimentary pairs of on and off ganglion cells appear anti-aligned, suggesting that these mosaics are coordinated with one another. Now, I think because this analysis is somewhat unusual, I think there's a bunch of controls that are important to do here. And in the example that I took you through at the beginning to generate some intuition for how this analysis worked, I showed you two hexagonal mosaics where you had the same number of cells in each mosaic and you're not missing any cells. It's kind of the perfect case. Then the analysis clearly works in this perfect case. But of course our real mosaics don't look like this, right? There's several deviations. So first off, they're not particularly hexagonal, the real mosaics. Second, they don't have equal number of cells between the two mosaics. And third, we're sometimes missing cells. These gaps in the mosaics I think are probably produced by a failure of our spike sorting algorithm to find the spikes for cells that would fill those gaps in the real retina. Okay, so let's test these. We're gonna start with analyzing the effect of the mosaics being not hexagonal. So we generated mosaics from a Poisson point process with the green points are repelling each other, but there's no interaction with the purple points. And the purple points are repelling each other, but they're not interacting with the green points. So we've got two sets of points here that form a mosaic. It's not hexagonal, but there's an exclusion zone, but they're statistically independent to one another. We can then do the same thing, but have a repulsive term between green and purple points or an attractive term between green and purple points to produce aligned anti-aligned or independent mosaics. And when we do this, we do our radial energy analysis, we see the same results as we did for the hexagonal mosaics. So it doesn't matter that our mosaics are not perfectly hexagonal. Does the analysis depend on having the same number of cells in each mosaic? Well, the short answer is no, we can make 20% of the cells green and 80% purple, and the analysis still works with falling energy functions for aligned, rising for anti-aligned and approximately flat for independent. What about missing cells? Well, this is slightly more complicated. We could take our measured mosaics and either deplete them of cells or try to fill them in where we think maybe cells are missing, shown by the darker circles here. And when we do this, there's certainly a tendency as we deplete these mosaics more and more and we get noisier results down here. But we can deplete them somewhat and the results remain fairly consistent. We can also fill them in and the results remain fairly consistent. These are for the on and off brisk transient cells in the rat retina. This is for the brisk sustained cells and this is for the parasol cells in the lucky ratna. So again, I think the 10%, sometimes maybe 80%, we can depletion, we can still see the anti-alignment. When we fill in the mosaics, we still see the anti-alignment. But certainly if we get down below 80% down to 60, 40% completeness, the effect starts to go away probably because we don't have enough cells to support the analysis. Okay, so my conclusion from those things is that the energy analysis is reasonably robust. But there's still a question about is the observed anti-alignment statistically significant? While the analysis is robust, we're not quite sure if we should, are these rising steeply enough and fast enough to be convincing that they are not produced by actually statistically independent mosaics? So to test that, we did the following analysis. So imagine we've got mosaics for two retinas, pair of on, a pair of on and off mosaics and another pair of on and off mosaics. So we can analyze the energy functions for these real pairs because they came from the same retinas and we see some tendency for anti-alignment as I've shown you before. But we can also create a bunch of pseudo pairs where we take, let's say, the two on mosaics from different retinas or an on mosaic from one retina and an off mosaic from another retina and create these pseudo pairs and analyze them. Now, because these mosaics are from different retinas, by definition, basically have to be statistically independent. There's no way they can know anything about each other. And so this can build, from this, we can build a null distribution for statistical independence. And we do that by computing an index that's based on the area underneath these curves which I'm showing you here. So this coordination index will have positive values if the area starts out negative and then goes positive. It'll have negative values if the area starts positive and then goes negative, okay? So we can build a null distribution of these coordination index values from our pseudo pairs, like something like this. You can see that for this collection, there is a bit of a bias toward the anti-aligned energy maps, but it's small. The peak is not quite at zero. It's a little bit to the right. The real data lies well outside of this distribution. So this indicates to us that the anti-alignment across these mosaics is statistically significant because it's outside of this null distribution produced by the pseudo pairs. This is for the parasol ganglion cells that I showed you. And these two other plots are the same analysis for the on and off versus sustained and the on and off versus transient cells. Okay, so we've got these four mosaics, two pairs of on and off, and they seem to be anti-aligned with one another. But this mosaic and this mosaic, two off mosaics of different types, were also from the same retina. And so there could be coordination across these two off mosaics or coordination across these two on mosaics. So to test this, we again ran this energy analysis on these mosaics. And this is what the energy map tends to look like. This is from multiple retinas, what the radial energy function from these energy maps looks like. And this is where our data lie with respect to the null distribution generated from all of these comparisons. So this does not appear to be, there does not appear to be any statistically significant coordination between two on mosaics for first transient versus sustained cells. We can then do the same analysis for these pair of off. Again, the point lies well within the null distribution suggesting that these are statistically independent of one another. We can of course also compare mosaics that have opposite polarity responses and encode different visual features. And those also appear to be statistically independent of one another shown here. Okay, so just to summarize all of that, pairs of mosaics that encode similar visual features but with opposite polarity appear anti-aligned. Pairs of mosaics that have the same polarity responses but encode different visual features appear to be independent. And mosaics that encode different visual features and respond with different polarities also appear to be independent. So this is for mosaics, but recall earlier, I said the retina contains something like 100 mosaics. And so I think this is just the start of understanding how coordination across multiple mosaics could play out in the retina. And I like the term meta mosaics for this analysis, for this field of research because it's the rules by which mosaics are organized with respect to one another. So just to explore that idea a little bit, here are the receptive fields from the spike-triggered averaging method for 12 different ganglion cell types in the rat retina. The four we've been analyzing are these four here, but there are many other ganglion cell types that do interesting kinds of computations. For example, direction selectivity cells that like motion to the right or to the left. Looming detectors cells that like expansion within the receptive field. Orientation selective cells either in the horizontal or vertical orientation preference. All of these cells form mosaics and there's the possibility at least for those mosaics to be coordinated. So we're really interested down the road to try to unpack that. All right, so in the little bit of time I have left, I wanna answer the one remaining question that I've kind of left hanging so far, which is why is anti-alignment efficient? So far I've basically said efficient coding theory predicts that there is anti-alignment between on and off retinal ganglion cell mosaics. We showed that in the very first part of the talk when we optimized that model of the retina according to efficient coding theory and the mosaics were anti-aligned. The real mosaics that we measure also appear to be anti-aligned, but why is anti-alignment efficient? So Na Young in my lab has been tackling this problem with lots of help from John Pearson who's been actively involved in every step of that. In fact, we have a paper on bio archive now with the three of us. So if you're interested in more details about this please look at that. Na Young's also put together a nice little demo that goes through what I'm about to talk about and you can find that demo at this website. So I'm gonna go through this quickly with the time I have left and I encourage you to go check out these resources if what I say doesn't make any sense. Okay, so we've got this model where we've optimized the output of this simulated retina to communicate as much information about the library of natural scenes as possible. And I showed you that this produced center surround receptor fields that are either offset center or on center and that the mosaics are anti-aligned with one another. And so these are the centers of all of the off cells. These are the centers of all of the on cells. Clearly the two mosaics are offset. Na Young did the interesting manipulation though which was to take these noise values and turn them way down so that there's much less noise in the system. And whoops, this is what happens. The mosaics go from anti-aligned to aligned. And so we started to explore this in more detail and Na Young developed a very simplified model where there's just seven cells of each type and this allowed us to explore a wide range of noise values for both the output noise and the input noise. Note that output noise is increasing as you go down does access not up. You can see that as you increase the output noise there's a pretty abrupt transition between mosaics like these two highlighted over here that are aligned to mosaics like these two highlighted down here that are anti-aligned. And there's a strong dependence on output noise in this transition. There's also a somewhat weaker dependence on input noise. There's a shallow but statistically significant slope here. Okay, so that's one observation. Whether the mosaics are aligned or anti-aligned depends on noise but it doesn't only depend on noise. It also depends on the statistics of the images that you use to train the model. I'm gonna show you that in the next slide. Here are the distribution of all of the images that Na Young used in training the retina, this simulated retina. And some example images and where they lie in this distribution. So an image like this, which is relatively dark and doesn't have a whole lot of contrast structure in it lives down here has a low mean intensity and a fairly small standard deviation, right? Not much contrast in here. Where as an image like this, the has very dark pixels and very bright pixels. It lives way over here in the space but the dark pixels and the bright pixels on average cancel each other. And so the mean intensity of the image is not too far away from zero. Okay, so then we took just sets of images lying in different regions of the space. So we started with sets of images just within the one standard deviation contour of this density map. Out to two, out to three and we trained our simulated retina with those different sets of images and varied the noise in that training. And this is what we saw. So when we took images just from the middle portion of this distribution and we increased the noise, we had to increase the noise a lot to go from aligned to anti-aligned those acts. But as we included more of these outlier images, say out to three standard deviations, the transition between aligned and anti-aligned occurred at much lower noise values. We could even go out here and boost the occurrence of these outlier images and the transition occurred almost right away. So there seems to be some connection between noise in the retina and the statistics of the scenes that drive this transition between anti-aligned those acts and aligned those acts. And we were quite interested in this. I'm gonna just give you a very intuitive feel for what's happening here and then I'll end the talk. And again, I'll point you to those references a few slides ago if you want to dig into this more. The key that links all of this together are these non-linearities, which we haven't really talked about. We've mostly been focused on the linear filter portion, but these non-linearities are important because they determine what part of the image distribution the neuron actually responds to. So imagine this is the distribution of stimuli after filtering it by one of these filters. The non-linearity determines what portion of this distribution the cell is going to respond to. So stimuli that lie over here produce no spikes and stimuli that lie over here start to drive activity of the neuron. Now imagine we add a bunch of noise to the system. Well, now the stuff in here isn't very informative because it's just noise. And so that encourages you to take your non-linearity and push it out to the right so that you're not encoding or transmitting with the brain noise. Interestingly, if you have rare images, so this distribution becomes what we might call heavy-tailed, that has a very similar effect. It pulls the non-linearity over to the right so that the cell with more fidelity encode or transmit information about these images out here. So again, just to summarize, the noise pushes your non-linearity to the right, whereas rare images out here pulls your non-linearity out to the right. And the non-linearity's essentially serve to de-correlate nearby cells. This connects to a very nice paper from Marcus Meister and Zach Pitcau several years ago showing the important role that rectification in these non-linearity plays in making nearby neurons independent of one another. And I'll just skip this stuff and go here to say essentially when you have weak rectification, you have a very limited zone around any given cell, say a particular on cell, within which to place an off cell and have them be co-activated as infrequently as possible. However, when you have strong rectification, that zone of independence increases, allowing you to pack in more off cells around your on cell. And that is essentially the connection between the rectification that you have in this optimization and whether the mosaics are anti-aligned versus aligned. So I'm running out of time and I wanna go through a few conclusions and then allow some time for Q and A. So the main conclusion of the talk is that functional pairs of on and off mosaics are coordinated. And I think this raises a number of interesting questions such as how is this coordination achieved? Is it determined by the anatomy or synaptic connectivity? What developmental mechanisms produce this coordination? And how does this coordination impact downstream visual processing? I think these are all open questions that I hope our group and other groups will begin to answer over the next few years. The other conclusion is that mosaic coordination is predicted by efficient coding theory or at least some forms of mosaic coordination are. And we're very interested to know how far this mosaic coordination extends. Does it extend to direction selective cells, to bipolar cells, exhibit any kind of coordination, so on and so forth. What happens when we extend efficient coding theory into the temporal domain? Notice that the model that we optimized had no time in it, it was just static images. If we try to optimize that model on full natural movies, what happens? Finally, I'd like to just emphasize that mosaic coordination is really an emergent property of the nervous system in the sense that it cannot be predicted from the individual parts and it arises at the level of populations of diverse cell types. Okay, so with that, I just wanna thank the people involved. Again, Suvaroy did all of the experimental work and the analysis of those experimental data. Na Young did all of the theory work and this was all a very close collaboration with John Pearson. I wanna thank the list of collaborators that have had the pleasure of working with their blasts several years and the other folks in my lab. So with that, I'll take questions. Thank you very much Greg for this really interesting presentation, lots of fascinating work and there are already a number of questions appearing in the chat. I would like to remind to our audience that they can either post their questions there or hold them like for later when we have like the post-talk informal gathering at the Zoom room that I will post the link for soonies. And I would like to start with the questions. First one is from Simon Laughlin. How much more efficient are anti-aligned versus aligned and random coordinations? I think you barely touched on it, but it didn't go. Yeah, it's depending on how you look at it, it's either a tiny effect or a moderate effect. So I'll start with the pessimistic view. The pessimistic view, if we take, we look at the information transmitted a pair of mosaics in our model, okay, from the model. In the anti-aligned case, so where we have the noise and the image statistics so that anti-aligned it is favored. And we get a particular mutual information value from that for the mutual information between the spike rates and the library of natural images. That value is decreased by 4% or 5% when you go to aligned mosaics, keeping everything else the same. So you just take the two mosaics and you align them, you lose something like 4%, 5% of the information. That doesn't sound big, but I'll point out that if you eliminate the surrounds of the cells, which is an effect that we all think is super important for optimizing information transmission or whatnot, you only take about a 20% hit. So something that we think is super important, and I think it is important, but something that we think is super important for accurate high fidelity visual encoding, when you eliminate it, it's about a 20% effect. So I would say the optimistic view is that the mosaic coordination is about one quarter as important as Center Surround Receptive Field Organization. For economy of time, I will go directly to the second part of his question, which is how sensitive is the optimum mosaic to misalignment? How sensitive to misalignment? I'm not sure I understand that. How sensitive is it to anti-alignment or misalignment? I assume Simon will be joining us as soon as he is here. So I'll be joining us after and we'll unpack that. Next one is from Tom Badden. Can three, four, or five mosaics be meaningfully anti-aligned? Where is the limit? Like talking about higher statistics, let's say, higher order. Yeah, I love that question. It doesn't have a simple answer, but I'll try. If the mosaics are, let's say perfectly hexagonal, perfectly ordered, it obviously gets basically impossible to anti-align a bunch of perfect hexagons. But when you have a certain amount of disorder, you can have degrees of anti-alignment and you can have many things be weekly anti-correlated. You can't have many things be perfectly anti-correlated, but you can have many elements be weekly anti-correlated. So I think it's possible for there to be many mosaics that either have some tendency to be aligned or some tendency to be anti-aligned. Next one is from Hervik Bayer. Hi, Greg, you and Barlow seem to believe that the visual system has evolved to generate a veridical image of natural scenes. Is there a more objective way to determine the optimization function? A more objective way. That's a really interesting question. I'm not sure any theory is objective. I think every theory starts with an idea, a premise, and then you see where that premise takes you. And if the premise is successful at explaining the system, then you keep it. And if it's not, then you throw it away. But I'm not sure how to generate those premises in an objective manner. I really like that we go down epistemological reverse as well. But, Harig, if you have an idea, please tell me, because I would love to know how to do that. Next one is from Tim Gollis. Great talk. Thanks. How is anti-alignment affected by differences in receptive field size in the model and in real data? Yeah, so thanks for asking that. Tried to get at that. If you remember, there was a slide talking about differences in cell number and how the analysis works if you've got very different numbers of cells in your two mosaics. And I didn't say this, but one of the things I kind of have in mind with that is imagine you've got one mosaic with really large receptive fields. And so there's not many of those cells because they don't need to be, they don't need to densely sample space. And you've got another mosaic with really tiny receptive fields. And so there's lots of them that they're densely sampling space. So take melanopsin cells compared to midget ganglion cells, let's say. And it's perfectly coherent for those two mosaics to have some tendency toward alignment or anti-alignment, despite the fact that they're sampling at very different spatial frequencies. And that's just, you won't have every cell, every one of the low frequency, sorry, every one of the high-frequency sampling cells aligned with a low-frequency sampling cell, but you can get many of them aligned. Right, right. So Hervig clarified efficient with respect to what? And like this looks like it's getting into a discussion. So I would like to invite Hervig if he wants to join us in the Zoom room, we are currently seated in. There are another couple of questions if I'm not mistaken. So after that I will be terminating the broadcasting and I would like to invite you, like if you are interested to keep track of what we are discussing to join the room. Again, one from Tom, the contrast distribution and noise are strongly position dependent in natural things. Presumably that would drive different mosaic behaviors in same type pairs depending on where they sit in the eye. Is that right? I mean, Tom has thought much more deeply than I have about the spatial dependencies of scene statistics. And so I take him at his word for that. And yes, I certainly think it could drive that if you've got, well, like in fish potentially where you've got the upper visual field that's sort of looking at the scene filtered through the water and that's decreasing the contrast a lot. You might have a prediction that those mosaics, the mosaics looking up in that direction might favor alignment because there's less high contrast structure in those scenes or in those positions in space. Also, like now that we are here and I have a question of mine, if I may, did you try to play natural images to actual retinas and like record receptive fields with natural images? We have done some of that and not as part of this project. There's certainly things that change in the estimate about the receptive field when you do that. However, the center of mass of the receptive field which is what we're analyzing seems pretty robust to that manipulation. Okay, thank you. The thing like the surround can appear stronger and the temporal integration can be different, things like that. But the center of mass of the receptive field seems... Right, so the coordination we would expect again to be anti-aligned between like complementary pairs. Okay, next one is from Anguiera. How do you think that coordination between mosaics happens? What clues point to an active mechanism and not to coordination being a byproduct of physical spatial restrictions? So how do I think it happens is the question? Yeah, so why is it like a mechanism of coordination and not just the effect of having size and space constraints? Yeah, I don't... It's very unclear to me how the anatomal loop constraints which I'm sure exist actually constrain this problem. And the reason is right that you've got, let's take a set of on parasol cells and a set of off parasol cells. Their dendrites are in quite different layers of the interplexiform layer. And so I'm not sure they really interfere with each other at all. They could, but it's not clear to me that they do. Now the soma locations, they're basically a monolayer at least in the peripheral retina. And they cannot inhabit the same physical space. So that's gonna introduce me some constraint. But the fact that the dendrites then reach up and can grow in myriad ways seems to me that it makes that constraint likely to be quite weak. But if someone has a different intuition, I would welcome hearing it. I think I suspect the cell body location doesn't play a big role here. And my reason for that is because many people have analyzed cell body locations and they conclude that they're independent by and large. Now they've used somewhat different statistical methods than we have in many cases, but not all. And I think some of those statistical methods are not quite as strong as the one that we're using or not quite as powerful. So they may be missing something that's there. But I suspect the answer is right, at least most of the time that generally the soma locations are fairly independent of one another, save for the fact that they can't actually be in the same physical location. I think it probably has much more to do with synaptic connectivity. Right, and the last one, the last question that appears on the chat is more technical. And again, I would like to remind you that you can join us here if you wish. The last one is from Michael Pasek. Thank you for the great talk. How much does the analysis depend on the inverse square distance scaling of the force used to compute the potential energy? Good, so yeah, there's a lot to unpack there. It doesn't depend strongly on it, is the first thing I'll say. Particularly if you have exponents that are greater than one. If you have an exponent that's one or less in the bottom of that equation. So let me just back up for a second. So the higher that exponent, the more you are focusing in on local interactions, because the more distant interactions will have very tiny numbers because you're taking that big distance and you are taking it to a high exponent. So it'll become a very tiny number. As long as the analysis is reasonably local, meaning exponents of like 1.5, 2, 3, 4, the energy maps are very similar for different exponents. And don't depend very much on exactly how you draw the region of interest to kind of exclude boundary effects. If you have an exponent of one or less, the results start to get much more sensitive to missing cells and to exactly how you choose the ROI, the region of interest. So I would say the result weekly depends on that choice of the exponent. It's not completely independent of it, but it's robust to some variation. Thank you very much Greg both for your fascinating talk and for addressing all the questions that appeared in the chat. I would like also to thank the audience for joining us for another session of our series. And yeah, like I will be soon terminating the broadcast. So if you want to follow up, make sure to follow the link. Thank you Greg. Thanks everyone, appreciate it. And officially I waive my moderator rights. So Simon, if you want to continue with what you were referring to before, please by all means proceed. Okay, yeah. So what I was, I mean, what I was asking is really just a subsidiary question. And that is how sensitive is the anti-aligned mosaic to small misalignments. And given that the percentage increase is so small, the answer is probably that the optimization function is really rather flat. So it's not terribly sensitive. So anyway, I don't think this is necessarily the most interesting question to proceed with. So. No, but I understand now. Yeah, so you're just saying you've got the two mosaics and you just like they're anti-aligned and you just generally over just a little bit. Does the information that you did, the mutual information with the images decrease rapidly? And the answer is, no. Oh. Yeah. Because it doesn't increase a huge amount to begin with. Yeah. So does anybody have any ideas about her with Dyer's comment? Yes, he wants to talk about optimization in view of what the analyst is trying to achieve, right? So if you know, if the only thing you want to achieve is, I don't know, escape from a predator, then you need to optimize for that. That's the sort of thinking. So much more of a labelled line, view of what the right man's up to. Gotcha. Yeah. I mean, you have to optimize to different things, right? I think