 This is a story about how a development of a high-throughput data processing pipeline provided a critical insight into solving a long-standing question in systems neuroscience. Specifically, we automated connectomics reconstruction to reverse engineer visual motion detection. Now, visual motion detection is an ethnologically critical computation for both predators and prey for invertebrates and vertebrates alike. And the central issue in the field is that a single photoreceptor reports light intensity in only one point in the visual field and therefore cannot tell the difference between the variations in light level and the motion in one or the other direction. Yet, the signals coming from multiple photoreceptors are combined somehow to yield a response in a downstream neuron that exists only to one direction of motion left in this case. A seminal contribution to how this could happen has been proposed 60 years ago by Hassenstein and Reichert, who put forward this very elegant and now canonical circuit where signals from two photoreceptors pointing at adjacent but different locations in the visual field are combined nonlinearly after putting one of the signals through a time delay. To get an intuition of how this would happen, let's consider an object moving from the right to the left, then it will excite the right photoreceptor before the left one, but because of the time delay, the two signals arrive at the multiplication unit at the same time, get nonlinearly enhanced and signal leftward motion. And these are signal profiles in the left and the right arm and after multiplication. Of course, this doesn't happen when the motion is in the opposite so-called null direction because the signal from the left photoreceptor gets extinguished in the multiplication unit before the signal from the right photoreceptor arrives and as a result produces no output, leading to directional selectivity. The Hassenstein-Rieckert model, beautiful as it is, is not the only way to implement motion detectors. There have been many other suggestions over the years that combine the three crucial elements that are of this here, which is the spatial offset between the inputs to the two photoreceptors, the temporal delay and the nonlinearity. However, the biological implementations of those three crucial elements has not been established or assigned to particular neurons or synapses in the nervous system and therefore it is not clear which, if any of these models actually reflects what happens in the real brains. We decided to address this problem in the fruit fly visual system about which a lot is known and in particular what you will need to understand the rest of the talk, the light signal is reported by an array for photoreceptors that are collectively referred to as the retina in fly, that signal to downstream structures which are organized retinotopically reflecting the honeycomb array of the fly eye and they're called the Lyman and the Medalla and the Lobula plate. We also know in the fly that the Lyman and neurons are not directionally selective, yet there is a particular cell type called L1, neurons of which repeat in every column that is crucial for detecting on motion because if you silence it genetically, the fly becomes blind to motion. So this must be the input to the motion detection circuitry. We now know that a cell type called T4 in the Medalla comes in four subtypes that respond selectively to one of the four directions of motion up, down, forward and backward. Although that information was inferred by us and not known while we did our work but was reported in the same issue of nature by the BORS group who recorded those cells optophysiologically. Because we also knew from BORS work that shutting down those cells genetically would make the fly blind to motion, we knew that they provide the output of the motion detection circuitry. But what lies in between and the input and the output has not been known and presented a gap that we wanted to fill in. And the approach we chose is to reconstruct the circuitry in between the inputs and the outputs exhaustively using connectomics. So the technology, so the only technology that currently can reconstruct neuronal circuits in their entirety is electron microscopy because only it provides the sufficient resolution necessary to detect synapses and trace axons and dendrites. The method for doing serial electron microscopy has been developed more than 30 years ago and is illustrated in this animation. It starts by slicing an appropriately fixed and embedded brain tissue into thin sections like you would a prosciutto, except that each is 40 nanometers thick. Then imaging each slice under electron microscope and the contrast comes from having metal ions that stick to lipid membranes. Then one must segment the cross sections of individual neurons shown here with false color. And after aligning those sections on top of each other as they were in the uncut block and connecting the cross sections corresponding to the same neuron, you can reconstruct three-dimensional shapes, identify synapses, and if you can trace from the synapses to cell bodies, you know which neurons make physiologically active contacts and draw arrows on the wiring diagram or the connectome. Now this animation might have given an impression that connectomics is a simple and pleasant exercise. But nothing can be further from the truth, and although preparing the tissue and slicing it is technically challenging, the main bottleneck in terms of time is computational. It consists in tracing of all those processes from synapses that you also need to identify to cell bodies, or at least far enough to identify cells by their shape. And that's what has been holding back the application of this technology to neuroscience. So even the data set that I used to make this animation, which is very small, about five microns on the side, took several months to trace fully manually to illustrate why that presents a difficulty. Let me show the data set that we needed to reconstruct to answer the motion detection question, which is 100 times bigger, tracing which manually would be prohibitively expensive. So this is a 9x9 montage of just one out of 3,000 sections from the fly visual system. And of course this is shown at lower resolution, so you cannot see the synapses. But if you zoom in to the square, zoom in again and again, then you get to the native resolution of about 3 nanometers per pixel, where you can identify individual cross-sections of neurons, which are pretty much all of these closed contours, but also ultra-structural features associated with the presynaptic and post-synaptic terminals. There could be several post-synaptic terminals for one presynaptic in fly. Of course, just finding that synapse doesn't tell you which neurons connect there, so then you have to trace to the cell bodies, and you cannot do this in this little volume, which is only 1,000th of the original montage, and also you would have to do it in 3D, going through the stack of images that you should be seeing now, which have been aligned, but you can also see that there are a lot of defects here having to do with image acquisition, making this job hard. So what I did at Genilia, I initiated and ran a project to automate this reconstruction by going through the steps, showing through the animation, but trying to reduce the amount of manual labor as much as possible. Well, we did not succeed to completely eliminate manual labor, and we still have a manual stage of proofreading, but we were able to reduce the amount of work by an order of magnitude and with the arrival of the new microscope by two orders of magnitude now. And that allowed us to reconstruct the conectom, which was relevant for motion detection and flying. One other advantage of the fly that I should mention is that we were able not to just reconstruct a connectivity graph, which you can see here, but you will see magnified in the next slide, but we were also able to identify the cell types by corresponding the shapes of neurons reconstructed from AM to the previously named neuronal types which were imaged by light microscopy using Golgi or more modern genetic techniques. And this way we were able to identify more than 50 different cell types in the medalla, including six new ones. So by putting, let me now put all of those neurons together and show how they all fit. So this is just one column of the medalla, and these are the neurons that we reconstructed from that column appearing one by one, and then the neurons from six adjacent columns filling up almost fully the reconstructed volume of about 40 by 40 by 50 microns. So that's the data set that we assembled from the fly visual system. With this data set, which includes not just the shapes of neurons, of course, but also the identification of synapses, we were able to fill in the gap, and because the data set had almost 400 neurons and more than 8,000 synapses, it's the biggest connectome to date, not just in fly, but in any nervous system if you count by the number of identified synapses. But of course, just having the connectome doesn't answer the question I posed about visual motion detection. We now need to identify the neurons and the synapses that are involved in that computation within that huge connectome. We did that by condensing the information in the connectome in several stages. First, we took advantage of the previous experimental results that showed that fly detects motion by comparing signals in just two adjacent columns, whichever individual field that pair of columns is located. And so we looked for the circuitry of the motion detection that repeated from column to column and used our reconstruction of adjacent columns to detect it. So this is now the connectome of the cell types. It doesn't represent the location of neurons in the columns, but it tells you how types of neurons that repeat in every column are connected to each other with stronger connections as quantified by the number of synapses in parallel shown by thicker lines. Then, knowing the input and the output to the circuitry, we looked for the shortest paths and the strongest paths that could underlie this computation. And as a first pass, we identified two such shortest paths that go through neurons types called TM3 and MI1 that's reducing the complexity somewhat. Once we found that there were two such paths between inputs and the outputs, we, of course, immediately thought of the Hassenstein-Reichert motion detector and tentatively identified these two arms with MI1 and TM3 types converging onto T4 neurons. Of course, at this point, it was just a guess. To make a case that this is a viable motion detector, we have to determine that it provides the spatial offset, the temporal offset, and the non-linearity. You could not tell whether there was a spatial offset in the input required to break the symmetry from the previous diagram because it was shown in terms of type-to-type connections. So all the neurons of L1 type were shown as one node. Now we have to go back to the full connectome and trace connections between individual L1s, now paying attention to which columns they live on. So when we did that, we found several surprises. So unlike the Hassenstein-Reichert model, where only a single photoreceptor provided input to each arm, we found that multiple TM3s provided input to each T4 and multiple L1s provided input to each TM3, meaning that that arm collects visual information from a wider region of visual field that is spent by a single photoreceptor. More worrying than that was the observation that sometimes the same photoreceptors provided input both to the left, hypothetical left, and right arm of the motion detector. So to make the case that this circuit can be used to detect motion, we have to show that those receptive fields mediated by TM3 neurons and MI1 neurons, even though they overlap, they're actually spatially offset. That is, their centroids do not coincide. And to convince you that this is indeed the case, I'm going to show you the actual 3D reconstruction of the circuitry from this part of the visual system. To understand what I'm going to show you, imagine shrinking yourself in size and sitting on top of T4 and looking towards the world or the retina of the fly. And then first, let's forget for a second about TM3 and MI1, and what you're going to see are the axonal terminals of L1s shown here which are arranged in a way, topically replicating the honeycomb structure of the fly eye. So each column has six neighbors. Although the six-fold symmetry is not perfect, that is because the neuropil is a bit squished compared to the eye. But this does represent where the visual inputs come from into the lower levels of processing. Now, let's add the dendrites of TM3s and MI1s converging onto a single T4. So I'm only showing the TM3s and MI1s that converge on one T4 that is not shown here. As shown in the schematic, you can see that MI1 and especially TM3 provides inputs from a range of columns, not a single one. The two inputs overlap, indeed, but the centroids of the dendrites are in fact displaced. Suggesting that the output of this T4 is asymmetric to the two directions of motion. Of course, these are just the dendrites. The function is determined by synopsis, but this is a full connectomic reconstruction. So we have the records of all the synopsis. And here are the percentages of synopsis provided to a given T4 from each of the columns. Red, the ones that provide signals going through TM3, and blue are the ones that provide signals going through MI1. If you cannot fit a two-dimensional Gaussian to those numbers in your mind, I should tell you that actually the centroids shown here were computed as the center of masses of those numbers, and the displacement that you see in fact reflects the functional displacement between the two channels. So this shows that indeed there is a spatial displacement between the two arms of the inputs that is needed to produce a directionally selective output. But you may object to that and say, well, this is biology, it's noisy, so there could be just a displacement because of that. For example, if one reconstructed the receptive fields of an on-center, off-surround, non-directionally selective ganglion cell in the vertebrate red, then one might see the same thing. So to convince you that this is relevant to motion detection, what I ideally would like to do is to show that this displacement corresponds to the directional preference of that T4. But to establish a directional preference of any cell, you would need to use physiology, right? Either electrophysiology or optophysiology and record its response after presenting stimuli moving in different directions. Unfortunately, this fly has been dead, and so we missed the opportunity to make those recordings. But fortunately for us, we were able to identify the directional selectivity of individual T4s by tracing their axons downstream to the next processing station, which is called the lobular plate, which is known to be organized in four layers that process four different directions of motion. And so this particular T4, inputs to which are shown here, terminates in the green layer, which corresponds to the forward motion. And that's exactly the direction of the displacement that we found for the inputs once projected into retinotopic coordinates if you count them from Tm3 to Mi1. So this supports the case that I'm trying to make that this is the motion detection circuitry, but this is only a single cell. Well, we repeated this experiment for 19 T4s, and here are the results of the population analysis. This is the average retinotopic displacement for all the T4s that terminate in the green layer and correspond to the forward motion. As you can see, it points in the right direction compared to the arrow shown here, and so does do the red T4s and the purple, and the blue one don't have a perfect agreement. And we think now that the reason is that that channel is special. It corresponds to the backward motion, which is the expected optic flow for the fly-flying forward, and there is another neuron that particularly contributes to that channel. But with that caveat, I think we found that there is a spatial offset between the inputs to the Tm3 and Mi1 channels, which correlates with the directional preference of the individual T4s, which makes a strong case for that circuitry being the elementary motion detector in the fly-visual system. So we were able to not just reconstruct the connectome between the input and the output, but to identify the specific neurons engaged in this computation and the synopsis between them and show that the circuitry is consistent with the expectation for the elementary motion detector. Now, this of course just tells you about the spatial offset that it is indeed present there, and we identified what the biological correlates of that offset is, but you also need to have a temporal offset between the two arms to have a directionally selective response. Well, initially we thought that we would be able to determine that temporal delay by measuring the dimensions of axons and dendrites of the neurons involved in this processing and associating them with the electrotonic delays. But it turned out when we actually did all the measurements that the dimensions of the axons and dendrites were both too small and too similar in the two channels, that they could not be responsible for the delay required for this computation. So we now think that delays are implemented by synaptic transmission because it is known in the vertebral retina, for example, that the use in different neurotransmitter receptors, say ionotropic versus metabotropic, would introduce different delays in synaptic transmission up to about 30-40 milliseconds, which is just within the range of what is required for the fly motion detector. But that's just a hypothesis, of course. How do we know that those motion detectors, those temporal delays, are really there? Well, this is not the work that we have done, but this is the work that was enabled by our connectomic reconstruction. And this was done by performing electrophysiological recordings from MI-1s and TM-3s in the fly visual system and looking at the delays in the physiological response. Why couldn't that be done without the connectomics reconstruction? Well, the reason is that there are about 60 different cell types in that part of the visual system. And the neurons are so small and so difficult to record from that it takes an excellent postdoc one year to identify a response of one cell type and characterize it fully. So to record from all those 60 cell types, you need either 60 excellent postdocs or 60 years of one excellent postdoc. But after we reconstructed the connectome, we knew exactly which cell types are candidates because they're the ones that are part of the circuitry with the spatial offset. And so we told Rudy Benia in Claude Desplan's lab at NYU that they are the neurons involved and in a heroic series of experiments, they were able to record from those neurons in Drosophila and they indeed found a temporal offset between the responses of the two neurons. Here you have, here I'm plotting reverse correlation filters computed to the fulfilled white noise stimulus. That delay is small, it's about 15 milliseconds, but it has been shown in simulation that it is indeed sufficient to produce the necessary directionally selective response. However, this experiment posed a problem for the Hassenstein-Reichert motion detector because in the Hassenstein-Reichert circuit, the delay has to be in this arm, in the right arm, if the preferred direction of motion is to the left. Our connectomic reconstruction showed that the preferred direction is from TM3 to MI1 centroids. So we would expect delays in TM3 if we believe in Hassenstein-Reichert detector. But instead, the delay was in MI1. That is, it had an opposite sign from what Hassenstein-Reichert would expect if they had our connectomics knowledge. Because of that, we switched our working hypothesis from the Hassenstein-Reichert to another model which is closely based on the Barlow-Levick suggestion which was based on the experiments that were actually done in the vertebrate retina. They used the same components as the Hassenstein-Reichert such as the spatially displaced inputs to the two channels, the time delay and the nonlinearity. But because they wanted to put the time delay on the other arm, they had to have a different sign of the inputs to the nonlinearity. And we also used linear threshold nonlinearity which seems biologically plausible to neurons and it works just as well. But it is different from the end-not nonlinearity that Barlow-Levick have used. So this is what we think is the most likely candidate for a motion detector now. And to get an intuition of how this works, let's first consider a target moving in the null direction from the left to the right. Then it would excite the left photoreceptor before the right one. But because of the delay, the two signals get in at the same time but with different signs and cancel each other out, producing no response, which is appropriate for null direction. Now if the target moved to the left, it would excite the left photoreceptor before the right one and the signal will pass through the nonlinearity before the chance of being canceled by the other photoreceptor. And that's the origin of the directional selectivity in the Barlow-Levick model, which we think at this point is the most likely candidate for the elementary motion detection and flight. Of course, we would like to know the signs of the inputs which we don't know at the moment and to conclusively rule out other possibilities. But with the data that we have so far, we think that this is the answer to this long-standing problem in systems neuroscience. Now going beyond the motion detection problem and putting this in the context of neural computation in general, I think that these two circuit motifs that we worked with with dropping the delays, which is the Barlow-Levick and the Hassenstein record, which can also work with rectifying nonlinearity, which is more biologically plausible than multiplication, they correspond to two basic computations that one may want to do in terms of statistical signal processing of noisy data. For example, if you want to average out the noise from two incoming signals, you might want to sum those signals, or if you want to find higher order correlations, you put the inputs through the nonlinearity after summation, and this way you would increase the signal-to-noise ratio. The other computation you might want to do if your incoming signal has high signal-to-noise ratio, yet is redundant between the two inputs, then you might want to subtract one input from the other in a predictive coding sense to save on the dynamic range of the neurons. And so we think that this could be the two circuits out of which you can build more complicated circuit. As an example, we showed that using Hebbian and anti-Hebbian rules, one can build a circuit which is similar to the Foldack or Oldshausen and Field architecture that is thought to model processing in the primary visual cortex and produces Gabor-oriented receptive fields. You can view this more complex circuit as being built out of the elementary units or blocks that correspond to this two circuit motifs. So finally, I would like to thank all my collaborators in this work. This was done not just by my group at Genelia, but as a part of a large collaborative effort that is called the FLY-EM Team Project at Genelia, which I initiated and led there for about six years. We got a lot of help from Ian Meinertz-Hagen, who is a lab in Canada, Lou Scheffer, another group leader that I recruited to Genelia, Shinya Takamura, who is the lead scientist on the project. And I also want to thank all of the members of my group who over the years contributed to this project in different stages of this highly complex enterprise, and particularly Arjun Barayok, who did the computational analysis, the motion detection, and is looking for a postdoc now. Finally, as I mentioned in the beginning, I moved a month ago to the Simon Center for Data Analysis, which is a part of the Simons Foundation in New York City, and I'm a group leader for neuroscience there and am actively recruiting. Thank you very much. Jeff. Thank you very much for the nice talk. I would like to know what do you think is the main element that makes this rectification at the synaptic level or in the properties of the cells? So do you have any clue about that? So that's a good question. I think it depends largely on whether the cells are used graded potentials or action potentials. So the graded potential cells usually have their membrane voltage going in both directions and rectification often occurs on the output synapses. And that would be appropriate for MI1 and TM3 and L1 because they use graded potentials. About T4s, we are not so sure if they use graded potentials, then this would have to be synaptic probably, but they could also be spiking. No one has yet recorded from them in Drosophila, so we don't really know. If they were spiking, then the rectification would occur at the action potential generation presumably or close to the cell body. Thanks, Mijia. Do you have any idea yet what the degree of variability between animals is? I mean, obviously not necessarily in the building, but in the overall circuitry. Yeah, so that's an excellent question. So that original reconstruction which our work has been based was done in a single animal and mostly condensed, mostly focused on a single column and its six immediate neighbors. But since then, we've reconstructed, using this new microscope technology, seven columns from another fly, and we were able to compare the variability between those columns within one fly. So that variability is very, very low. You know, it's well within 10% by the count of synapses. But the comparison between flies, we can only do between those two data sets at the moment, which can't really work because the older technology did not allow us to trace all the synapses, but only about half of them. From a reconstruction of the previous process and station of the visual pathway, the lamina in two different flies, also in two different, but comparable imaging methods, we know that the reproducibility is also better than within 10% of each other. So we don't have a complete answer, but that's the ballpark. So it's a very, very stereotypical circuitry. Your model of the, well, the Reichart motion detector model is very similar to the coincidence detection model of light Jefferson 48 for auditory space localization. However, if you look at chickens and barn owls, they've implemented those that model in different ways. In barn owls, it's delay lines. In chickens, it's because of time constants at the synapse. Your model is very similar to the Barlow and Levick one for the rabbit retina. Do you believe that only a single strategy has been implemented for motion detection in the visual system? No, I don't think. I think that the algorithm may be shared, but the implementation, mechanistic implementation could be different. So, I mean, we still cannot exclude that, say, within one arm, within TM-3s, you don't have a Hassenstein Reichart detector. On the other hand, there is now evidence from the Sebastian Song lab of a similar kind of computation being performed in the vertebrate visual system where the delays are also carried by the inputs and are not implemented by the dendrite that is directionally selective. Yet, as you pointed out, the exact mechanism is different. It seems like a hybrid between Hassenstein Reichart and Barlow Levick. So, in this case, you've taken a computational primitive and tried to detect it in a fairly homogenous anatomical structure. But my question is, so what's your speculation for trying to infer the computational primitives from large-scale connectons? So, going in the reverse direction, you have the circuits in further computation. So, this is a much more difficult task. And especially given the fact that vertebrate circuits, for example, are often not stereotypical, you would have to somehow combine a connectomic reconstruction with the prior physiological recording, either optical or electric, from the same neurons. Then I think that if you have all that information in your hands, you can make some headway in determining the functionality of neurons. But, of course, I cannot know that for sure because this has not been done. Thank you.