 Και εγώ πιστεύω ότι είμαστε εξαιρετικές. Γεια σας, καλύτερα και ευχαριστώ για έναν άλλο σημερινό της Σεμίναρας της ΑΕΚΑΙΣΙΑΣ. Εκείνοποτε, σαν εξαιρετικό υποσχέσμα. Είμαι ο Γεωργίας Καφετζής και είμαι ως εξαιρετικό σημερινό από τοΜασόλαιο's ΕΚΑΙΡΕΣ. Και στις τώρα, είμαι τεχνή σχόλου με τον Τόμ Παδεν. Και σαν το σημερινό σας για σήμερα, θα σας πρέπει να ξεκινήσω... για να προσπαθούνται για αυτή την ευρωπαϊκή επαγγένωση και ανοιχημένη πρόσφυγη, προσπαθούνται στους Ευρωπαϊκούς και πολύ πιο αντιμετωπίες σημερινές. Μετά από αυτό, αλλαόμουν ευρωπαϊκή για να πάμε back to the reason we all gathered here for today and introduce our guest from the University of Seville, Prof. Miko Jusola. Φollowing his medical degree from the University of Oulu in Finland, Miko obtained his PhD in neurophysiology in 2005, working on neural mechanisms in invertebrate photoreceptors with Martin Wegstrom. He then went to Canada for his first years as a postdoc, first at Alberta and then at Dalhousie before returning to Europe and Cambridge in 1996, where he started his own lab a year later. Since 2005 he has been located at University of Seville, where nowadays he holds the title of Professor of Systems Neuroscience at the School of Biosciences, but concurrent to this main trajectory, he has been a visiting professor in the Beijing Normal University and a high-end foreign expert selected by the Chinese National Recruitment Program. Throughout the years and with research interests focusing on sensory processing in Drosophila, Miko and his team have been developing novel experimental and theoretical techniques to elucidate the underlying mechanisms. A recent work of theirs includes discovering how the fly compound eyes exploit image motion to see in hyperacute spatial details, thus bringing together concepts of visual acuity and stereo vision with micro-sacadic sampling. Their work advocates for a paradigm shift in terms of how we think of vision in experimental setups from static eyes with an immobile image to dynamic eyes, dynamic vision of super resolution, active sampling. And thus I'm very happy to be living the stage for him for a token titled Seeing the World through Moving Photoreceptors by Nocular Photomechanical Micro-Saccades Give the Fruit Fly Hyperacute 3D Vision. So without any further ado from my side, please all welcome Professor Yuusola. Miko, the stage is officially all yours. Okay, thank you, George. Very happy to give this talk. There's a lot of stuff, so I'm going to start now. Share the screen. Okay, and let's see if this kicks on. Right, completely wrong position. Let's go up. Yes. Okay, so I would like to first just discuss a little bit about the eye and the theoretical constraints that people have thought about it. Because the exoskeleton of the insects looks really stiff, it's been concluded that the eye would be static. So that means that the resolution of the eye is basically based on the spacing of the photoreceptors. And the spacing of the photoreceptors, which is the crane of the film, is defined by this property called intromaterial angle. So the compound eyes are made out of these kind of little facets and the distances between the facets is this intromaterial angle. I hope in the end of this talk, make it clear that this idea is very inaccurate and the insects actually seem massively better than what this static eye prediction gives. There's an image of a man with a big compound eye and this is Kuno Kirschfeld's classic illustration where he shows that what would be the size of the human eye if we had compound eyes to have the same acuity as the human small lens eyes now have. There's many things which are wrong in this picture, but I'm going to spend time on that instead. I'm going to go to another constraint, which is the classic static theory also considers that the photoreceptors are the collect light. The integration time of the photoreceptors is often estimated by all sorts of impulse functions and impulse responses and using a classic engineering means. If that's so, that if the integration time is low, then when the eye or the animal moves fast then these images should blur. The idea has been that when we collect information from the world we would do it mostly during when the saccades are still at that fixation phase and during the saccades because it's so blurry we would basically just kill information. That's also highly inaccurate. We looked into this number of papers, these two are probably the most important. The idea is that if the nervous system is in a closed loop with the world and if the eye is reading the topically or the brain is reading the topically organized then you can actually collect images massively fast by doing active sampling and then build up by using memory and feedback functions the images, the neural images that are massively finer than the resolution of the static resolution of the eye and that the integration time is not really a big problem. So this changes the way how we think the insect visual system works and it all starts from the sampling so I will focus quite a lot of sampling on this one, this talk and also processing. I could talk about these other issues but that would be another talk. So how good is the insect vision actually? So there's one way to study is to do intracellular recordings and this is what we have a more or less perfected this technique so you can make a tiny little hole on the corner of the eye and then you can take a conventional class capillary microelectrode and you can penetrate through that whole individual photoreceptors and you can record intracellularly these responses to controlled light stimulus and here is an example where I'm presenting a light pattern with two seconds long kind of a thirsty pattern and I'm repeating it at the end of times and then these are the individual responses and it goes continuously and this is the light stimulus. Now when you have a collection of these kind of responses to the same stimulus you can then estimate how good is the responses or how well it can replicate that pattern of light However each time when you do these experiments you have to consider that you actually damage the eye and the cell without the electrodes of course would perform better than with the electrodes so everything what you get from these kind of estimates they're always underestimates of the true capacity of the nervous system models photoreceptors actually to collect information. So here because these techniques are really perfected then I can keep the same cell for hours and the recording conditions are very stable and the recording noise is as low as you can pretty much go but of course it's still not as good as in a real situation would be without the electrodes So I have here a number of stimuli that I was presenting to the same fly over and over again so if you go down here these are all called calcium white noise so there's some frequency bandwidth so here low frequency is up to 20 Hz and on here 500 Hz so the stimulus is getting basically faster the modulation and in these up ways we increase the contrast by starting to clip that light background so we actually have less photons but we end up in the situation that there are this kind of a bursty positive light contrasts on a dark background and if you look at the responses you instantly notice that this bursty stimuli they give massively bigger amplitude modulation than the calcium white noise and also the variability of the responses are massively larger these are not just individual responses there's actually 20 responses superimposed but there's so little noise in actually the nervous system I think there's a massive bias in understanding how well the evolution has actually tuned this system to operate so you don't really see much deviations here so when you calculate the signal to the noise for this kind of bursty stimuli which is done next so this is the one which is having the highest variability for 100 Hz kind of bursty stimulus and this is a classic white noise type of responses what you would use for estimating information capacity you can see that this white noise is not really stimulating or driving the cell very well but you can look at this in the frequency domain so these are the same responses this is the 20 Hz bursty stimulus this is 100 Hz and this is 500 Hz bursty stimulus and I show in yellow here the 100 Hz because it has the broadest frequency bandwidth so it is kind of whitening that energy but you see the signal to the noise of these responses are massive so they can in the best cells to go to like 10,000 so there is hardly any noise you know so there is mostly signal what you see in the real recordings if you do it really well if you try to eliminate other noise sources and the other interesting thing what you see is if you look at the probability distribution so the signals are extremely skewed so there is a bursty light stimulus from a background so this is far from calcium but when the cell is doing so called refractory sampling it actually allows to generate at the photoreceptor level calcium responses to these very skewed inputs so this is the input count you see this is in log scale so it is massively skewed but the photoreceptor is producing something which is calcium and if you increase the modulation as we do here then you see that it is the largest for the bursty stimulus for the 100 Hz bursty pattern so that means that there is a broadening of calcium and there is a whitening of the spectra and if you have the calcium broad spectra and the white signal of the noise ratio the frequency range then you go towards the information capacity so if you calculate then the information capacity what you find is that this bursty stimulus actually they are giving you in some cases like 850 bits per second which is very high because the previous estimates in the lidid survey around 200-300 bits per second so if you give bursty patterns which are kind of a typical of what you get in the SACCAT sampling the information rates they just shoot up from the classic calcium white noise type of stimulus so the calcium white noise is not really testing for the systems very well because we done lots of modelling and we can simulate this system also very well and this is an example how we have done it so the photoreceptor you can do this kind of function models we know that the light sensitive part of the photoreceptor it's called the raptomere is something that has 30,000 microvilli and each microvilli is a sampling unit or a transduction unit this is a microvilli here it has a cascade of reactions and Roger Hardy from Cambridge has really characterised this well and then this work was done together with Roger so we can now simulate models of this sampling so we can first model an individual microvilli and then we can model using computer clusters about 30,000 of them and we can model the photoreceptor soma and it's membrane properties by using Hotskin-Huxley equations and so we can then see because we've done the interstellar recordings and we have estimated quantum pump dynamics from those if we know in different conditions and I've worked on this a lot in the past if we know the laser distribution of these quantum pumps and if we know the amplitude distribution if we know the refractoriness distribution and we know the number of microvilli we need only those four parameters and we can perfectly predict by using statistics what are the responses and that's what we've done here so these models they don't have any free parameters everything's fixed but you know it samples like a real cell because it has the structural identity of a real cell to sample light information and so when we repeat the same pattern you see the first stimulus are giving the larger very similar looking responses as what you saw in the interstellar recordings and when you have the Gaussian white noise you see these little ripples here now we do the same in the frequency domain this where the biggest most variable responses for the 100 Hz cutoff you find that there's the same whitening properties so you get the whitest frequency the signal-to-noise ratio for the 100 Hz bursts and you see that the distribution gets Gaussian and the broadest but the information of the transfer rates are in about 650 you do get interstellar recordings that give similar type of information transfer rates as the model but you do also have those higher ones so you may ask why it's lacking about 200 bits per second and okay and this is massively higher than the classic so the Gaussian white noise they are pretty much matching with the real recordings so what is this difference about 200 bits per second then it actually comes from because each point in space is more or less sampled by 6 to 7 photoreceptors and that information is pooled in the first synoptic layer in the lamina in these cells called the large monopolar cells and some of these large monopolar cells have feedback connections back to the photoreceptors so monopolar cell gets information from 6 photoreceptors plus true cap functions from 8 and 7 and 8 as well so it has the higher information content than the photoreceptor and it's feeding some of that information back and we can actually model this as well so we get that difference coming from the neighboring cells which are feeding back to the information of the photoreceptors and if you then look at that the individual photoreceptor response which is shown here in purple so this is a real recording and then you have to create what the photoreceptor model is actually producing if you put a feedback loop and you reduce that difference the difference actually turns out to be a monopolar cell response which is here which is a quite nice way to show that the monopolar cells are directly feeding into and this is a system that the information is going backwards and forwards to be optimized for the vision okay so there's a lot of detail but I'm enthusiastic about it and read it in the papers I'm not going to go into details but what I just showed to you that the experiments and theory they pretty much define how microvillage populations in fly fold receptors encode information and we've done these models also for Musca and Califora and KiloFly and the same principles as long as you know these quantum pump dynamics and you know how many microvillage they predict perfectly pretty much what the cells are collecting so the information is captured mostly through the high contrast burst and it's achieved by this refractory sampling and the connectivity in these high numbers so this is kind of an intro to go a little bit now into a different domain which is to see if you link the behavior into this so this is a fly which has been walking underneath some sort of a plexiglass so it couldn't fly off and it's these are experiments from by Curtin from Keppford lab in Köttingen and so they were looking at the fly behavior from above and then they were identifying those fast movements where the animal was turning rapidly and these are called saccadic head movements or body movements and if they are over 200 degrees per second they are called saccades so these spikelets which go up and down they are the type of saccades that the fly is collecting when it's just walking around what we did next is we first of course digitized these traces and then we took some images from Google, this was a spanoramic from nature and we collected light intensity patterns from these kind of scenes by using the way how fly is moving in the laboratory and this is an example so we had three types of ways of collecting one is this following directly what the animal has done so you have there's a crosshair which is collecting the intensity differences by using the movement patterns of the fly from that natural scene but you see that it produces very bursty light intensity time series and if you're too linear walking through the same intensity values but it produces slightly different statistical distribution and if you then shuffle these values what we have it produces again a different patterning but what I did next was I took these patterns and I was just feeding them to the photoreceptor by using the same recording technique so interest of the light intensity time series pattern and then I calculated the rate of information for these different patterns from the voltage responses interest of the voltage responses that I was collecting and it was clear that if I was presenting those light intensity patterns that were saccadic and these are from the same fly one after another and we found out that the saccadic way of collecting information allowed the fly to capture most information from the world and if we then just used the same model and we used the same patterns the model replicated this result so you get the highest information rate from the model if it was just getting the light intensity time series as collected by the walking fly ok so we showed that the saccades and the case fixations in natural environment they actually result in high contrast bursts and this implies that the eye movements work with refractory sampling to improve vision so all these subsystems from the photoreceptors and the biochemical reactions they were tuned by evolution to their behaviors so they seem to kind of a match so that they get a sufficient information content so that they are successful in their life ok so now that was about looking information in time but let's look at it in space so how fine is the special detail that a fly eye can result so to study that we have built many many different type of instrument this was one of the first ones so I'm just showing this example so there's a 25 LED endings on these light guides which design a closed loop LED controlled so there's 25 LEDs driving light from these light guides which are put in an array which can be then moved around the fly so when I'm recording intracellularity I can then just flip these lights and I can just get the receptive field on that particular sector of the visual field in one go but the same system allows also to illuminate two LEDs which are different distances apart and then move them across the receptive field with different velocities they could be also like you have all the LEDs up and you have dark points moving and then you can see how well the photoreceptor actually can resolve these moving light dots and what we do for quantifying it we just use the simple relay criteria which is to taking the smaller peak in comparison to the drone and that gives us the relative ratio of how well it could separate these two events and if it cannot then you have this kind of continuous line and you don't have a tip there so if you can see you have if you see two dots you see two harms in your voltage response if you don't see these two dots you just have a one continuous voltage response okay so when you do these experiments we used these movements which were quite fast so this is already saccadic 205 degrees per second the dots where I think in this experiment 6.8 degrees apart and then we run them double fast 490 degrees per second and then we have the interest of voltage responses of individual photoreceptor and this blue line is just using the classic kernel based impulse response model and so static model in that sense to respond and you can see that it really struggles to resolve those two dots but the real cells are easily doing it even when it's very fast and if you put the light background whereas the classic photoreceptor models they fail so they wouldn't be able to separate those so this is the idea of the slower integration time it says that the integrating is so slow that it cannot resolve them but in the real cells actually integrates them so fast that they can be resolved quite nicely and so I thought about that and I did experiments also from different flies different mutants for example histamine mutants but the photoreceptors are not synaptic connected to the network because I thought it might be the laminar network and feedbacks which are helping this resolvability however they gave equally good resolvability those histamine mutants so I then kind of ran out of ideas I thought that the only thing that could happen is that the photoreceptors are physically moving and so I drove to Cambridge to see Roger Hardy because we had the high speed camera system there and we rigged up some flies some intensity this is so called cornea neutralization method where you put a little bit of water on the eye and between the objectives and you can then zoom into these raptomy endings through the lens system and then if you have a beam splitter you can stimulate them with green or blue or whatever light and if you look at under infrared then infrared light is not stimulating photoreceptors so you can have the infrared image and if these lights are making the photoreceptor move then the photoreceptor movement is seen and indeed every single line this gray line here is when we just flash of light and you can see the photoreceptors in each case is responding and these responses are very big you know that this one raptomy is about two microns so in many cases this movement is bigger than all the size of the width of the raptomy when you are repeating this so these are directly caused by phototransduction so Roger has worked out that there is a molecule called PIP2 in the phototransduction cascade and its heavy end is cleaved from the membrane it's bit like you are squeezing a balloon more you squeeze the more it elongates and if you release it it returns back so these molecules are the light is causing these molecules to go bounce in and out the membrane and it makes like a squeeze box that the membrane bounce so it's nothing to do with the muscle it's happening inside the single photoreceptor but we demonstrate that by just separating the cells so this is just one non-material so these eight photoreceptors they are in a petri dish and we are just having infrared light and we are just flossing the white light the green light and the cells are contracting you see so there's nothing you can see any muscle it's just coming from the photoreceptors and you see there's two components there's an axial component this kind of lateral swinging component which is kind of waving the cells underneath are from TRP, TRPL mutants which are completely lacking the transduction channel so it's not producing electrical response but the transduction cascade to that point works fine so you can see that the PIP2 is able to be activated and inactivated and it's making these cells bounce over and over again okay so now when you look at this and you understand that these contractions they are part of the phototransduction and if they were not there the photoreceptors would be blind then you start to see the picture how this is really integrating into the active vision so this is an example this faster curve of interstellar voltage response and it's very fast and these are characteristic responses the contractions or these microsecades that the cell is producing they are a bit slower but they are still fast so you have to collect this with the high speed camera by negative eye you wouldn't see them you can see that the higher the light intensity then the bigger the response so they are working like a normal voltage response you see they give a big amount of light you get a big saturated response little light you get a pump so there's the same kind of thing the movement is really reflecting the activation of the phototransduction but slightly different than the voltage response now if you look at the different light intensity so this is a massive log scale so if we have individual photons they can even make the photoreceptor contract a little bit and we can separate those and then when you have a lot of photons and millions of photons per second then you get a bigger response so you see the same dynamics what you see if you just look at the voltage responses or light current but they are a bit slower now there's many interesting things so this is one what we use the system we put an extra piezo element so we could move the fly underneath the microscope so we could focus basically on the lens on the top of the materia there are some funny cells these are cone or pigment cells that makes a separate aperture that people have not really talked about and underneath them there are the photoreceptors these raptomia endings that have some sort of tip links into that aperture and if you then activate them by light you see that the lens is completely still and that's no wonder it has the exoskeleton there's nothing happening there outside when you're activating but the photoreceptors are moving very vigorously and you see that this aperture is also moving it's kind of a pendulum swing effect so this would be where the heavy end of the pendulum is where it's swinging the largest and here the movement is relative to that geometric relationship so it's moving less but the aperture is also clipping light and we've done other simulators I'm not going to talk about that and it has a very interesting effect that photoreceptors are collecting To the theory the classic theory says that the receptive fields are still because the photoreceptors are still it's a static cell you have the classic lens and the receptive fields if you do estimation of that by using intracellular electrophysiology they are from maybe about 7-8 degrees to about 6-5 degrees depending on the cells and if you have a 6.8 degrees part two dots which are crossing that receptive field because it is so wide there's no way that receptive field is going to show those two dips because it is kind of just mirroring them so there's no way that the light input as it comes through the optics would be able to activate phototransduction so that you have two dips however if the light comes to the edge of the photoreceptor you have to move against or with it and at the same time contract so the photoreceptor is moving away from the lens so it's collecting light from a narrow angle so the receptive field is narrowing and if you put that dynamic into your models it predicts that the light input it actually turns into these two peaks and these two peaks can very well try the phototransduction so you get that little dip there between those two humps and the photoreceptor is actually able to respond in time to two dots even though they are going very very fast and they are very very close and if you do the recordings and you see how the recordings and the dynamics pretty much match the real cell if you have that dynamic narrowing in your model narrowing and moving and so now what we can do is we can just simulate how well a photoreceptor could resolve two dots which are very close to each other and they are moving with different velocities by using the stochastic photoreceptor model and plus that new movement model that we have included in it and so all these which are in yellow are basically hyperacute so the intero material angle in trosophilis 4.5 decrease so it would be a line here and the old theory would not show any of that this would be just single hump things but the simulations based on the data what we collected from the real cells is giving them a quite nice responses and then when you do the recordings you do find that the recordings occasionally show the same kind of patterns which are very similar to what you have in simulation there's one caveat at that point when we were doing this we didn't understand that the photoreceptor movements are directionally wired so they move in different directions depending where they are in the eye so sometimes when we were stimulating we didn't even know which way we should be running the stimulus in order to get the maximum responses but regardless we were quite certain that it could result because some cells we by chance we had in the right movement axis and we got this resultability which is matching the simulations and we do discuss this also in the elite data when it was published now you can of course finally then demonstrate that this is true if the fly actually can respond the bars are 1.1 or 1.2 degrees apart and we are then rotating it here 4 to 5 degrees per second in one direction or the other this way and that way and the fly has this tendency of trying to prevent at least we believe the slippage of the image so it tries to follow and the fly cannot now follow because it has a torque meter so the visual field is still because the head is fixed but however it still makes the turns in order to follow the light as it turns and you can see that with this a hyper acute patterns is 1.2 and then 2.9 the fly is perfectly following them and then when you use the control patterns they are bigger because these are coarser but still the resultability is 70% in some cases for this very fine hyper acute patterns and the responses get smaller if you speed up the stimulus ok so that's the stuff that we published in elite paper was in 2017 and it was all good but it was mostly looking at individual cells and we didn't know how this happens across the eyes so we put a big effort since that to resolve the magic of the eye and I'm going to talk about that next but first just summarizing what we have so we had this micro-psychotic sampling and so this is purely phototransduction making these cells to contract in certain ways and this contractions moves the photoreceptor away from the lens or sideways and when it does that it is almost like a voice of action because it is trying to reduce the light load and the refractoriness in that sense so it can respond to the next set of lights or light changes and you can replicate this dynamics with the stochastic model and it predicts pretty well the acute investments and this whole system seemed to improve the acuity maybe 4-10 times beyond the optical limit we actually don't know how well it can perform because we have difficulties of making stimulus which is finer than that to test the system so it was clear to us that trosophila achieves hyperacute vision by encoding space and time so that's the half part that's the kind of stuff that we've talked some years now but now it comes to new stuff but it's far more exciting I hope and so let's get in it so I went to funeral my old supervisor Matti Vex from my dear friend unfortunately he killed over and it was a moving ceremony but there was other physicists there that I met and I then started to talk to them after the funeral it would be nice to somehow use X-rays to see what's happening inside the eyes and to see the global dynamics of these movements and they said we can do it and you just have to code the synchrotron and they came in contact and there was one guy Reimut Mokso who is a physicist who then worked with us and so we wrote the grant application to put flies in the synchrotron and so this is the European synchrotron in Crenouple and here we are we just put the flies in the tube we put it in this massive machine there's Marco, a Finnish physicist and then JP another Finnish guy with lots of fins doing X-rays who is working in the European synchrotron and we put the fly eye there and we turned the X-ray on and we give small pulses you can see that the whole eye the photoreceptors are moving in synchrony so it turns out that actually you can activate the photoreceptors by X-rays if you have a shitload of photos even though you have a very low probability to activate them and you can see them even better if you look at the same time on both eyes so this is a nice horizontal cut so you see the left eye and the right eye and when you turn the X-ray on they actually contract here inwards and so they move symmetrically they do opposing movement you can study this you can build the heat map and you can find out where these movements are fastest and you find out that happens where the photoreceptors are the longest and that makes of course sense because you have more microvillage there so you get bigger response in the frontal part of the eye facing forward and these movements are directly intensity dependent so the more X-rays you put the bigger the movements and after the when you've done these experiments of flies you can repeat the experiments you can do this for half an hour until they die and if you use dead flies you can kill them by freezing for example you see no movement so this is not heat induced artifact but we have to show that this is true but it's quite interesting because the X-rays they are 6000 times shorter than the visible light or the UV pigment sensitivities so what's going on here so we propose couple of things that could happen one is that you just played a game of probabilities you have billions of high energy X-rays and even if they have a probability of 1 in 10 million 100 million to activate one rodopsin you have a shit lot of them and you can still activate them by the numbers the other option is that the X-rays are activating some heavy atoms and they like phosphors and you have a quantum type of effect where there's individual photons which are being scattered and those are visible light photons and they are then caught by the the UV pigment or the cream pigment which are in the eye we don't know what is the right answer because it works so we can prove this so we built a portable electrophysiology this was done also in european sinctron also in desu, the deutscher sinctron in hamburg and so we put the fly in a tube, we put an electrode there and this time we just give a light first here to demonstrate you get the plastic ERG electroretinochrome you have the on and off transients and then when you do the X-rays you have the electrical responses there's the on transient and off transient the same way as using the real light and in addition you see these movements and which are shown here unfortunately they are not synced but they are happening actually in sync and you see that the higher the X-ray intensity the bigger the movement and also bigger the electrical response what you get and then after the experiment you can just the white light and you can see that the electroretinochrome is pretty much identical to the one what you had in the beginning of the experiment so the X-rays had not damaged the eye and so this allows you now then to work out a bit more about the global dynamics of the photoreceptors so we had now good evidence and that's a big part of the plus paper that we published to look at these but of course in real life these cells because of phototransduction and where they are pointing they are eats in different phases so its light little change in the environment is selectively activating only few cells the cells that they are facing to that light source and you can actually look at this by using a deep pseudo-bubble microscopy so we build a new microscope system and we the principle of pseudo-pupils is well characterized by Franciscanian others and we just use the new fancy cameras that are so fast to actually look at these dynamics so the principle is that you focus inside the eye and then you get to the point by way of optically these virtual images of neighboring omatidias eruptomias fuse together so this is about 200 microns from the surface of the eye and that's the pseudo-pupil so you have a non-invasive way of using infrared light which you can illuminate from the back of the eye and then look at the camera at the pseudo-pupil pattern and see the pseudo-pupil pattern moving so that's the theory so you can simulate them so we build this kind of computer graphics eye you can do this nowadays and so you can test by optically by simulations that how well the eye is able to move when you get this principle of superposition which I'm just spending little time here so the omatidia are those lens systems that make up the eye and inside it's omatidium are these 8 photoreceptors and you see the eruptomy index these are the flight sensitive parts and then when you focus in you get this deep pseudo-pupil pattern now if you then stimulate them frontally so we build a system by using a beam splitter again so through the optics of the microscope that we use we could stimulate these cells by using the local beam of light and now by in these simulations now we can increase the size of the light stimulus itself the special size and see how many photoreceptors in the neighboring omatidia are being activated by it and that leads to then the movement of the pseudo-pupil so again there's no muscle here so there's many many cells which are not moving it's only those local cells which are looking at them and because of the curvature of the eye and the screening pigments which are separating the omatidia or optically from each other then this movement is local and you can increase the number of cells so this is just simulating how it is and of course it also depends on the numerical aperture of the microscope that you are using how you get that light into the raptomias but again you can simulate so this is our microscope we use this Olympus Stereo and we had the numerical aperture of 0.11 so our stimulus where we were stimulated through the OIPC was about 12.6 degrees okay so this is the system we built so we have a microscope there's the fly it's in the tube and you can move it so it's a coniometric stepping motor based system so Joni, sorry who is here the little face here, he built this system and there's the fly in the tube Joni also made this video so he's really clever boy so there we have the infrared light so that goes through the head of the fly and we can then look at these movements when we give a light flash and this is what you see actually in a real recording so the microscope allows us to see the direction to which these individual photoreceptors in different parts of the eye are contracting laterally so which way they are moving and you can also see when you carry on doing this that there is a sterilization so the two eyes are simultaneously responding when you get to the center of the eye I'm going to go a bit further here because this video is so terribly long and I've got the next one so this is what you get in the end is a map of the photoreceptors across the eyes which way they are moving when they are being activated by light and the next on the right is a histamine mutant where there's no connections between the photoreceptors and the cells underneath because there's no transmitter to convey this information further and you see that these movements are identical in different flies and different mutants so it is already a property of the eye itself and indeed it is and we can show it so we can do the mapping of the stereo range so this is using the same system so we can see how these are responding to light so we get this kind of stereo receptive field and it's about 30 degrees in width we also can map the organization of the raptomere so they curve around the eye and then form this kind of diamond pattern and it turns out that the photoreceptors when they are contracting which is shown here so there's a fast phase and slow phase which is just moving here and if the photoreceptors are moving along this R1, R2, R3 axes everywhere then you get this patterning and that's indeed what happens so that means that during development there is an anchoring which sets the cells, the curvatures how the raptomere are moving around in this diamond pattern and then allows the fly to move laterally when it contracts in a certain directions only and so there are ways of doing these equations and we can combine them with simulations we can look at this movement so this is just showing that each time when the photoreceptor is moving it does actual movement so it's moving away from the lens which is shown here and you can actually measure this directly from the videos because when the raptomere move away from the lens they actually get dimmer so you see the intensity drop directly which is in the same phase as is the lateral movement so they both happen at the same time but also look at this movement so I'm asking here what do the photos the microsecades do so they actually because they are part of the phototransduction they are not reflex like contractions they are more for dynamic active sampling and here we just use the same system but giving this time a different patterns so sinusoidal light adaptive patterns and you can see that how dynamically they are responding and those responses are different depending on the light history how they have experienced this speeding up sinusoid and if we use the square wave then the responses are bigger and then we can just use just the normal contrast pulses of different types and they give you big responses so again I really had the emphasis these are not just reflex like some silly things they are really reflection of the active sampling and which is completely continuously doing into the existing light conditions and they are responding the positive contrast so light increments in moving one way and negative contrast light increments moving the other way but bit slower so there's two phases there's a fast and slow phase and this rippling happens across the eye all the time underneath that or material hard outer surface so now what it means is we now know the directions of these microsecades and we can then see what happens in different positions in respect to the optic flow field when it's flying and then it turns out that it actually helps fly I'm going to stop here to resolve battens differently so when you are flying the things from the earth are whooshing by faster and so the fast phase of these are tuned into the things that are happening underneath and then the skyline with the clouds are moving slower and the slow phase is tuned into those really identically or ideally when the fly is in its normal flight position and also when the fly does a rotation because of the microsecades it enhances the contrast of these two eye signals that are coming in so you get these two phases and so there's many type of ways it can enhance the photoreceptor responses and beyond in the synaptic signaling so the integration time of the photoreceptors because it's a dynamic sampling active sampling is not playing that big role anymore so next thing we can ask that is these saccades doing anything for the neural processing and this is Kavan so Kavan was building on this two-photon imaging system and particularly looking at the different stimulus so we have a fly moving on a pole and we are looking at the monopolar cells and we are responding to dark patterns so they give a bigger response to that and we use the same type of stimulus that's getting faster and faster and finer and finer to see what is the real resolvability this is just demonstrating how the information is going from those eight photoreceptors to individual monopolar cell and that the photoreceptor is actually moving in certain ways for the given stimulus and then we are predicting that this would give us orientation sensitivity and then we will see that when we change the orientation of the stimulus the resolvability is getting better as our theory would predict and then you will see that when it gets this particular cell and this is how we test it so we have an orientation which is moving now from right to left and the resolvability is there where we can see that it's bigger changes again right eye criteria but we will see that when it gets this particular cell is really favoring the vertical stimulus which we are getting soon there and you can see that how much better the resolvability is there and now we go to the vertical and all of a sudden it goes Wheee Look at that so it's the receptive fields if it was a static eye we would be having the same resolvability in all directions but you see the monopolisos are tuned and these receptive field orientation axis they actually matches the photoreceptor movement axis so we map that as well and together and you can see that when you go for one terminal to another this map is moving as predicted by the photoreceptor inputs which are moving so this then leads to the theory that you have 3D sampling in time so the classic old theory says that those cells which would be sharing the frontal receptive field left and right eye would be perfectly matched the receptive fields and there would be no photoreceptor movement so when there's a stimulus coming you would get basically two identical responses from the left and right eye corresponding photoreceptor so you could not really use that working what is the depth of the stimulus so here is that one set of the cells are moving with the stimulus and other set of the cells are moving against again you have to remember that there's a lens which is like a lever so it inverts the motion and so that means that the photoreceptor inputs they look basically different depending if it's left or right eye and depending which way the motion is going when there's two peaks and what we were speculating that these phasic time differences would be good enough for the fly to work out what's the distance of the object and how do we do that so what we did we used the Fourier transform beam propagation method so we took those photoreceptors you can see that they are different sizes so this is just a classic raptomius from 1 on Matidia you see the R1 and R6 are always bigger than R2s and 3s and 4s and 7 and 8s are in the center and because this is not symmetric but you will not have a perfect mirror superposition because it would only happen if these were like a flower pattern so everything would be the same distance away from the center and because the left and right eye are mirror symmetric on their organization and how the raptomius are arranged there then these all together mean that when the individual cell is collecting or the individual material is collecting information using these photoreceptors their pickup pattern in the visual space or the receptive fields as predicted by the optic light guide properties of the cells they are not going to one dot but they make this kind of rhomboid pattern and so that means that when you actually collect information through two eyes the images how they are superimposed from left and right eye they don't make clear peak cells which would be the old theory but they were completely mapping the whole visual field and that mapping is different depending what's the distance what you are away from the eyes and so these are the stereo fields so they vary and so the classics theory says that here we have David Beckham by the way we have a copyright for that image so Kevin asked the photography and he said yes you can use it so we don't do anything illegal here but we are having this four and a half decree sampling matrix by the omatidia and the inner omatidia angle being that four and a half if you are superimposed how David will be collected David will be very blurry you can't really tell that if this is David or Miko somebody and so we think this is all rubbish and instead what happens is that the photoreceptors this is the real dynamics when we are simulating them over the matrix so they are moving as when the stimulus get the receptive fields they start to move either with or against and they are at the same time narrowing and we have now pretty much proven that this is true, we haven't published this so I'm not going to talk too much about this but this is quite exciting so I'm going to show you so we used artificial neural networks okay, one did this and what we've done is that we look at the real connectom which was published by Rivera Alba a few years ago and so we put that structure into the RNN and then we took the real photoreceptor movements, you saw that matrix how it's happening so we had three cases, we either had a static eye the first model so we have ANN or RNN and the photoreceptors are not moving or then we have just one photoreceptor feeding all these cells and moving or then we have the real dynamics of all these together and here's a pattern, so we do the calcium signals and this is what you see and you can see that it can follow to hyperacute range if you have static eye everything else is normal but it cannot produce any hyperacute if you have a one photoreceptor feeding it cannot produce a bit better but if you have the real dynamics of the real cells it gets the same phases it gets the the same kind of resolvability these others they are face lacking because these movements are actually doing predictive coding they are making the stimulus the neural response actually lock onto the stimulus as they are moving along so you really have to put everything there is that evolution has thrown at you you have to have the stochastic sampling it has to be arranged the right way across the eyes you have to have the real connectome and if you do that you can predict pretty much bang on how the network is working and its sensitivity and that's what we've done here we've done really much more on this but I'm not going to talk about it because we haven't talked about it ok so what we showed is that if we have a static eye it fails it cannot have hyperacuity if we have real data it does have and if we have the realistic microsecond it pretty much replicates the real data and now we can go to how this works in a dynamic sense so we have these two receptive fields we look at the R6 from the left and right eye R6 5 cm away this white line and we are just using this white line to cross these receptive fields and that's how they move and that gives the face differences of those inputs which are feeding into the lamina and into the fly brain and if you have a a correlated model so if you have two fold receptors left and right eye which are the same point in space and another one which is the neighbor this is you need this extra cell to do the normalization of the signal you can directly translate the distance in space to distance in neural time and when you do that you can predict what is the stereoscopic range of the fly and it turns out to be that it's less than a millimeter to about 10 centimeters and the fly is seeing perfectly stereo then it gets more difficult so we can test this by behavior so we have a hypercute pin here against a black dot they have the same base so this is actually this dot and the pin but if you look at sideways and if you have a monocular vision they would look the same but if you just tilt a little bit you can see that the center one actually is the pin and so the fly is free to fly and controls this panoramic arena it can look at whatever it wants and we are just testing its salience which one it finds more interesting three things to choose from and we are hoping that when we put it in different positions that it would select the pin because it's more interesting and indeed when you do this in many flies and this is one fly and we test it always all these three options the control and the different pin positions and we've done it in 20 flies and the stats are very clear the flies find this hypercute patterns more interesting when they are three dimensional and it focuses on them we can also put a hypercute dot and hide it among a hypercute stripes and it finds them we can put it in different locations and it always finds them and we can test it in controls and it finds it's not interesting I just realized my talk is getting very long but we are almost in the end then we can do it by learning experiments we can teach the classic T inverse T that Heisenberg has shown to work very well and the fly can resolve or avoid if it's linked with the heat whether it's a T or inverse T or vice versa so the learning index is there but we can also have stripes with a hypercute then we can hide a three dimensional pin and it learns very well or we can have just dots and pins on different types this is just the one variety that would be tested and they all learn on the other hand if we paint one eye black the fly doesn't learn anything for these hypercute patterns this course T inverse T patterns and this is consistent with what Tung and Heisenberg showed that there's a visual invariance in the eye so if you learn something on the left eye you can also test it on the right eye and so big patterns which doesn't require stereo vision it still learns but if you need some small patterns which requires stereo vision it cannot learn and then of course we have controls of all sorts of blind versions flies that shows that this is nothing to do with any other stimulus but most beautifully by accident because we want to test how there are one to six which are these other photoreceptors and are seven and eight how much they are contributing to stereo vision we found out that if we had this Nord A, these blind flies and we only rescued those other photoreceptors something goes wrong in a small percentage of flies and in those flies we get electrical responses of both eyes but somehow the anchoring of the one eye is not showing lateral microsecades so we did imaging by this pseudo-pupil method and we did electrical retinocrones at the same and then the behavior from the same flies and it was a massive heracolian task by Ben and Joni but we could show clearly that if the microsecades were not synchronized the fly was not able to learn it was not able to see those stereo patterns and it was not able to see either AT inverse T patterns and we also showed that you can also learn if you use only the center of photoreceptors you just you learn better if you have both working but both are contributing in the stereo vision both are seven and eight and are one to six and then finally we showed that this old classic believed that there would be a spatial adicing that if you have a coupled interromatidial angle somewhere like seven degrees pattern in terms of waveform that you have this optimal reversal but what we found out is that this optimal reversal comes actually from a perceptional aliasing and that these hyper acute patterns here they are strongly if we use a small cup where these are closer to the ISL what theory predicted and so they are still giving good responses to hyper acute patterns in a big cup but they are not seeing this as well because the ability to resolve 3D decays with distance and then we tested this by painting one eye black and when you paint one eye black then actually with this same six to seven degree patterns you don't get any optimal reversal so it is the competition between the two IS when they see it and it is also depending on the velocity so you don't see any aliasing if you have a fast patterns but you see only this reversal when you have a slow pattern it is the same velocity as the velocity of the micro saccades so in one eye they are completely locked into the stimulus and the other eye goes against them doubling so the eye which is locked to the moving stimulus of course don't see anything because it moves with them the other one sees the contrast so it thinks that the world spins the other way around and so you get this illusion that you can actually explain by the micro saccadic mirror symmetric sampling now you don't need the spatial aliasing you cannot have any effect because of the ripple effect of this and the organization of the raptom is being not symmetric so there is no way you could have an aliasing effect especially and it is actually a behavioral thing and we can prove it by using different velocities so you see here with the higher velocities it works normally and in lower velocities not and then when we paint the eye black one eye then you never get a spatial aliasing so I'll stop here this was the peak message so the micro saccadic sampling photoreceptors contract to light and it's already at the level of sampling that defines the stereo vision or the abilities of stereo vision at least in these insects and that the refractory sampling I didn't really make a big case of this but you just have to trust me it is very important in improving the temporal resolution of the signals and it's together with the micro saccade and that refractory sampling that really gives these cells and the ability to resolve things in hybrid Q3D and that's it, that's the team ok I'll stop here Thank you very much Miko for this very interesting and entertaining talk I'm going ahead and posting the zoom room link in the chat already there are a couple of questions appearing but I would like to remind to our audience that they can either ask there or join us already in the room to ask directly before I go on with the questions that are already in the chat I really enjoyed how you go from x vivo to in vivo to x-rays to behavioural to all sorts of modelling from numerical to neural networks and the first question that appears is from Sam Budov who says very cool work can you please elaborate on how movement was integrated into the recurrent neural network Yes, so we you saw those movement receptive fields this is exactly the same dynamic so we put it in the sampling matrix which was then feeding the network and then it was then looped so we were minimising the error between the you have the connector and the movements and then the calcium signals and then this was the training was happening by massive data sets what Kevan has collected and so that's all I'm going to say about it because I don't want to rain on Kevan's parade we hope to start working this very soon and then turn it into a paper and so I don't want to keep all the fine details that he needs to tell himself and get all the glory for it and then the second question from Sam which is a little bit more extreme I would say can you speculate based on your x-ray stimulation results how this may relate to hallucinations experienced by astronauts that's a really good question and some other people have asked this and I think it's x-rays that you experience in the International Space Station or Kamara's they can I'm pretty sure activate there's studies by looking at the x-ray vision in 1950s and so by some Americans and Russian and Krekko Pelusi he was pointing these studies to me and there was evidence really that you can activate the phototransduction in frogs for example by just using x-ray machine so I think this also we know now there's been a couple of works on the vertebrate vision that the rots and cones they also contract and they have very similar dynamics so this may be also a way of reducing the motion blur in the same way as what happens although the retina is different so I think the evolution is really clever it really goes for like all this nimble AI comes with tricks how you can you can go over the limits of the classic optics like a stationery by just using motion there in sampling right thank you very much for addressing both of them as there are no more questions appearing I will take the advantage of asking some of mine and in the meantime I would like to send to tell you that there are a lot of people congratulating you for your talk because you don't have the YouTube open yourself so the question I have is like also seeing Kit Longden being in the chat is we know that there is like a synergy of color and motion and we also know that you know in the real world you don't only have like human white or just drosophila white light have you tried like either to model like more of the like not only to stick to the r1 to r6 but also the experiment to try like different lights yes so we that's part of the the penance paper so the penance paper has 150 pages long supplement where we we use all sorts of mutants to look separately r7s and r8s and different cells and having different spectral sensitivities and yes so you get the biggest moments then when you have the most cells activated so if you have a UV light it activates all r7s it also activates all r1 to r6s because they have the sensitizing pigment the antenna pigment and so that moment is then of course bigger than if you were to use selectively say some blue light or ample light ample for example to be only activating the r8s tree receptor in the r8 cells and so you would see only one cell movement it's very interesting because the movement this these photoreceptors normally they're connected their link so they are not moving independently so if one is activated the whole lot moves and if they all are activated they all move and it comes to really interesting dynamics and there's a bit of that in the supplement where we do simulations and this is also related to that slit effect I thought about talked about a little bit about the aperture how it's influencing how the light beam is actually being collected by this so this really leads to some sort of predictive coding by motion and how the eye is organized by this motion vector axis is how the photoreceptors are basically looking at relating to the most probable way of you moving in the world where many things are vertical and so it's and it was clear in the cave and simulations where we looked at the calcium signals in the RNN and the moving photoreceptors is that if you had the moving photoreceptors going all the right dynamics the signal is actually replicating the real so you don't have face still cells they are lacking so yeah please please go on so that's quite nice so the whole system is able to gather to get the benefit so it's so what we have been doing and very kind of systematically is really have this kind of a bottom up approach where we built this multi scales models rather than make a big claim that I have this idea it must be high order we jump onto some cell and we put there some engineering maths and say oh there must be this kernel and there must be this static nonlinear and this is what does because nobody knows so those cells in that high order they are equally clever to photoreceptors they are all more for dynamic they have a structure which is moving and doing something there in a more complex way that we keep them credit for but what we want to is we want to kind of give them a job which is what our models have been useful but it I'm pretty sure in many cases it doesn't give you the absolute limits or absolute thing it also gives you the illusion that the eye is killing information which I often hear people talking about they said oh it uses only the spike 0s and 1s we don't need this information but you know the information is already moving and changing the structure of the cells it's changing the biochemical cascade this might not be translated directly into the patterns of something that it's constantly making that tissue it's living it's like your bicep when you're doing so so the information is not killed in a sense that you are losing information they may be that the information is distributed in a way that is a kind of a combinatoria distributed in complex ways that we don't know yet but so holding into one cell and giving it a high order function I think it's dangerous that's my point right so the question I have was saying is like because it looks like there's kind of an orchestrate I know it's not orchestrate but it looks like all the photos are moving towards a single direction right have you tried to shine light at two different areas of the eye to see if it is actually like driven by a higher process or if it isn't there's few things that we want to do so these maps they're really we are giving light into different positions of the eye and that location specifically moves the photoreceptor but not simultaneously right so you're giving light to different areas for multiple areas we only can do it so when we have a frontal stimulus so we have two sets of cells which are doing it simultaneously so the left and right eye so they show the dynamics and they are mirror symmetric so we know that but we haven't done experiment where you are looking at one point in the eye and then you stimulate the other I have to also say that what we are not looking at here is the muscle movements and I'm sure the muscle movements the intraocular eye movements they are important they have a function and we looked into this to some degree the problem is that when you are having a head fixed flies the flies actually reduces this kind of spontaneous eye muscle activity they are like oh I'm in a straight check I don't want to do anything and then you only get this kind of low order things but of course in the real world you have the head movements you have the body movements you have the intra muscular retinal movement you have the photoreceptor movements it's a very complex thingy and it works together there so it's not easy so what we simulated it's only data in the head fixed flies how we see the situation but I'm sure it's different if you have all this decrease of freedom messing about right and like one last question I have from my side and like I would like to remind to our audience to either join the zoom room link that I'm posting in case they want to keep track of what we will be discussing soon or you know like we will see them next time I guess so like the question I have is like what I find hard to put together in my mind so the LMC neurons kind of pull information from different photoreceptors right so you say that the photoreceptor output improves and it can detect like these two bumps in the output that comes from the input but how will the downstream LMC neuron be able to resolve that if both of them move together like if two photoreceptors like I have one slide I prepare I mean I know this is pretty much too much but if you want to see it I can answer that question so that's okay so I'm just going to go down here we've done work on Muska so one of my PhD student Neven Manchur was looking into you see this now not yet you have to start screen setting again sorry sorry I thought I did but I didn't share yes share now do you see now anything meaningful yes we see these slides okay so there's a couple of things first you know these micro saccades they happen we looked at them in Muska in Housefly and we looked at them in Honeypiece which has opposition eye and they happen there as well they are tuned into the receptive fields of the cells because Muska has such a small interromatal angles the micro saccades are massively faster and smaller so they go in the same scale so this is just Muska doing it so in in case of Drosophila you remember it's about 100 microseconds 100 milliseconds and then they were peaking but you know these guys go in 10 milliseconds so 10 times faster and so then when we go to Muska photoreceptors so here we use the same stimulus that you saw for Drosophila and you see the same thing but here they go to 200 hertz burst and then you look at the monopolar cells with what you were asking they go really really shit fast now if we look at the information rates in the monopolar cells what you get is like 4000 bits per second if you have burst the stimulus which is tuned and why it does it so this is illustrating what is really amazing that hasn't really been looked at in monopolar cells and so hopefully will be published as soon as but if you look at the photoreceptor patterns to these burst things they are these monotonic changes but you see the monopolar cell is a biphase it gets responding when the response is going up and it's responding when it's going down so what it effectively does is it does a frequency hopping so when you look at the light stimulus which is shown here in cray so this is the sign of the noise rays of the light stimulus stochastic pattern and you see that the sign of the noise rays of the light stimulus goes to zero about 700 hertz but the monopolar cell can follow up to 920 hertz and it's because it's doubling the frequency it's doing the frequency hopping so it's taking the energy of that high reliable low frequency information and it's backing it because it gives the two signals to the same pattern one pattern you have this kind of a kernel like monotonic response and you can double on that and these are so there's no noise there's hardly any noise when you have stochastic sampling because the sign of the noise rays are so massive so there's a massive underestimation of the goodness of how good these things are operating and most of the noise is actually operation or just working adaptation and it's not an issue into this and so it really does the informax and then uses these kind of things that we really talk about them as a nimble AI so you do these tricks and there's other tricks but this is just one thing for your question that how can it take that information and turn it into time and it can Right, thank you very much for this clarification Miko, I have a lot more questions but I think it's time for me to wave my moderator rights so I will be stopping the broadcasting so we can continue here with people that are interested so thank you very much once again for giving this talk in our series Okay, my pleasure, thank you