 Okay, let's finish where we stopped last time due to the fire alarm. We'll talk about masking and the remarkable fact that of backward masking, which means that you have a stimulus and then let's say 50, you have an image for 30 milliseconds on and then let's say you can have, if immediately afterwards, for instance, you have a second image with lots of lines, lots of high contrast lines or random color patches or something like that, then the impression of the first image can be totally wiped away such that they erase from your mind, such that if you do it properly, you won't even see it at all in the strongest case, that you can totally remove the image from visibility as if you never saw it. On certain conditions, you can still show priming, in other words, that something in your brain did register the image, albeit not at a conscious level and you can show that statistically that the person will respond to something that was present in the image, although the person claims that he didn't see the image. And this masking can go on up to 50, 60, 80 or up to 100 milliseconds afterwards. So that one interpretation is not the only one, but one of the interpretation is that you have some sort of either an integrator or feedback system that integrates up the activity, that feeds back the activity over 50 to 80 milliseconds and that's how the second image can interfere with the perception of the first image. There's some interesting illusions that I think are very, very useful to study the microstructure of consciousness, to study the micro phenomenology both at the phenomenological level, in other words, at the level of what you see, at the level of phenomena, what is it you see, as well as the detailed psychophysical time and ultimately the time cost of the underlying neurons and the, for example, functional imaging on neurons. So this is an illusion that was found by Michael Herzogor, who was a postdoc here in the lab, two years back, and is now in Germany. So it's quite a strong illusion. If you do it strongly, you have no idea what's going on. So the main version, there are lots of different versions of this illusion now. He sort of made, published eight different papers on this. So in the original version, you flash up, this is called a Vernier Stimulus. It's a stimulus where the upper bar is a little bit to the left, a little bit to the right of the lower bar. And you're exceedingly good, your visual system is exceedingly good at telling what this bar is to the left or to the upper, or to the right of the lower bar. You can do this with probably a 20th of the spacing of your photoreceptor. So the performance in trained humans can be very, very strong. So you flash this up for 30 milliseconds, and then it's immediately followed. This image is removed. So this was done on an oscilloscope for precise timing. This image is removed, and then this mask is flashed up. So this we call the prime, and this is called the mask. Now, in standard, for example, backward masking, you would have sort of random lines that flash up here instead of these ordered ones, and you wouldn't see anything. If I just flash this, and then immediately I flash a whole bunch of random lines, you wouldn't perceive anything. You just see noise. Now, here what you see is a very powerful, very compelling percept. You see this. In other words, this grating is offset, and the direction of offset is given is determined by the direction of the prime here. So if the prime is here to the right, then you'll see the entire grating offset to the right. If it's to the left, you see the offset to the left. Now, this distance here is not equivalent to the distance here. This distance here is larger than the distance here. I mean, this is not a scale drawing. But the idea is, I mean, there's this very strong effect that you'll always see it on the right or on the left, depending on whether the vernier is like this or like this. So this is rather interesting because what you're doing here, your brain combines, there are two discrete stimuli. So A, the brain doesn't see two discrete stimuli. It sees one stimulus. And B, it combines properties of both stimulus into this synthetic one. And these stimulus actually never existed out in the world. Only these two stimuli, physical stimuli, existed. Yet you meld, the brain melds, or this is why it's called inheritance. This grating inherits a property, or this prime inherits one of its properties, namely its offset from the preceding stimulus. And this generalizes two different dimensions. So for example, if you instead of using this vernier, you use these gratings that are these bars that are slightly oriented to the left or to the right, you see the same thing. And if you do it with motion, so if you flash up these bars first here, then here, then here, then here, then here, you see it move, then you'll also see the final percept move. So, and there are lots of subtle variation of this paradigm. So I think it's really, it's extremely useful, and we have, there's a postdoc in the lab, Wei Ji Ma, who's trying to study the, sort of trying to understand from a point of view of dynamical system, from a point of view of neural networks, what is the nature of the dynamical system in your visual cortex that can give rise to these different phenomena, right? So if we just had this thing without, if we just had the prime, without the mask, you would see the prime. So it's not that this is so short that you don't see it. So with the prime by itself is visible, but here it interferes, but it interferes in this very funny way that it, you know, that you see this, but with that property superimposed, or you see this mask with that property superimposed. So it's non-trivial, and the nature of the underlying processing is non-trivial. I like it because it's very simple. These stimulus are very simple. In principle, you can get an animal to train this, to get a monkey to discriminate them, and you can do, you know, all sorts of different techniques. In Germany, they're doing TMS now, they're trying to interfere with some of this perception by sort of blocking, by briefly shocking the brain with these brief magnetic pulses. So it's just, it's one of many phenomena, I could have also shown a bunch of other phenomena, where A, what you see is not what's out there, right? This is not out there, that's out there, and what you see is governed by, it is still, though, governed by lawful phenomena that I think are very revealing about the underlying neural dynamics. It's always less. It's probably like, I don't know, 20% or 30% or something like that. So here we did not work. I don't think we pushed it all the way to the Bernier limit, which would be like three or four minutes of arc. I don't think, at least we didn't do that here, he might have done it since then. But I would suspect that as long as you can see this by itself separates, this also separates. That would be my feeling, yes? I mean, this distance here, it's not the same as this distance here. Well, I mean, you can, so here we give, we ask people to, we do it with a comparison, and we ask people always to do the discrimination, whether the grating is to the left or to the right. We ask people to do a two alternative false choice threshold judgment. Speed of processing, so as I said here, if you flash up, and we know that from, and I showed you this math last time, you can flash up in single image, in a single frame, and you'll see it. So what people, so this is what you see here. This is flash for single frame. Now, of course, it's on a Macintosh going through the LCD, so it's not, it's probably slower. But you can clearly see, you can all clearly recognize any of these images, right? You shouldn't have any trouble. Now, of course, that's an interesting point. For example, if I ask you, you know, if there's a butterfly in here, you can all see that. But if I show you a hundred of these images and I ask you at the end, was there a butterfly in it, you have great difficulty doing that task. But that has to do with memory, not so much with perception. This task was used by Simon Thorpe in France to try to get at the question, how fast is the visual system? So we know that a single exposure of 30 milliseconds is less than sufficient to mediate seeing. The question is, how long does the brain take? Of course, from reaction time, I can tell you, you know, the reaction time in a speedy trial where you push down, where the thing is down on the switch and you have to lift it as soon as you see something, recognize it. So some of the fastest time can be as well as, you know, as fast as 50 milliseconds. But that, of course, tells you, that tells you the entire duration, including, you know, the motor behavior, but so you want to get at the specific question that the visual system take to make this decision. Which is, of course, different from how long it does it, will it take you to develop consciousness for it? That's a much more tricky question. And right now, I really don't know, there isn't really any good technique to get at that question. So here what they did, they had these images and then half the time, these images contained one or more animals, a butterfly or a lion or a pride of geese or something. And half the time, there were no animals, for example, there were vehicles or something. And it turned out it wasn't only, this is not what I'm telling you now, it's not only true for animals, it's also true for vehicles, so it's probably something general. And then essentially, they measured, in this lab, they measured EG, they had a whole bunch of EG electrode, and they looked at the difference between having an animal and having not animal. Now, your performance on this task is almost close to ceilings, almost always perfect. Humans are incredible good at this. So this is not only true for trained subjects, but they can essentially take people off the street and put them in front of this monitor. This is something we all do natural, presumably this is something that we learn in the first year of our life. And then they look at different electrodes, it turns out it doesn't really matter too much. And they always subtract the signals, for example, here when you look at the animals versus non-animals or vehicles, when vehicles are targeted, when you have to push a button or release a button, when there's a car or a bus or a plane or something as compared to non-vehicles, etc. And what you can see that always here, there's a signal at around 150 milliseconds, this is always a different signal, these are the individual signals, and you can see depending on where you record them, they go up or they go down, depending on what part of the brain you're recording from. But here what they do, they always subtract at the same location, they subtract, for example, when animals are targeted versus non-target, and then they see here is roughly around 160 milliseconds, some process in the brain could discriminate an animal from a non-animal. Okay, now that's really very fast, it was in 150 milliseconds, so that's 150 milliseconds, it's been flashed on the retina. So these are bright images, it's going to take you at least 30 milliseconds to get to the retina. By the time you're in V1, in primary visual recording it's probably 50 milliseconds, so you have only out of 100 milliseconds to go from V1 all the way to wherever in the brain you make this decision that an animal is present, an animal is not present. And these are very common, these are natural images, A, the natural images, they're complicated, colors in fact doesn't matter, they also show that, although colors make it much more vivid, you can also do the same with grayscale, it's not slower. And the animal, of course, it could be, you're not being told it's always a cat, it could be one of a gazillion different animals, so it's really quite a remarkable performance, in particular if you compare it against the processing time, as I mentioned last time. With a little neurons in cortex, you hit it with input, it probably takes 5 seconds to spike, and then by the time the next stages reach, it's probably 10 seconds, it's probably on the out of 5 to 10 seconds between stages. And so with 100 milliseconds between V1 and let's say infotemporal cortex where this computational future from gyros is likely to happen, you've got 10 cycles, 10 iterations, that's not a lot for taking this decision. So it's really quite a remarkable performance for conventional digital machines. It didn't really matter where you recorded the signal from, that reflects the fact that due to the skull, the signal is quite attenuated, and it's very difficult to do any sort of specific pinpointing. And this sort of experiment would be very difficult to do an fMRI to try to see, I mean, I guess you could subtract, but that would be too slow, right? You could try looking at the hemodynamic time cost of animals and non-animals, and that just would be too slow. That's the trial with functional imaging, you're looking at the regulation of blood flow in response to increasing neurons firing, and that's nothing to do with the technology, that's just the fact of our human brains, they take blood flow relatively sluggish and takes a few seconds to set in so you really couldn't do this, but you could do it in an animal. So the bottom line is it's very fast and this gives rise to theories, we'll talk about next year in vision class, we'll talk about theories that suggest that a lot of this recognition, at least for things like animals, has to be done in a completely feed-forward wave of activity, what Crick and I call the net wave of activity. You have these neurons that raise up, you've got the spike wave that raises up, the optic nerve into the algae and into V1, and then essentially you only have a single pass, because by the time you hit the fusiform gyrus, infotemporal cortex, well, most of us think this computation actually occurs, you know, you don't have time for more than one or maybe two or three iterations at most. So some people sort of have now models based on this that suggest that at least these tasks can be done in a sweep of a single spike wave moving through the system. Now as I mentioned, this is not to be confused with the time it takes to be conscious, it's not the same. We do not know right now how long it takes for you to be conscious of this, it might be a couple of hundred milliseconds later. So this might be evidence for some sort of crisis in that you do this unconsciously, and the conscious decision there was an animal present and might come later. It's very difficult to get at that, very difficult to get at that experimentally. This shows timing, but this is based in a monkey, so humans it's going to be a little bit different because everything is a little bit bigger, but this is for a macaque monkey, the times, you know, the retina, this depends on very much on the brightness, like dim stimuli are slower than very bright stimuli, but you can see here within 40-60 milliseconds in V1, then infotemporal cortex where these high level object descriptions are stored, at least in a monkey, it's 100 milliseconds, and then you know, maybe there are special shortcuts that go directly from here, or you go from here to prefrontal, then motor cortex and then down to the spinal cord to push the button or to release the lever. So it's a rather fast system. Yeah, let's skip that. Okay, that was just to finish off last week's lecture. So now today's lecture. Yeah, before I come to that, we looked at the homework for, and the homework for among others were these short little essay questions from an article that we asked you to comment on, and we saw almost everybody made this mistake, which either is, well, this question asks you to distinguish content from consciousness, from consciousness per se. So it revolved around the question, is there not a single brain area that can knock out consciousness? Now consciousness per se, that is true. There are very small lesions in a part of the thalamus called interlamina nucleus, and we discussed it since chapter 5 if you want to read up on it again. In the interlamina nucleus, either a whole bunch of them in the thalamus or in the brainstem, the very small lesions that can lead you, that can make you unconscious. You can be in coma, you can be persistent, you know, vegetative syndrome or even persistent vegetative syndromes, like a few 10,000 Americans each year. And then you are sort of, at best you might have some diurnal rhythms left, some sleep week cycle, but you're for sure not conscious. That's true that there are single, very small discreet lesions that enable that or can disable consciousness. But if we look at the content of consciousness, which is what we are mainly interested in, the NCC, there's no specific one lesion that can knock out all, there's no one area, let's say in cortex, where all the different NCCs come together and such, if you knocked out that area, you wouldn't have any content of consciousness, although you would still be conscious. Now it is true that for each individual class of percepts, there might be such area. So we talked about it for motion. It is true that in a few cases of patients who have lost motion area bilateral, this is bilateral represented, that relatively small lesion, comparatively small lesion, I mean the lesion might still be as big as my entire thumb or even somewhat larger, can lead to specific loss only of one aspect of consciousness like color consciousness or face consciousness or inability to perceive fear, but not generalized fear, but only the inability to read out, and experience fear in other people, if I'm looking at faces of people. So some of this loss of consciousness can be very, very specific. Yeah, I wanted to talk about today a little bit about the underlying neuronal basis for attention. So we mentioned attention, I showed you some of these illusions, some of them rather spectacular, that show that if you don't attend, you can be oblivious to large, not only to these little effects that slip through, like if I show you things for 100 milliseconds, but very dramatic things that you're looking straight at, you're concentrating on something else, you're attending to something else, and you totally miss them. In general, today we know that we're beginning to understand the brain basis of this, and that the underlying neurons are up and down modulated as a function of attention. I think if you think about attention, it's extremely useful to keep a metaphor in mind, and I use the election metaphor, I really like that. So the idea is that the brain is, you have all these neurons, and they're competing, they have excitation and ambition among them, they compete for, if you have, they compete for firing, if you want, for the strongest response. And I believe that most of the attention at the different stages in the brain can be understood as arising out of this competition, when you have two or more stimuli present. When you only have a single stimulus present in the visual field, like a blind or a bunch of moving, you know, cloud of moving doors or a single face, then there isn't any competition, because the neurons will respond only to the face, there isn't anything else to respond to. But when you have two or more stimuli, like very often occurs, usually occurs, when I look at, for example, all these faces and bodies and colors and body parts, and so if I look at you, particularly in my higher part of the brain, where the receptor fields are very large, then the neurons are going to be confused as it were, so some of the stimuli in their field by themselves might excite them, some of the stimuli in their receptor field might inhibit them, so if they see all this together, you know, it's a mishmash and they might not respond at all or they might only respond very weakly. And so that's why you need attention and people have sort of tried to make this more formal. The most popular model in the field is called the bias competition, where you have this competition and you bias it from outside sources. And I think a good metaphor is the election metaphor. You have a country like, you know, a big democratic country like India or the US. You have these coalitions. Well, you have the election for prime minister or for president and you have to have these coalitions. These coalitions are dynamic. Of course the time scale is very different over the time scale of weeks or months or years in the political system. It's at the millisecond level in the brain and everybody has a vote and there are very strange coalitions of convenience that might arise for short times, right, depending on some piece of legislation there are sort of people who usually might oppose each other. They sort of... They are partners because they want to get a certain piece of legislation passed and then you can have certain events enhance or reduce the visibility of a particular candidate. So you can think of that a little bit like attention. You can have, for example, outside money coming in that certainly boosts the campaign, boosts the saliency, boosts the visibility of a candidate. You can have scandals or, you know, sex scandals or anything else that sort of boosts... you can think of it as top-down or bottom-up attention. Ultimately, at any given point in time you only have one winner. Now for consciousness it's probably not quite true. You can be conscious of a few things, not only at one but maybe two or three things at a time at least for short times and so the analogy doesn't quite hold water but the idea is that you have this competition even if you win the competition it doesn't mean you win for all the time it just means you win until the next election until somebody else can come in. You also have timing effects in the election it's really a very rich metaphor. I really like it. Just like attention, you attend to one thing I'm attending to one of you and then I shift my attention to somebody else and I have to understand how that happens in terms of these underlying collisions in my head that briefly coalesce for 100 to 200 milliseconds and they give rise when I look at somebody what happens literally in your head you have these neurons at fire they build up a collision and a whole bunch of neurons at fire are probably close together they're probably mainly pyramidal cells in cortex because they're probably mainly excitatory they reinforce each other that sort of makes these collisions sustaining and they inhibit their competitors that represent other concepts that represent for example when I look at somebody that represent the neighboring the neighboring percept and that's sort of inhibited and these can be biased both by bottom-up cues as we are mentioning by salient cues if you wear something very salient that's going to make it easier to attend than if you wear something very dark and they can also be biased by top-down cues like I'm looking for a friend or I'm looking for somebody who's wearing red so this just epitomizes the problem once again so if you're looking at a scene like this so let's say in your V1 you have a small receptive field in V2 it gets bigger, V4 it gets bigger and by the time you get to IT in for temporal cortex a single receptive field can encompass my entire hemifield or often they can also cross and so now if you think about if I'm just looking here to the this person's looking at my left ear so if there's a neuron that's tuned for the left ear and it just essentially incorporates the left ear then it's going to fire but what about now a neuron in V2 or V4 in IT by the time here there's the optimum stimulus but there are all these other stimuli so if there's somehow a mechanism that allows the neuron for example to shrink its receptive field just to find response to its optimal stimuli or if it gets biased because it says well I'm looking for something that looks a little bit like an ear but this is sort of one of the key problems why you need attention the other is many people so this is sort of a summary of many people conception of the function of attention one is this is particular due to antrisman you need attention to dynamically bind features that are not expressed at the single neuron level that relates to the binding problem when you have to combine in order to detect the target you have to combine features that are not present at the single cell level like green and horizontal and you don't have a green and horizontal cell that's what you need attention for or then to resolve this competition among multiple objects with overlapping representation but that's not the problem if you have a single object in an otherwise totally empty visual field then you don't need attention so some of the best evidence for our studies our first talk about single cells and a little bit of fMRI comes from somebody called Desimone Bob Desimone who was a large group at the NIH studying monkey vision here's the old classic paper from Science 1985 where they recorded an area before and an area IT I think this was before and they have two stimuli so these are both inside the receptor field so this sort of the inside box is the size of the receptor field and they're two stimuli there's either a horizontal bar that's a preferred stimulus of this neuron and a vertical bar that's a poor sorry the vertical bar is the preferred stimulus for the cell and the horizontal bar the cell only responds poorly and so here what you see now you're both inside the receptor field and you force a monkey to attend to the preferred stimulus so the cell imagine you're in this scenario you're now looking let's say v4 cell and there's both a horizontal and a vertical stimulus inside that receptor field and if the monkey is attending to the preferred stimulus of the cell the cell will respond strongly but if the monkey is attending to the sort of poor stimulus for this particular neuron the cell fires much less strongly in both cases the initial strong initial response but then it fires much much weaker so if this is if conception you can think a little bit it's like the receptor field of the cell shrinks around the attended stimulus of course you can ask the question how do you define the receptor field how come why can't you just say well the receptor field is just this big well because first of all in the if the monkey isn't attending then the receptor field is much bigger the monkey isn't attending to anything in particular the receptor field is much bigger and you can put the preferred stimulus over here over there over there over there so all of these parts of the receptor field the monkey will the cell will respond to the stimulus it's just that if you add attention and somehow it biases the network that de facto the cell will only respond this particular cell will only respond to if the monkey happens to attend to its preferred stimulus and they call this bias competition this sort of the framework it's not really a theory it's sort of a framework and the years of following that you have let's say a whole bunch of cells in de-force that call let's say for vertical and you have a whole bunch of cells that call for for horizontal and when the monkey is told to attend let's say to horizontal this is something the monkey has to do a task okay it's in a short-term memory that somewhere in prefrontal cortex that's where the instruction for the experiments is you know just like you if we ask you to come down to the lab and do an experiment you're going to push buttons but someone you had you have to keep the instructions so they're in prefrontal cortex and the monkey knows it has to attend to to the vertical stimulus and so then there's this bias signal that's generated somewhere probably in prefrontal that gets sent back that somehow manages to bias to give a boost to the cell to the cell with this particular orientation to all the vertical cells how that happens we really don't know so now imagine so now you have these two populations in the absence of any other bias they're roughly balanced they're roughly as many as the vertical for horizontal to first approximation and now by this top down stimulus you're biased and okay so now let's look at these conditions so here we have this stimulus just by itself and the cell by itself really fires very strongly to it so by definition it's a good stimulus this stimulus by itself the cell hardly fires at all it's not a it's a poor stimulus okay so now I add I have these scenarios these three where there are all three stimuli both stimuli inside the receptor field if the monkey attends somewhere else somewhere outside then essentially you're going to get some sort of linear superposition between this response and this response i.e. some sort of linear position between this and this which is somewhere here in the middle it's just sort of you know you have two stimulus now the the neural networks sort of fire it out among each other and they settle down somewhere in the middle but now if you buy if the monkey is biased to look because it's been told to tend to its good stimulus those neurons fire much strongly and so now the response to these stimulus is very close to the isolated case again so you can think it's like a little bit like this receptor field shrinks around this this particular stimulus conversely if the monkey is being told okay now you have to attend to this stimulus which this cell doesn't respond very well the neuron responds much less why? well again remember you have these two populations of neurons these are let's say the vertical one these are the horizontal ones and now your brain your prefrontal cortex tells you to look for this so this is biased so this stimuli now these neuron populations fire somewhat stronger it could be very subtle bias but they somehow fire stronger so in this competition now they win but now the monkey is being told to attend to this one so this one is now stronger and suppresses this one and therefore the response of the vertical cell to attending to the horizontal stimulus is very small so at least it's a conceptual framework it's not a quantitative theory but it's this bias competition it's a conceptual framework how at every stage of these very complicated neural networks you have a whole bunch of biases they can arise from bottom up and top down here they come mainly from top down because stimuli by themselves are highly visible there's two isolated bars and anti-background so they're both very visible at this top down bias and so they can influence the firing now you must imagine this happens at multiple stages this happens not only before but evidence seems to suggest single cell evidence and physiology at least back as early as v2 and probably also in v1 although there there's some discrepancy between the monkey physiology and the human physiology pretty much most stages of the brain excluding the retina there's no evidence for attention modulation in the retina itself there's basically no feedback back into the retina in mammals and so that you have this sort of competition at multiple stages of course after you cascade the three or four or five times at the end the idea that this stimulus will not at all be represented if it's not intended while the attendant one will evoke a very strong firing and there's still lots of things that are being sort of it's a very active area of research so for instance what happens if you have a single stimulus as I just said if a single stimulus there is no competition if you have a single stimulus whether you attend or not attend you might argue there is no big effect on a single stimulus now that's not quite true some people at least this is from the lab of monzel they find sort of some multiplicative effect that the neuron will respond with an increase in gain to an isolated stimulus if you attend to it as compared to when you attend away from it let's see on the other side of the visual field this might be the response the firing rate if you have sort of different oriented bars of motion in different direction this might be the response without attention and this is the response with attention so some slight enhancement so in other words there's not only a competition but also there might be a generalized gain control that the more you attend the more you turn up the gain of your neurons corresponding to the attended stimulus it makes sense so some people have argued that essentially what they started shifts it's akin to making a stimulus more more brighter increasing the firing rate of a neuron increasing it gain for the next neuron it looks like for the next neuron it looks like the input just became brighter when you attend to stimuli the stimulus you attend to are sort of more a more salient, a more brighter than the stimulus you don't attend to now I don't think, when I last talked to him I don't think anybody's directly tested that I don't know if you know this idea of them, of Monserland and Bob this that changed the contrast available but has he tested psychophysical in humans so he's done this in monkeys but I don't think he's any specific predictions here they have a very concrete prediction based on this physiology and for the postsynaptic neuron you can argue this is like the contrast of that particular stimulus increased but it doesn't really it doesn't really look brighter yes, I see so you're seeing almost by definition so you're seeing as threshold it's almost like that because when you don't see it you don't see it when you see it it must have gotten brighter there's also, as I mentioned to you so attention is a multifarious it's a complex creature and it corresponds to the fact that neuronal competition exists at many if not most ages of the brain and so it manifests itself in different forms you can attend to a particular location in space I can choose to look over here and attend to just this location you can attend to objects so you can attend to let's say my entire torso or something or you can also attend to particular features things like color or motion that I'll attend I know I'm looking for my daughter and she wears red so suddenly red will look brighter that red somehow stands up and there's some nice psychophysics evidence and some single cell evidence and some of the nicest one is the reason I'm working here of Melissa so I wanted to talk about that and to give you the feeling for an fMRI experiment in attention so now we're going to switch from single cells to fMRI so this is Melissa's PhD that she did at the SOC so this is a human brain and here you can see I guess the left side you can see some of the topographic areas V1, V2, V3, V3A and MT MT+, it's just another name for the motion complex and in monkeys it's called MT these are all topographic areas and you can map them out using a technique that I guess was invented by Stephen Engels and Gary Glover and these people at Stanford, Jeff Boynton these people at Stanford and essentially the different ways you can do it so you can for example take a wedge like this and then you can slowly rotate this with it activates neurons and then you can slowly rotate it it's called the rotating wedge or this is something similar where you have a ring and the ring expands and then goes starts again at the center and expands and essentially what you're doing now so if you're doing this very slowly you're activating neurons remember the topographic representation sort of if I flatten out V1 I have this sort of a and then here I have the phobia and then sort of I have this logarithmic mapping in eccentricity with constant lines of angle and so essentially you can essentially look at that you can directly visualize it by doing this very slowly you can see how you can march if you expand slowly you can march across the retina V1 and you can essentially track the phase relationship between the stimulus in real space and the phase of the advancing hemodynamic activity that's essentially what you're doing and then you can also establish there's V1 and then you can by switching those things you can see where the borders between V1 and the next map V2 and V3 so there's a standard technique that's now routinely used to imagine in the living brain directly not only the fact that those neurons respond to vision but there's a topographic representation now that doesn't work for these high level areas like V4 particular IT because there is almost no more topography it's very like I said they have this very big receptor field and it's very difficult to predict so in V1 if you move a couple of hundred microns across a visual cortex in monkey you're going to do some sort of systematic trajectory in real space by the time you get to IT that's not the case anymore so that's why these topographic techniques fail in high level areas but they reveal I don't know the current state of the art is probably like a dozen areas or something or ten areas that's also topographic isn't it but that's it for topographic areas so LO is not topographic and MST I guess wouldn't be probably not anymore okay there's LIP I don't know frontal eye field that's topographic so and these of course overall very similar I mean these areas were first defined with the possible exception of V1, V2 of course these were defined physiologically first in monkey and so there are interesting debates now going about the homolog between monkey and V1 particular for some of the more high level areas now let me see would I get this together her experiment so she looked at the fact this feature based attention so the claim is that when you're attending let's see to me because I'm moving to the from your left to your right then all other stimuli that move from left to right will also tend you'll also tend to pay attention to them or they'll stand out as compared to stimuli that move in the opposite direction and not only attending to me in space but you're attending let's say to my if you're attending to my motion all other motion that moves in that direction will be facilitated so the way she did that you fix it here and then you have these circular apertures and their dots that move here they only move in one direction while here they move in both directions they sort of inter-digitate clouds of dots that move up and then move down at the same time so it's a little bit like I mean the stimuli sort of derived from these motion for you and stimuli like you had to do in the homework like Newton and other people used and here there's some sort of cue that tells you at fixation that tells you do you attend to the downward moving dots or do you attend to the upward moving dots okay but physically you always have upward and downward here so let's say for the sake of I mean this is downward okay and you're attending downward motion here over here and you have to do some sort of discrimination the speed discrimination to make sure you know because you want to make sure that people are actually attending there so let's say this is downward well so now you can compare the bold response when this was downward when this was in other words the same direction as when you attend here as compared when here you are told to attend to upward motion okay you can compare the response bold to when this direction is the same as the direction that you were told to attend to versus when it's different from the direction you were told to attend to okay so here nothing changes physically nothing changes at all about the stimulus in either case it's just the instructions here that differ you attend to upward and downward and so you can compare the bold response to upward and downward motion and so she has these long stimulus period again so this is now you know we're talking about a total different realm compared to monkey physiology right so everything is very slow unfortunately just that's because of the nature of the sluggish nature of the hemodynamics and that's of course troubling because you're looking at different things than action potential but that's on the other you can do this in living normal human beings without doing without doing you know training in monkey photography anyhow the 20 milliseconds the person just attends here to up and 20 seconds down 20 seconds up etc and then here Melissa now plots sort of averaging and doing a lot of signal processing that I'm not telling you about she plots sort of essentially the bold signal so the bold signal bold stands for blood oxygenation level dependent contrast changes so it's the dominant it's not the only but it's the dominant mass signal that people extract from magnetic resonance from functional magnetic resonance imaging the different signals you can look at and this is the dominant one and it reflects a conglomerate of different things of changes in the blood how much blood rushes in how much of that is oxygenated and at what speed does it rush and also the arteries can dilate they can become smaller and larger as a function of demand all of that is confounded in there but essentially exploiting the fact that in your head you have this natural contrast agent namely your blood and the blood with and without oxygen oxygenated blood has slightly different hematomagnetic properties it has different optical properties also right if you cut yourself it's different if you cut an artery in a vein it's smaller than the other one like one is a slightly different diamagnetic than the other one and now these are small signals you can see here that the signal changes 0.1% here at peak so these are very small signals so that's why if you operate in a magnet you have to try to remove all artifacts you want to carefully monitor breathing and heart rate you want to monitor movements so very often you get a bite power something put the person in cement it doesn't work so well and now what you can see here this is the white periods are now the 20 seconds white periods here is when the person so this is the bold response to motion over here the person always attended over here but the white one is when the person attended here the same direction as this direction of motion we argued it was downward so when the person here attended downward and when the person here attended upward then you get the gray so you can clearly see the response over here is stronger when the person attended to the same direction of motion that's the important implication now I mean you can read it out from the data one implication is that this should actually stand out that this stimulus she didn't test this at least yet right that this stimulus will actually psychophysically stand out as compared to that threshold as compared to when you're attending to the other direction of motion now this is not only a peculiarity of color it's of motions also for color so here's the same thing for color where you look in a different area that we think is involved that people believe for various reasons certainly has color nuance and seems to be selective for wavelength information the same story is true for color if you modulate color here you have green and red and so you ask you know you plot all the the periods of the white it's a period when the person over here attended to red which is the same as that and the gray is when the person attended to green and this shows a response the bold response to this part of the visual to this stimulus so you can see clearly when the person attended to red then the response here is enhanced as compared to when the person attended to green and this enhancement is actually quite large enhancement can be let's see here the enhancement can be actually for example for motion in area MT and for color in area V4 well here the enhancement can be almost half as big as the sensory signal itself so the enhancement can be quite significant so this is just to give you a flavor that you can visualize the effect of attention now not only at the single cell level in monkey but also directly in humans and what you see when you attend to certain things sort of what you might naively expect that the neuron representation for that are the if we are more careful the hemodynamic representation of that in the brain becomes enhanced for that particular signal in fact there was just a paper that Patrick just sent me that sort of it goes even further and shows that you can show remember we talked about the spot light of attention the metaphor for spot light that when I you know that my attention acts like a spot light it's dark and where the spot light is lights up so there are people now using functional imaging will say you can actually see the spot light and you can also see it change it as you zoom in and concentrate sort of more called computational resources at a smaller spot this is a larger bold response compared when I zoom out and attend to this all of this part of the visual field when I get a large footprint a bold footprint but the overall amplitude of that footprint is much less sounds almost to good to be true okay let me finish here that's going to be short lecture today I'm very I'm satisfied so I'll be brief here where is the because we don't really know the exact at the neural level where's the source of these attention biasing signals and the source are many and so here we run up against the problem that the brain is a highly networked system which we often tend to forget in our enthusiasm we say well there's this motion area color area here and the eye movement area over here and over here is the area that responds to scary faces we tend to forget that partly we view the brain that way because yes we can show that this area responds strongly to scary faces and to happy faces but that does not mean that does not mean that that's the only place in the brain where scary faces are processed or happy faces are processed it just means the neurons there respond a little bit better to scary faces and to happy faces and so one always has to keep that in mind it's a you know normal human tendency we want to sort of have boxes and have labels there but in fact a lot of this processing is done in many many different parts of the brain so you see that when you look in attention bias the attention biases that originate here in the frontal eye field or in parts of prefrontal cortex those are probably related to the task I told you before like attend to color attend to faces things like that then there are also low level cues that originate from parts of the brain the palvinar and posterior parietal cortex and so everybody has a favorite their favorite area that they study and we really don't there really hasn't been a convergence yet to something much more specific than that the last thing I wanted to spend a few minutes on is a deficit in attention there's a specific what do you call it? it's not a disease a pathology, there's a specific pathology of attention at least I think about it as a pathology of attention which is visual neglect which is the correct term I guess is sensory hemi-neglect what do we mean by that? well let's see the typical patient, neglect patient would be somebody whose wife brings her husband into the neurologist because the husband himself doesn't come and he says I don't know what my wife is complaining I can see everywhere and I can see fine but then she relates to the doctor how every time he parks a car in the garage now he hits it on the left and he seems to sort of run into things on the left side and you can look at these descriptions sometimes these patients don't eat from the left side of the tray until you turn the tray around or they'll go into the female bathroom because it says woman and they don't read the first W.O. they just see man and there are all sorts of sometimes bizarre sometimes amusing episodes that tell you that there's something wrong with these people but it's very pernicious I mentioned this in one of my first lectures it's not like they see nothing there it's that they don't intend it's a little bit difficult for me to imagine how the world looks to a patient like that so if you had a patient if you had a lesion let's say in V1 let's say I lack my right V1 then basically this will be sort of it'll be like this essentially there'll be nothing here perceptually it'll look like this on the left side blind side but certainly from a phenomenologically point of view it'll just look black here I guess no it won't look like anything it's just like the back of my head but I know that and so I can perfectly well you know if I've done this one for a couple of days to amuse myself I ran around with a patch it's a little bit disconcerting but you get used to it and you know you constantly do this it's not really such a big problem because you do have two eyes fortunately you have two eyes I think it's probably a big reason why you have two eyes rather than stereo now neglect people don't have that neglect people they claim they see and they don't have any sort of so there are different ways now there are different varieties so in some people actually if you draw their attention to this demo they can actually see it so they have a variety called extinction which might actually be a separate syndrome but there's nothing, let's see there's an empty visual screen and you put a the doctor puts a finger there the person can see it and then now if the doctor will do this then this will disappear what happens now there's competition between the stimulus and this stimulus will sort of will extinguish this this percept so there are different ways of course you would expect because these lesions are in different parts of the brain it's their own unique history one way you test it for example a standard clinical way you do a clinical bisection test you have a bunch of random lines and you ask people to bisect it now a normal person will do this here right now a neglect person might do this I don't know what he'll do there so the idea is to us this looks equal distance to a neglect person this will look the same distance that's one way for example now we know it's not the original description of clinical of neglect going back to the 19th century thought this was something in retinal coordinates this was a lesion in retinal coordinates oh let me come to that so if I lose my right B1 then I lose the left visual field this is a bit more complicated this is not in retinal coordinates it's where you attend to because you can show people have done this with drawings I don't have it here where they had houses and then they had let's say a house on fire you can for example perfectly well attend to this then you'll be perceptive people will neglect this or you can attend just to the right half of this and then people will perceptively neglect this and then they neglect everything on the left that's why people now think of it as an attention pathology that somehow you've lost access to attention lost access to those parts of the brain that's represented let's say to the left or to the right of attention most of these lesions occur in a structure called the interpridal lobe although this is somewhat controversial now there's a proposal by that involves a slightly different structure but for the most part the typical patient has a stroke on the left in the left posterior parietal left inferior parietal lobe and so no wait sorry it's right it's in the right because on the left you have vernicus area on the right the typical patient has a lesion on the right interpridal lobe that's exactly as I showed it there on the left you usually don't have this strong as you don't usually get neglect so there's an interesting asymmetry there between left and right that's typical it's not it can be large degree of variability and as I said the exact size still remains controversial which is surprising because neglect is a relatively common phenomena people have made for example this classical experiment by Bisniach in Italy where you had people neglect patients had them in mental space ask okay there's this piazza and there's the cathedral of where were the Milana I guess right and ask them to imagine you're standing on the steps of the cathedral the church is behind you you're looking out tell me what's on the left or on your right and then people have this is all in mental space they have an inability to do that for the left they can only describe things on the right and now you picture yourself you're going down the piazza and you turn around so now you're looking at the church and now describe what you see and now people were blind to the blind to the thing they could previously imagine the right side but now they could only describe the left side that were previously blind that were previously neglected so you can see so this is all mental in both cases the people were just asked to imagine this so you can see it really has nothing to do with visual input it's all about representation it has to do with the internal reference system and where do you attempt to relative this to this internal reference system to the left or to the right John Driver's done some of the nicest experiments there that's visual neglect it's a very interesting it's a very common affliction very often fortunately it's transient one it'll go away or it'll become less it'll become less severe but it is quite a common phenomena and then like say you have we probably in English neglect and extinction so like I said in neglect you would totally neglect this even though there might be nothing else in my visual field in extinction if the visual field is empty I can see this it's only if I put a second stimulus over here on the other side that's sort of very vivid that I can see this and then this stimulus essentially disappears yes it's now people again there is nice evidence there is interesting unconscious processing going on in the neglected regions and of course this will depend on patient to patient but you can show there's one very striking study also from Italy where they showed very strong priming effects remember I mentioned priming at the beginning of lecture the priming is when you when there's a stimulus that you don't see you can get auditory priming or visual priming you can get negative or positive priming we usually talk when people in priming usually it's positive priming it's a bit like in what's it called that advertising subliminal perception I mean that subliminal perception is one instance of priming although it's very weak and all those urban legends that have grown around it you know all those myths with coke and Pepsi they're just not true because usually you can measure priming in the lab but it's rather weak it's not like you know if a flash one frame of coke I'm gonna go out coke coke coke it's just that if I go out in the lobby and there happens to be a coke and a Pepsi in there equally space and I might be 5% more likely to go to a coke machine than to a Pepsi machine assuming all else is equal and I'm totally unbiased versus Pepsi and coke so that's sort of the nature of these priming effects we're talking about they're quite weak and in sort of this hysteria that came in the 60s I mean there are people trying to propose legislation to try to outlaw it so here what you get for example in these are of course patients these are not normal people so let's say this was done with those images and I remember the colorful images they were like animals or vegetables or flowers and so they put animals they were put animals or vegetables in there so these are people who had neglect they put these things in the blind in the right stroke so they put them in the left visual field they couldn't see them because they're blind sorry they were not blind they neglected them now I flash on let's see an animal or vegetable or a flower and I have to press a button just say as quickly as I can flower with vegetable versus animal now if I put if there's a vegetable in my neglected hemiphoel I'm much quicker at saying there's a vegetable here than if I put an animal there you can show that reliable okay so once again let's see my reaction time if I just have a flash of a vegetable I don't remember the numbers but let's say it's 500 milliseconds then I put up a vegetable in your in the hemisphere that these people neglect so they don't see it at all if you ask them they just don't see it now they only take 450 milliseconds or 460 milliseconds it's a very significant effect as compared when I have a let's say not a flower but it was flower versus vegetable if I have a flower versus vegetable here so in other words something in their brain had to register that information that information was not made consciously accessible but it was sufficient to enable me to respond significantly faster so that's a typical positive timing effect positive because you know it reduces it enhances processing and this is a I mean this is a well controlled effect it's strong it's there then also fmi there was a study by Geraint Ries who did this when he showed an extinction patient so the nice thing was extinction patient they give you a natural control which you can do now on fmi is the following you put a stimulus here in the in the legional in the field so my reason is it's typically the right post-aparite lobe so let's say I have extinction here so if I put this stimulus here I can see it and you can measure response I put this stimulus here I can see I get a response I put both stimulus here now I don't see this anymore because like I said this it sort of will perceptively suppress this will extinguish it however you can still show this there's still a response to this perceptual suppress this perceptual invisible stimulus in my right fusiform gyrus which is intact what is damaged in this patient is the post-aparite cortex but not areas around the infotemple infotemple cortex so you can show this response albeit a weaker response is still present and that would probably explain some of those priming effects now one of the least known but most interesting of all patients I find because it's a patient with a beneficial lesion this happens very rare usually if you get a lesion you're in bad shape so this person was in Switzerland was a swiss guy and had a classical lesion that's bad he had a lesion in his right post-aparite lobe and was admitted to hospital had all the symptoms of you have all sorts of other symptoms like confusion et cetera but particularly had classical neglect syndromes then in the clinic he had a second minor stroke in his left side but close to in the frontal cortex close to brokka he had some aphasia he had some language difficulties from which he had recovered what's remarkable is that his neglect symptoms just totally disappeared I mean that's it so overall he was worse off because now he also had this language deficit so it's not something one should do on a regular basis but in this and there's some precedent in the animal literature for that something known as a sprake effect what does this tell us I think it's exceedingly important I'm amazed that this patient is not sort of discussed everywhere most textbooks often are talking about neglect this patient that tells you like I said there's some animal precedent for this some precedent in the animal literature that the information that the suppressed information neglect information is probably still around somewhere in the brain it's just not made accessible you essentially have think of this saliency map remember this idea of a map that represents the saliency of the world well and that's based on inhibition that if you're over here this is much more salient then attention will be drawn over to this location so the idea is here that you have competition but you can see that in extinction this by itself is perfectly salient but now you have a second stimulus here this draws all the attention and although this physically might still have the same representation in the brain or some of the same representation some of that information might still be present as we can see in this patient as we can see through the primary literature that information is not made accessible anymore because you cannot attend for some reason you cannot attend to those parts of the brain anymore but the information is essentially still present so people sort of think today about these sorts of ideas starting to make a formal models of this with respect to and to neglect what's interesting for us that we care about consciousness is this question and this answer so in neglect you usually only have a stoke in one interpital lobe usually the right now what happens if you lack both now that's very rare attention here here this is very rare this is a form of pathology that was first described by I think a Hungarian doctor called Balint where you have a number of symptoms but you have sticky vision so it's very difficult for you to move your eyes from one location to the other and you only see the things that are inside the spotlight of attention typically you will only see one or two things and typically when you look these patients are relatively rare but they are described in the literature they usually have bilateral destruction of parts of most of the pastia parietal cortex and so it's really like this is really like the tunnel I was arguing last week that even though you can perfectly well attend to something it's not that the rest of the world disappears but in these people if you look at their description it seems to be what goes on that they can only attend to one thing at a time and everything else just isn't there it's probably just like the back of my head it's just not there and for them it's difficult to move because they have this sticky vision it's difficult to move they move their eyes but once they attend they can again only see one of a few things at the location where they are attending now what this tells us that the pastia parietal cortex is necessary because they don't know where things are they are lost in this so the entire consciousness is reduced to this one thing and they don't know whether it was to the left or to the right of the last fixation they've lost all reference marks with respect to space so they are in pretty bad shape and this fortunately happens very rare but they can certainly see they can certainly still see the color and they can see texture and they can recognize things when they attend to it so what it seems to suggest is that the pastia parietal cortex is necessary for attention, for doing where you are in the visual field and where you are going and reaching out and moving your eyes you can do that with pastia parietal cortex but I don't think it suggests that you really need that in order to be conscious of anything that part of passing probably is done in the in the ventral path in the inferior temporal cortex remember we talked about the two pathways one going from V1 to infotemporal cortex it's called the ventral or the Watt pathway or vision for perception pathway and then the other pathway that originates in V1 and goes along here to pastia parietal it's called the dorsal or the Watt pathway or the vision for action pathway that this you need for action in order to move, in order to attend but you don't need it for conscious vision really that's what I conclude from these patients okay so on Friday we'll talk about experiments that directly look at the neural correlate of consciousness using a rivalry and flash depression and other bistable illusions