 Okay, so with Nau's help we got this very powerful after effect, right? Because last time some of you had trouble seeing the after effect. You probably didn't believe me. Now let's see whether this one works. Lights off. So what you should do is fixate the cars in the middle for 10, 15 seconds. And when Elvis appears, you see him? See, isn't it striking? This is gray. We can do it again. When I first saw this many years ago, I said, I don't understand. Where is the illusion? It's colors. Well, in fact, there were no colors. So this is the original. Now you also may want to notice, remember, red, blue, green, yellow. Just remember that. Red, blue, green, yellow. And you notice it waxes and wanes. But then it comes back, yeah. So in fact you can do it so straight. I mean, what you can now do, you can now go back and forth. And at some point, yeah, now it is because at some point you can arrange it. I'm not sure I can do it here, that you cannot tell after a while. I mean, you know, of course, which is real now. But after, sorry, after a while, the effect becomes so compelling that it really looked that the colors are as vibrant. Yeah. I mean, isn't this amazing? But in fact, there's just no colors there. Okay. Yeah, I agree. Well, we can look at that again. It's a little bit strange. Yeah. Yeah, I know. So does everybody see the after effect on the right? It's more or less the same? Yeah. Yeah. Yeah. Yeah. Yeah, almost. Well, if they're not here in the first place, they're not very distinguishable. I think that's the main problem. Yeah, so you can do games and you can sort of play. We used to do, well, we do that, I guess, in the vision class. You can do all sorts of interesting things with these after effects. Anyhow, I mean, you can use them, although I don't know anybody has used color after effects to look at, you know, neurons that respond, that express the NCC for color, right? You can ask where the neurons that respond to color that express the after image that's physically not there, but the neurons still find. So they still convey the sense of seeing, you know, a hue or pastel color. Where are those? Nobody's done that yet for color. In general, for reasons I don't fully understand, probably just having to do with the organization of the brain, there's less of a... It's more difficult to find specific loci in the brain that respond to color. It's very easy to do for motion as we'll see today. And seems to be fairly easy for stereo, but not as easy for color. Okay. So we'll cram. There's no lecture. Tomorrow I'm going to Stanford. And so there's no lecture on Friday, so we'll cram two lectures into today and save ourselves a lecture on Friday. Okay, so we'll talk first about some principles of cortical architecture. If you want to understand the brain and consciousness, you have to understand cortex. Cortex is, after all, where most of the specific action happens. In other words, we think... Well, there's a lot of evidence to suggest that all the specific representation of color or pain or hearing or words and all the specific action and specific memory representation are laid down in cortical structures. And in general, this doesn't have to be true, but it seems to be true in biology. In fact, it's probably true for any highly evolved system, including some modern computer architectures. They were structure and function are intimately related. So these, if you want to understand a function, you should look at an understand structure, try to differentiate structure. There's this relationship between the two. It doesn't have to be, right? You can assume that there's some sort of universal Turing machine architecture, and it's everywhere exactly the same. There is no difference in the structure because it's a universal architecture and it performs everywhere the same function. But biology doesn't work like that. And it probably has to do with the fact that it's a highly evolved system. It seems to have lots of highly specialized structure that analyze specific functions. Now that's also beginning to be true to a certain extent if you look at a CPU, right? You've got a bus, you've got memory, you've got the CPU itself, you've got cache one, you've got cache two, you've got the ALU. So you're also beginning, and the more highly designed the machine is, the more optimized it is for in terms of power consumption, et cetera, the more you can see these structure Many people have, many clinicians, this is started off at the turn of last century, many clinicians but later on pure neuroscientists have tried to sort of to make maps of the human cerebral cortex. The various version of these maps in existence, the one that's proven to be historically the most influential is this dude to Brodmann. Brodmann was a German who worked in between the war, before the war, around the Great War, 1910, 1920. He worked in Berlin. He worked mainly in humans but also in monkeys and I think in other animals. And based on what he had available at his time was the microscope and some simple stains like particularly the one of Golgi and some myelin stains. So they could also look at myelin. Remember myelin is sort of the installation sheet of axons and so the white matter is white. It's mainly myelin, what you see there and it serves to speed up the propagation of action potentials. And so using, by today's measure, a relatively simple technique that was called cytoarchitectonics. Cytosell and architectonics are obvious. So they try to differentiate cortical tissue based on the organization of cells, what cells are present, subtle changes in texture based on the types of cells that are present. And again, the belief driving these people was the fact that their differences in cellular architecture probably reflect differences in function. And by and large, that's been proven to be prophetic, that's been proven to be true. So broadman, as I said, is the most influential. It's still used in most textbooks today. You hear people say broadman area 17. For example, that's primary visual cortex or broadman area four. That's sort of part of motor cortex, or broadman area 44, that's part of broker area, the speech area. What he did, he, now there's no simple principle why they're called 1789, 1920. Essentially he ordered them in the way he studied them. They're also non-contiguous. There are four, I think there's a gap. There are four numbers that are missing for various historical reasons, people have wondered about. Well, they're on the out of 50 areas. I think it goes up to 52, but like I said, four or five numbers are missing. Now, so you can see the front, the back. Here's primary visual cortex, area 17. This is the calcium fissure. Remember, this is sort of, if you unfold it, this is the credit card at the back of your head in the medial wall. And outside of area 17 is extracellular area called area 18 and 19. Now, you should not be misled into, Joe Borg makes a point in his class about this. One should not be misled by looking at this. This is, if you look at these borders, they're not geometric borders. They're not incredible fine. So if you look at maybe at the, they might exist at the level of one or two millimeter, but if you go down further, you can see invagination, you can see transitional territory if they're not very clean. So there are a few clean borders. One is between 17 and 18. And there's also relatively clean border at area four, which is sort of motor cortex, pre and post-central. But most of the other areas are anything but clean. And just like, you know, if you look at maps of most countries, it's a remarkable exception of the U.S., but certainly, folks, if you look at the Middle East, you look at Europe over the last 100 years, you see sort of countries fuse and fission. And you also see that today, not so much fusion, but mainly fission. You see a lot of fission here. In the sense that people with modern methods recognize that an area that used to be sort of homogeneous, because we didn't have any better tools, now turns out to be actually composed of three or four, five, six different areas. That's very common. Some of those areas might sort of might occur across some of these earlier broadman areas. So broadman areas should be used. It's a very, very rough guide. It's not the gospel. Yeah. If people with modern methods of neuroanatomy, they have better dyes, or they have many more dyes. They also have metabolic dyes available. In other words, dyes that stand for metabolic active tissue, or dyes that stand for the presence of various other chemical components, dyes that stand specifically for various proteins. And you might, you know, you might wonder why is all of that relevant. It just turns out that if you use some of these dyes, many of course not, but some of them, sort of they, they, they, they lineate specific compartments in terms of the neurons in those compartments do very specific things. So, probably more recently what was discovered that in the LGN, so that intermediate relay station between the retina and primal visual cortex, the neurons that stand for this one protein, it's called calcium-calamodine depended protein kinase 2. And those neurons seem to specifically involve the mediating, the sense of blue-yellow, and so it's unclear, I mean, nobody knows at this point why this protein is localized within these neurons, but they are, that's a fact. And so A, that tells you, if you look at that, that tells you that any time you find such a difference, it tells you, well, there's probably difference in function. And B, using more than molecular biology techniques, you can then exploit that and sort of design magic bullets, design if you want viral, design other sort of molecular syringes, disposable molecular syringes if you want, that can maybe specifically target those neurons and turn them on and off. And so you can take a normal person or a normal animal and then just turn off all the neurons that are involved in blue-yellow. And so then you can make the jump from correlation, from me observing the system to actually interfering, to perturbing the system to causation. Now, as you'll see in a second, well, I can sort of glimpse, show you a glimpse. Modern methods have revealed lots and lots and lots and lots of different cortical areas. So this is the, it's a very famous pictures from Dan Fallerman, David Van Essen, who used to be here at Caltech for many years. And so this is just the different areas in the visual system of a macaque, which occupies on the order of 35 to 40 percent something like that of all cortical tissue is dedicated to visual perception or visual motor function. In us it's a bit less, it's maybe a third, a quarter, 25 percent, 28 percent. So a lot of vision, a lot of cortex is given over to vision. Most of which, of course, proceeds in a total absence of any conscious interception or intercession, right? As we emphasize many times, you're just conscious of the result. So the part of cortex that analyzed things that you actually consciously represent or that you do like thinking is probably a very small 30 much less than 30 percent. Anyhow, so it starts at the bottom retinal ganglion cells in the retina, and there are these two compartments I mentioned marked on tavus cell neurons corresponding to sort of, and there are lots of other ones that we don't show up here, this is not a complete picture. These are neurons that tend to respond to, they tend to be selected to rapid, to changes to high temporal frequency and low spatial frequency. They don't seem to care a lot about color these neurons seem to care a lot about either high spatial detail and they also care about wavelength selective information. And then this is the LGM, then it goes into V1, and from V1 it goes into all these other areas, V2, V3, Vp, they have all these funny abbreviations, some of them Latin, some of them English, some of them idiosyncratic where you have in England on the continent different nomenclature, you might see that if you read in the homework like this area I'll talk quite a bit about today area MT, middle temporal area, it's very often also called V5 for various idiosyncratic and historic reasons. Up there is hippocampus and entorhinal cortex, that sort of the if you if you draw a map of the areas and you you ask the question where's the most distal area, distal both from the input and from the output, where's the area that's most removed from the input and the area that's most removed from the output you know you'll end up something in hippocampus of course if the architecture doesn't end there it goes all these areas that go on further either down, either they project directly to output structures or they go on further to the front of the brain so you can see the trouble is you've got all these areas, let's see 40 areas in vision so far and people might discover more as they find finer and finer distinction, how can you think of them, how can you arrange them are they randomly connected, is it a random graph, is there some sort of hierarchy is there a circular structure so people discovered anatomists, Kathy Rockland and Panja and some other anatomists discovered that if you look at the detailed patterns of the wiring of the projection system within cortex you'll see that at least in the back part of the brain so the back part of the brain I mean crudely everything that extends from the central sulcus backwards there's this big central sulcus that runs down here and there's a sylvian fissure and that divides the brain into front part into back part, very crudely spoken and the back part is kumgranosala is concerned with perception in all its guises, visual perception auditory perception, somewhat sensory perception well the front part roughly is concerned with action, with motor action and planning etc so in the back part what people discovered that there seems to be that you can classify projections of two types that there seems to be one dominant type that can be of either two forms two versions so either the neurons let me see either the neurons sit in the upper so remember I emphasize it's very important to understand when neurons sit in which layer they're organized now it turns out now you'll see why it's important so there are sort of conventionally in most cortical areas divide cortex into six although you can divide into many more if you want to be a little bit crude about it you can say well there are really three cortical layering, there's the superficial layer there's the intermediate layer, layer four that's usually where the input comes into and then there's deep layer, layer five and six and so if you adopt this type part type division then you can see there's one type of projection that originates in the superficial layer two and three that projects into the next area into its post-synaptic target into the middle into the input layer, layer four sometimes you can have neurons that sit either in the upper layers or in the lower layers that both project into the next stage into layer four and this you can think of a forward connection this is a feed-forward connection you can think of a feed-forward connection because what I'm going to tell you now turns out if you classify all the axonal projections in this way based on where there's cell body and where there's a sort of termination then it turns out context is not a random collection of regions but seems to have a hierarchical structure there's a hierarchy there and then a hierarchy is very important because now you can define, that allows you to define and let's say that area X is above area area W and below area Y now there are lots of exceptions although we are not sure whether those exceptional random or whether they have a function and very often sort of neurons skip a layer projections skip a layer so you might go from you know from layer two from sort of a neurons at stage two directly into stage four so anyway so you can define a feed-forward connections you can also define a feedback connection so they usually sit in lower layers layer five and six and project into another area and they avoid immediately and seem to mainly project into the upper areas these are the X the synaptic terminals so these are the with the accents terminated and sometimes also into the lower layers and that seems to be a back connection that seems to be a feedback feedback connections and then there are some sideways connections this hierarchy this was then formalized by John Manziel when he was here for his PhD and then extended by John Manziel and formalized by these people so they took these rules and applied them to the visual system and then by and large they can define this visual hierarchy it looks a bit like a steam plant well you have you know 12, 14 levels it's somewhat in the term that since for you know if you do graph theoretical you know if you just think of those as graphs and those as nodes and the nodes are directed now because you can define a feed-forward and feedback connections you can of course always define a hierarchy with more levels but if you sort of impose some constraint such as minimal numbers of levels you arrive at something like this which has I don't know 2, 3, 4, 5 6, 7, 8, 9, 10, 11, 12, 13 14 layers something like that the point is not so important with that 12 or 14 the important is that by and large it seems to be a hierarchy where you can define you can say this area is above this area and vice versa and the hope is that this will ultimately cause us to lead to better understanding right now this is right now I think by and large the majority of people accept it a majority of anatomists and physiologists accept that there is a hierarchy although there's still certain elements of it as controversial is it a unique hierarchy there seem to be a number of exceptions are those exceptions are they there by design or is it just you know like are those just random exception and why do you really need such a hierarchy I mean humans like we like to think of hierarchies because most of you know if you look around in organization in our lives and our families and our universities and our armies everywhere have organization and of course it makes management of people much easier it's a little bit less clear why national structures such as this have to have hierarchy people who study such things organizational schemes again argue that it's a nature of any complex hierarchical evolved system that it has to have hierarchies because it makes evolution it makes evolution of complex system possible if there's no hierarchy the argument goes it's very impossible to it's very difficult to evolve truly large and complex systems without having a hierarchy that you can recur onto okay the visual system is a hierarchy now one thing I have to know the hierarchy is not necessarily reflected in activation times what I mean by that's the following if you shine a light into the retina down here then you can look at what we call a net wave you have this wave front of action potential that moves through the system it is not true that you know it moves through the system that this area will always get activity before any area here will always get activity in any area here the analogy is a little bit like an interstitial tide pool right you have this surf and you have the rocks and you have this interstitial pool and sometimes depending on the wave and depending on the depth of the sand and other considerations you get you know one part of the wave advances way over here but it sort of hasn't really come that far in another part of the shore and likewise here cause these neurons are these magnesium neurons and they seem to as I mentioned they like transient information they also signal information much quicker than the power cell neurons so if you just put an electrode in and you flash a light on typically see neurons responding much earlier than these neurons and there seem to be speed ways so for example if you go from this pathway from V1 into area called MT into the FEF frontal eye field which already part in the frontal part of the brain that's a very rapid connection well some other connection like this one from here down to V4 down to infotemporal cortex seems to be a slower one it seems to be more 160 140 milliseconds well you can get to MT into frontal eye field within 60 milliseconds something like that so it does not mean this hierarchy does not mean that you have sort of that it's a clock thing that the input moves here at one sort of one stage per beat of the clock doesn't work like that okay another principle of cortical organization that seems to be true this is mainly for vision the previous hierarchy people have by far more is known about vision than about touch and audition but people have made similar hierarchies now for some other sensory domain and for the auditory domain and the claim is that you like what they have hierarchies it's not something just unique to vision but possibly to all sensory processing with olfaction always playing a special role it's an evolutionary much older system the cortex is much older also and it bypasses the thalamus venom of other differences so olfaction is always always an exception this is in the side how many of you have ever dreamt that you smell in your dreams do you smell okay I'll say it's one claim that because there's no direct primary connection so in vision you have you go from retinal to the thalamus to a cortex also true for some other sensory and for auditory it's not true for olfaction there you first go to olfactic cortex and then you go down to another part of the thalamus and some people have claimed that might be the reason why most people don't dream of smells but it's obviously not true in five of you well I guess so I mean so doing dreams of course doing sleep your eyes are closed there's no input coming from the retina and the entire thalamus is in a different state depends on what it's usually not in a relay state yes so it doesn't pass along sensory information partly because there is no sensory information depending what state of sleep you are it can be in one of these different oscillatory states so I don't know anybody who's done your sort of experiment the one you suggested there's been claim of unconscious or factor discrimination similar to blindside but when I looked at it the data was never terrible convincing okay so in the visual domain it might be something that's unique to vision there's another organising principle this was first discerned based on clinical this goes back to 1910s or 1920s based on clinical damage patterns of clinical damage so neurology really grew tremendously at the turn of the last century especially because of the existence of high speed rifles that penetrated right through the skull and right through the brain and went out the other end and gave rise sometimes to unfortunately to lots of people who had injuries and some of them were rather specific and because the bullet left it didn't stay inside the cortex and ricocheted around and made huge damage sometimes they made very specific damage and so some of the earliest work that we know, modern work that we know about the brain is brain damage and so we know about the patients from the Japanese-Russian war 1906 Sebastopol I think and the First World War of course and so what people they're found that there seem to be distinct patterns of injury relating to specific type of behavioural deficits so people define a ventral and dorsal system so ventral, stomach, vent if you speak French and dorsal you can remember it very easily if you speak French so there seem to be a ventral and dorsal system for the ventral system also sometimes called vision for perception or the what systems starts in primary visual cortex and then goes down here goes down here into passes into before and then infer-temporal cortex posterior infer-temporal cortex caltech and anterior infer-temporal cortex this stands for central infer-temporal cortex there's also somewhere in MIT yeah, medial I think it's just around there anyhow so this is, and from here the information then goes on to lateral prefrontal cortex and this if you have damage here, now this is Cumbgranosalis if you have damage here in humans the typical deficit is that they have for example inability to see faces or their various object agnosia that they're unable, there's nothing wrong with their eyes they're unable to recognize or identify objects or they've lost the ability to recognize things that are independent of distance or independent of size or things like that so this is called the what system or the vision for perception pathway that the vision you need to actually perceive things consciously is impaired to various extent that's the second pathway called the this is the ventralis the second pathway called the dorsal pathway because it goes on the backside of the brain the dorsal from primary visual cortex to posterior parietal cortex in and around the area of the inter-parietal sulcus, the one that Richard Anderson and his colleagues and students study and from then goes also to the lateral prefrontal cortex, so here sort of those two streams reconverge to a certain extent here the typical pattern deficit is neglect that we'll talk about later or that you're unable to, that patients are unable to localize things in space, that they have deficit they focus on they cannot reach anymore there's nothing wrong with the motor system but they're unable to connect you know, to know where things are related to other things and they have you know, difficulty precisely reaching out and grabbing, they'll do consistent you know, they're unable to grab this they make consistent errors and this is called the where system because if you look at in monkeys in this homolog system in monkeys there's something very similar, those neurons seem to care a great deal about where things are about the location of things or sometimes also called perception for action this is the vision you need to actually act in the world, to adjust your optical flow you know, if you move around you have all these subtle choose about motions, you constantly adjust your body all the time or to reach out and grab to make eye movements you get deficits in those sort of systems so that's called the vision vision for action, vision perception pathway alternatively the ventral and dorsal stream okay then lastly I briefly wanted to mention we won't talk about it too much much less is known about it in both humans and in monkeys the front part of the brain which is fairly big it's like 35% or something of the entire cortex this is an amount of a living person and from Hanas Damazis Atlas and you can see the very I mean those you should all know certainly it was part of the homework the various lobes so you have the front lobe in red seen from the various views so the front lobe is essentially delimited everything in front of the central sulcus and down the the sylvian fissure is sort of the front lobes and then the back lobe is of course occipital lobe lobe just means sort of large part of the brain sometimes there's an old fashioned lobe world it's also sometimes called then you have the temporal lobe down here then you have the parietal lobe sort of above here now the frontal lobe themselves are conventionally divided into these three parts and the frontal lobe is really the frontal lobes are really uniquely what makes us humans if you can say that I mean sort of anathema to a biologist in some sense but if you really look at sort of the things like speech that make us unique human or symbolic processing ability mathematics, invention of Macintosh and other things, those are all you crudely two parts of the prefrontal cortex so the great aim of the frontal lobes is action and at various timescales so one part of the frontal lobe the one that's best easiest to define is the motor strip motor cortex runs pretty much on top of here you can identify this yourself if you volunteer for example in transcranial magnetic stimulation experiments like Shinshi motor in this lab is done then you put this coil this magnetic coil above your motor strip and if you have it at the right location and you trigger so you briefly have a current through it generates a brief magnetic pulse that somehow interferes we don't understand the biophysics of it yet but it interferes, it excites and inhibits neurons if you do it above the motor strip for example on this side everything is contralateral of course of course then you might twitch here doesn't really hurt although Patrick tells me if you do it a couple of thousand times you get a big headache what? oh, it's good for you actually it is used as a technique of last resource for chronic depression it's called repetitive TMS I think of it like you reboot your computer so I think that what happens in TMS but it is used and seems to be very effective at that it's just a single pulse and there seems to be no damage as far as we can tell and it excites it's a great technique for interfering with the brain albeit it's very cruel that's a trouble from a neuroscientific point of view you're probably activating a fraction of a centimeter so if you remember the cubic millimeter there are 100,000 cells we might be talking of neurons that you affect here so if you do it over the motor that's the motor representation so there you have most of the output not all but most of the output in layer 5, the output structure in layer 5 the the pyramidal cells that project down that make up the pyramidal pathway that go down to the spinal cord that ultimately make you do things then there is premotor cortex and then there is prefrontal cortex proper which is usually defined by the idea of a specific salamic nucleus so you move from as I said the great aim of the frontal lobes is action so here you move from very specific action a specific set of muscles very rapidly in the next 50 milliseconds as you move forward here this is the back of the brain as you move forward you move to the brain is involved in actions that are more and more distant so if you want to do a long-term plan and see how do I get from here to this building that is a whole sequence that would involve probably prefrontal lobe if I am just getting ready people have done this and if I am thinking about playing soccer if I am thinking of playing of kicking a ball or climbing or something then part of the premotor cortex would be involved these are also as I mentioned the parts of the brain that make us uniquely human so if you have lesions here you get all sorts of deficits and moral reasoning and social judgments inability to come to any sort of decision typical people who pre-varicate who cannot make up their mind when talking about a menu in a restaurant or a life decision usually that sort of thing all involves frontal lobe dysfunctions quite a bit is known at this clinical level about frontal lobe dysfunctions either through strokes that people have in the frontal part of the brain or through benign tumors or malignant tumors those patients usually don't survive for that long to study the benign tumors that require the excision of the part of the frontal lobes and of course you also have Broca's area here area 44 so that for most of us for all right-handed it's going to be in the left and for most left-handed it's also going to be in the left lobe here area 40 it's up there this is historically very interesting it's named Broca after a French neurologist who were taught at the South Petrie in Paris and it was sort of one of the sounding hours of modern neurology or modern neuroscience since he'd observed this patient Broca's patient for many years and came to the conclusion that this person had an aphasia so he was unable he was unable to have fully formed languages he could sort of utter things but he couldn't really speak in any sort of in any sort of no syntax couldn't really form any real words and sentences and there wasn't anything he could understand language of course you can also have difficulty with understanding language and then when he died in the post mortem they found a tumor I think a tumor a stroke I think a tumor a benign tumor I can't remember they found it definitely in that particular part of the brain and then for the first time that sort of that really established sort of a very nice link between I mean hypothesize of course was only one patient but then other people did this and found more patient between a function namely the expression of words and language and a specific location in the brain of course then the next step is that people discovered the motor strip you know in dogs I think where they stimulated the open skull of dogs and could evoke various actions depending on where they stimulated this came all before people know about the specific location of vision that's what Broca's area is rightfully named after Dr. Broca and plays such an important part now even in monkeys by the way or even in chimps you see a homolog of Broca's area it's not that this is a total new area but it seems even it's in monkeys you can find it and in monkeys it has to do with fine motor coordination of hands or lips and possibly vocalization so you can see how it evolved it didn't you know like anything there was a precursor there it's not a sort of Deus Ex Machina suddenly out of nothing popped this language area okay so that's one lecture now I'm going to speed up I won't just kidding and I'll show you half of these slides do you have any questions with the first part okay so area 4 is part of the motor cortex that's relatively distinct because it has you can see the you can see the cells in what are they called in layer 5 yeah and it's called a granular because it doesn't really have a very well developed layer 4 it has a very thinly developed layer 4 so that's part of you can define motor cortex by the absence of a strong layer 4 I mean it sort of makes sense it's main functions output it's main functions not input no so it then there's a transition zone so if you really go to the frontal cortex it has layer 4 so layer 6 bottom area 6 is transitional where you have a poorly developed and it's called disc granular cortex I think there's somewhat and then if you go in front of the brain you have a layer 4 yeah and then of course there's other specialization for example some parts of the frontal lobe including the anterior cingulate have these spindle cells that are unique to us layer 5 in motor cortex has these very big cells that project all the way down to the spinal cord that provide the output the output I mean the story the story that you typically read at first reading of many textbooks there's this total division distinction between post and pre-central gyrus that pre-central gyrus is sort of everything motor and post area 1, 2, 3 is some other sensory that's not quite true you also have cells in layer 5 that in some other sensory cortex and that go down it's not in all or none but statistically the vast majority of the output is in area 4 the palvina well so you can so much less is known so the the colliculus receives input from the eye and in many animals before mammals and other animals outside mammals it seems to be the dominant or one very very big visual centers in us it's less dominant now because we evolved all the superstructure on top of it and it projects to the cord it does project to directly through cortex but there's no direct projection it does get a direct feedback from cortex and it's involved it seems to get input from parts of the brain that are involved in motion probably because it's big job and nothing seems to be in moving our eyes palvina so the thalamus is a more complicated story that it's a big mystery right now so thalamus is like a quail egg and only the one that people know best is the LGN the recipient of the output of the eye and in turn projects to prime visual cortex it turns out to be one of the smaller visual nuclei there are also other parts of the other visual parts of the thalamus called the palvina and there are four or five or six different palvina independent maps they project to different cortical regions of the visual so I don't want to exit this file they project well they do project to some of these areas so LGN projects almost uniquely into to be one some of the other palvina areas also have very strong relationship with one some of these individual areas it's a big mystery we don't really know what the thalamus does apart from the LGN the other big mystery is that all of the thalamus nuclei envision in all the other modality get massive feedback so I counter this once in a cat and turns out there are ten times more fibers that go back from the visual brain from the visual cortex to the LGN fibers from LGN to visual cortex so it's like you have a camera and you have a CPU and you have a coax cable going from the camera to the CPU that makes sense and then you have a coax cable that's three times bigger that goes back from the CPU to the camera why does anybody guess people have made lots of hypothesis including myself about attention and gain modulation and probably some of these are along the right lines we just don't know it's a mystery to get back to your question we don't really understand the relationship between the various maps in the thalamus the visual maps in the thalamus and the visual maps in cortex proper there isn't too much there's some work done on it but not too much probably because we really don't understand yet what makes those things tick and what is the specific stimuli that excites a palvinar it probably has to do with attention that's mine many other peoples guess it other thalamus nucleus probably have to do with regulation of flow through that structure this shows your map comparison between shuman and monkey can anybody tell me what is what which one one of them is shuman and one of them is monkey which one you're an expert doesn't count the one on the right is shuman the one monkey so what was done here you take the brain which you put people in a magnet and you do various techniques that we're going to talk about in vision class but not in this class you have techniques that essentially allow you to map the visual environment in a systematic manner onto the brain and then you can image that using functional imaging where you track the hemodynamic activity and then you can use graphic tools to map this complicated 3D structure onto a flat structure that in the various ways and there's one way that now seems to be standard so you have to think again where the aesthetics is that's the bottom of the calcium fissure and remember also for homework you had to do that oval that was v1 so that sort of the calcium fissure is run along here really need a laser pointer and think of the calcium fissure the depth of the canyon being cut and then sort of splayed that's the structure you have here if you can imagine that if you remember from the homework or from class you had you had sort of this retinotopic mapping this log mapping right where the phobia was represented here in a human the blind spot of somewhere here so now what you do sort of you cut this here and then you fold it outside that's v1 now next to here you have another area that's called v2 v1, v1, v2 so this is the lower this represents the it's the upper part of the brain representing the lower visual field and this lower part on the lower banks of the sulcus represents the upper part of the visual field and so here's v2 and then outside here you have an area called v3 so again this is cut and you can see that here those areas so I'm not going to talk about the techniques if getting quite sophisticated how you can do these imaging techniques and here it's sort of compared by Roger Tuttel and the MGA in his people here you do something similar based on electrophysiology now today you might have seen some of the talks here over the last couple of weeks today you can also do this imaging in monkeys and directly compare them but the bottom line is that the organization is rather similar it's not identical and you wouldn't expect that since you know we diverged like something was it 15 or 20 million years ago I mean a long time ago that we had a sort of share the common ancestor between the macaque monkey and humans so you wouldn't expect those things to be identical but they're rather similar that's the main point you can define similar areas that have some of the same properties so here what you can see it's the same retinotopic mapping that you find in v1 you also find in other areas here you know you you shift from the let's see this is all the the upper visual field and here you move from eccentricity from things close by to far away and then you move continuously between the various maps there's not an abrupt discontinuity it's continuous you would expect that so you continuously as you move along here you continuously move at one particular angle you move towards the periphery and then back again and then again towards the periphery in these different areas so it's a little bit like a collage like a painting let's see an impressionist painting where you have where the world is put together in these different maps you have these multiple map representation of the visual world we don't really understand the existence of these different maps I mean they seem to be there these maps are different like for example the grain of resolution in v1 is highest compared to even the next map in v2 we don't really understand all these differences why they exist and there are probably many many properties that we haven't at all that we have no idea of there we just don't know right now so mt is this area I'll now talk about pretty much for the rest of the class mt of v5 that seems to be highly specialized for motion now it's a big, there are lots of people fighting in this field about how do you name these different areas and how many different areas are there and people write very the tupperative and very nasty articles about each other because somebody calls this v4 v and somebody else thinks it's just preposterous it should be called v4 I kid you not I've had debate with these people they're very very upset about you know the fact that one calls it v4 it's an awesome area that has no function and that's typical what scientists get okay for a what's not quite fair this represents most of the visual field human this only represents a smaller part of the visual field just you can do mapping quite precisely instead of in the center part but in the periphery you can do it as precisely so this is not the same so this in other words only encompasses a smaller part of the visual environment compared to the monkey you're asking me whether the upper visual field is represented more cortical area is represented is needed for the upper I thought it was the other way around isn't it yeah I mean do you know this answer to this question but attention is also more yeah so it's a good it's an interesting observation yeah you're right based on this map it looks that way it's a good point but I mean you would expect and I think it's true as in Patek also that you've got attentional you can for example do more attentional modulation is stronger in the lower part of the visual field in the upper it makes sense if you grow up in Savanna where we supposedly evolved in the upper part of the visual field there's less interesting stuff because usually it's covered by the sky then in the lower visual part of the field but I agree there seems to be an asymmetry here yeah exactly so if you want to find stereo information you want it you rarely do things like this but yet here it's the other way around good point okay let's skip that okay let's talk about one particular area because it's been one of the better characterizing there are some classical experiments that I'd like to demonstrate that have been done here this area called MT is relatively small maybe like this big like my thumb near 50 square millimeter lots and lots if you go to society of neuroscience a meeting with 30,000 neuroscientists all your best buddies are there and 20,000 on the art of talks and posters it's overwhelming and I sort of wax and wane my feelings between sort of being elated and being depressed elated because all these people care about this minutiae that usually you think nobody else cares about on the other end you also feel a bit like a data is painter that goes to a conference with 20,000 other data is painter you think unique and only you think of these thoughts and then you find other people who lots of them so at a meeting like society of neuroscience you find many sessions just dedicated to area MT and associated areas in a very hot area partly because it can be relatively easy to identify it's been first seen in post mortem tissue in humans by the degree of myelination it has lots of myelin and so that tells you already certain in retrospect that because myelin as I said is involved in making as in speeding up action potential propagation she might expect based on that argument an area that's involved in fast temple processing so this is done here you can focus and do a study I'll show in a second what you're using here for stimulus it's going to be relevant because we will ask you to do this for homework so you should really pay attention now there's a part of the brain this is the front this is the back these are various cuts, various slices this part of the brain here is called MT and it's relatively easy to obtain I've seen it in my own brain you can see it pretty much in have you ever not seen it? this is one of the cartographers of MT it's always there good, you're showing this is why brain imaging is so fantastic I'm dissing it a lot here in the lecture in the other way it's fantastic as you can do before you can only identify this in post mortem in dead people in the brain of dead people but now you can take a normal subject and then you can tattoo like one neuroscientist Nancy Kavrischer has done what you can do, you can tattoo the location of the area empty on your skull using blue ink and then you can use TMS she's tried to do this, you can then use this transcending magnetic stimulation to specifically put these the coil above onto the skull to try to selectively interfere with motion processing there at some point I thought this was going to be the next in cognitive neuroscience that everybody with their stuff tattooed onto their skull but it didn't turn out quite that way so now I'll talk you about some experiments like they are very important and we'll ask you for the homework in fact we'll ask you to actually do these experiments yourself not in a monkey but in your own brain and we will not ask you to move from correlation to causation so what these experiments that I'll now tell you about have shown as the following A they relate to other areas but they relate here in a nice quantitative way the firing behavior of individual neurons to the behavior of the animal and indirectly therefore through perception but the only way we can get a perception in animals is by their behavior and furthermore in the second set of experiment the same group around Bill Newsom at Stanford use Kern injection so they can directly buy those neurons and perturb the system and thereby move from correlation to causation okay so in it's part of the it's empty part of the ventral system sorry the dorsal system the vision for action system in the middle temple though that's why it's called middle temple not motion area some people think and many of these neurons I mean this was characterized first by Van Essen and Monserl here many of the neurons like 90% plus are highly selective to motion so this part of the brain if you stick an electrode in it reacts very reactive to moving things and for instance many of them are highly direction selective so if you discover so what you do you know you have the monkey fixate and then you move a bow or symbols like this you move it over the receptor field and you find if it moves in this direction it really fires very strongly if you move it in another direction the firing rate falls off in the opposite direction which you call it's preferred direction and then if you move in the opposite direction you may get no spikes or one or two spikes that's a null direction and they are orthogonal they are anti-parallel okay so what you do you go into a monkey a trained monkey you find it's preferred direction and you find it's null direction that's the opposite direction let me show the experiment that you do you are first pioneered by some psychologist this motion signal because it's a relatively pure motion signal we don't have any contours but if you take an edge or move an edge you have not only motion information but you also have edge information so this displays the advantage there's no edge information there's no form information there's just dots that move and what do you see in this stimulus hello hello what do you see what? okay so that's a random dot stimulus so here each dot is randomly in various ways of doing it each dot is basically randomly re-plotted and has a finite lifetime in other words each dot only lasts for 100 to 200 milliseconds and dies and then another one gets created but each one moves randomly so there is no motion signal this is called 0% coherency now what happens here let's get a wrong answer what do you see so here you have 20% of the dots move in one direction 80% move randomly so 1 out of every 5th point moves in fact diagonal and 4 out of 5 points move randomly and what happens here so what you can do now you can titrate what you can do now you can titrate the motion signal from 0% coherency the cell prefers direction so now you can titrate the amount of motion signal in that preferred direction you can go from 0 when there is no motion signal they all move randomly to 20% 30% up to 100% where that fraction of dots moves in the cell's null direction or you can also go negative 100% that moves all the dots move rigidly coherently as one into the cell's null direction so you can this is a parameter you can now vary in a human and monkey experiment and you can ask and this is the question we'll ask you in homework let's give a threshold 82% for various theoretical reasons 82% so I'd like to know for what fraction of dots at this density of course it depends on all sorts of parameters which will specify in the homework so you don't have to explore those but if you're looking at it directly with your eyes what fraction of dots have to move in one direction in order for you to reliable say yes the motion signal moves from right to left and all the others are just noise they move randomly turns out to be a small number if you really pay attention it's definitely below 10% if you're looking at it in other words your brain is good enough that if only a few percent of the dots move in one direction all the other ones move randomly you can pick up that signal and so we'll ask you to do this curve for your homework so and this is called psychometric curve because it relates this you can really see the encapsulation this is where psychophysics originated using sort of physical quantitative tools to study perception because here you vary a particular physical signal you vary the coherency from minus 100% to plus 100% and you ask people to say so you have to do what's called I guess a binary choice you can't say well I don't know or you can't say well I think it moves there the choice is either it moves to the left or to the right and you have to tell me does it move to the left or to the right and if you don't know you have to guess so there's only two choices binary choice that allows you then to plot this curve where on the x-axis you put the coherency and the y-axis you put the fraction of trials where you correctly predicted that it moved in the preferred direction and now for 100% even anything above 30-40% it's trivial you can very easily see it moves in that direction likewise if it goes to anything below minus 25% in the other direction I mean things are symmetric you can trim these seeds in the opposite direction in principle if your brain is unbiased but that's never entirely true your brain unbiased at 0% coherence remember that by definition no motion signal 0% now you might be biased for example if you're doing this button push your right hand might be quicker bigger whatever you might 55% of the time push the button right and only 45% your visual system might be biased there are all sorts of biases the people are usually never exactly at 50% at 0 but close to and then sort of you have this deep curve and I don't know here the threshold okay this is called the psychometric curve now oh shit I forgot a slide oh that's a bummer that is a bummer okay well I can draw this so because now we can do now what Newsom did what Newsom did he now asks the same question of the cell so I'm recording now from a neuron and while the monkey does a behavior okay so the monkey is sitting in the chair you're recording from its brain you can do that because as I said there's no you know it doesn't hurt while you record its monkey's behavior while it does its task you record from an area in MT and you optimize the stimulus for this particular cell in MT in other words what you do you first with the electrode find a cell you discover it's preferred in null direction it's the third direction horizontal to the right to my right it's null direction to my left and now you have the stimulus and you know you ask the question you titrate the fraction of the coherency in one or the other direction and now you ask the neuron the animal's behavior and you get one of these curves at the same time you now query the neuron and so you discover for example that for motion in this direction let's see how do I do that okay let's do it like this so you plot here you plot the response how many you move the stimulus for two seconds and then the animal has to make its response has to either indicate by hand or indicate by eye movement where it thought the motion signal was in which direction and then so let's say you count spikes you count how many spikes there were over the last two seconds so for motion in the direction you start off with a 100% coherent signal okay so in the preferred direction you're going to get lots of spikes let's say on average 100 hertz 100 spikes over the two seconds so since it's a random distribution so typically this is number of spikes so if you move in the preferred direction typically it's spikes let's say this is 100 sometimes it might only spike 80 times on the next trial it might do 110 it's a stochastic variable in the wrong direction then you see sort of it looks you know a little bit like this this is 0 obviously it can't go below 0 so it's biased but let's say typically it's around you know let's say 10 spikes 10 spikes 10 spikes so now you can ask mathematically the question you can use what's called ROC curve receiver operating characteristics which is a simple mathematical technique from an ideal observer theory the assumption that there's an ideal observer that can count spikes and that knows about probability distribution so if I'm now looking in this one trial I have four spikes I have a very good idea what the signal was you know if it's four spikes it's extremely unlikely to come from this distribution so with very high confidence I can say well it was a zero signal also on this trial the neuron spiked 110 times it spiked 110 times it's extremely likely that the signal was high coherency now this is let's see for 100% coherency so this is minus 100% this is plus 100% now let's say I do this for 20% coherency which most some of you picked up and well once you do this a couple of times it becomes fairly routine so for 20 certain things are going to be more complex for 20% it might the two curves might look like this so you know on average if you move in the preferred direction 60 times and in the null direction it might spike let's say 30 times because now you have all these other signals so now if the if the neuron spiked 80 times you're pretty sure it's here if it spiked 20 times you're pretty sure it's here but if the neuron spiked like see 40 you know 40 spikes then you know then you're going to be confused you know you say well I don't know it could be here could be there and you're going to likely make mistakes of course for 0% if the cell is unbiased if the cell might also be biased for 0% they're going to overlay so now you can use this essentially in compute probabilities and do it in an ideal observer way and then you get what's called a neometric curve so now you can essentially use this use this to predict just based on observing that cell many times so essentially what you're doing you listen to the neuron and say okay it spiked very strongly I predict motion in the null direction and this time it spiked very weakly in the sorry I got it the other way around you know what I mean and then you get a curve well so the big question is how will the curve look well the curve in general looks like this this is coherency and this is you know percent response that you get out of this mathematical operation I'm not making any assumption what the brain can do I'm just saying well I'm counting neurons and I'm putting it into my mathematical machinery so if C as I said there if C is very big like plus 100 I can make this decision almost perfect this is one if it's minus 100 I can again make it perfect and then it's going to be just like with the behavior it's going to be some curve like this now of course you would predict now remember well I mean what would you predict if you relate I mean what do you think the relationship between the behavior of the animal and behavior of the single neuron you guys know the answer what do you think it is what? so here I'm averaging on one cell but just generically one case we're looking at the output of the entire system the monkey whatever it does the monkey comes to a decision now I'm querying one of its microscopic variables I'm just querying one of the neurons and so in both cases I can do these curves this is called the neurometric curve if I do it for neuron or psychometric if I do it for the animal what do you think the relationship if any between the psychometric curve of the animal versus the neurometric curve of the animal neuron this question is likely to be on your canon incidentally what do you think the relationship what do you think the relationship is likely to be yes that's who if I for example pick a neuron in the olfactory system it's going to be flat right no it's a good point if I do this for an olfactory neuron it's going to be flat, it's going to be sunfrying it's independent but here remember I go in an area that I know already from previous experiments and I optimize the stimulus for that particular neuron so they're likely to be related so what do you think the nature of the relationship is well this one would be an olfactory neuron that doesn't care anything about motion you know it might be like this or it might be like this you know they should be ideally monotonic and practice often they're not what? so I guess what I'm asking what do you think what do you define a threshold here for the neuron just like before so let's say you ask 0.82 and you ask okay what coherency level see threshold this is my threshold and I want to know if I just based on listening to the neurons I on average want to get 82% of the right answers what coherency level do I have to use in order for the neuron to be able to predict that what do you think the relationship between this value is for the single neuron and for the monkey how much better how much more accurate do you think is the behavior of the animal versus the behavior of a single neuron if you assume it's in both cases are just optimal detectors either uncrying the single neuron how good is the single neuron at telling the direction of motion or how good is the entire animal at telling the direction of motion the neurons should be a little bit better well okay we rule around any neuron we're just looking at motion specific neurons well okay so I mean the intuition okay my strong intuition the intuition of people in the field is that of course neurons are going to be much worse why? because the brain has access to the behavior has access to many many neurons so all individual neurons won't be terrible good but I can pull over many of them and therefore I can do much better this is not just hand moving you can make this argument very rigorous I guess as you sort of were implying it turns out remarkable that on average half of the neuron you could not distinguish statistically speaking the psychometric curve of the single cell from the behavior of the animal in other words so typically the curves look like this a neurometric and the psychometric curve will also look like this on half of them you cannot tell and on some they're worse than the animal and some the cell so some the cell is better than the animal the cell is worse than the animal so this is remarkable because if there are many neurons you might argue if there are many neurons that do so much better than the animal well then something must be corrupted then in other words there's some neuron that really knows the information very well but the answer doesn't come out now that could be true but everybody's expectation is that the brain is optimal there's some signal in the brain that really knows about it the brain will somehow learn to exploit it so this has been an embarrassment I have to evoke noise and correlation I mean there's also explanation why this might be which again are favorite topics for candidacy questions but the point is you can make the point that's relevant for us because it's rather artificial task the monkey has to wait for two seconds all of that the point about it you can make a quantitative systematic relationship between a simple variable the integrated number of spikes and behavior although it is remarkable that if you use this mathematics you can get curves that are very similar between behavior and neurons but the point is averaging over number of spikes in a two second window does give you a quantitative relationship to the behavior of the animal not just sort of hand waving but actually quantitative and it's not oscillation it's not synchronization those things don't really seem to matter a lot in this case it's just that the simple thing the dumbest thing you can do you just count spikes over two seconds and put them in a relationship to the behavior in a nice relationship in fact here you look at this this shows you the relationship for all the 212 neurons between the threshold at the single neuron relative to the threshold of the behavior and it clashes around once in other words roughly half the neurons do better and half the neurons do worse and nobody expected this now for the second part of the experiment what's really important is that the fact that neurons don't occur randomly in the brain but are clustered and I pointed this out already in D1 so if you go to MT it turns out if you look at direction of motion where the neurons that respond to this direction to this direction to this direction well once again they occur in columns in other words in this area and this is roughly a millimeter on the side so let's say for 100 or 150 micrometers all you're going to encounter are the neurons that roughly code for this direction of motion and then if you move with the electrode over here they roughly code for that direction of motion they're not randomly intermixed now what do you think will happen if I now take a current a wire and I poke it into the brain and I inject current so let's say I inject current into this column so I'm going to excite neurons for motion in the upper direction it is not unreasonable to suppose that now remember there are postsynaptic neurons that look at this area MT so let's say the postsynaptic area is called MST another brain motion area it gets input from area MT let's say MST looks at MT and it sees an increase in these neurons these neurons now all fire twice as much of course MST doesn't have an eye outside and can say well wait there's no comparison motion signal outside all it sees is it gets input from area MT and sees this increase in motion in this direction so supposedly probably when one would assume well it would signal oh there's something moving there even though there was nothing in the environment and you just happen to cheat and put a current in there so that's what you might expect in this case that's what you do see the complicated experiments I don't want to dwell on them but the point is you can get a there again you can quantitatively shift those neometric curves you can get one neometric curve and you can systemically shift you get a different neometric curve for example here so you can quantify now you can see the threshold that before was let's say 20% now with the current injection if you look at the intersection at 82% it's 10% so you can say well my current injection corresponds to adding 10% coherency because in the absence of the current the neuron needed 20% of the dots in the direction in order to tell whether it's left or right now with the current injection all it needs 10% so my current injection corresponds to motion signal of 10% and so it's very there's a systematic relationship and furthermore if you I mean you can ask what goes on, what does the monkey actually see very interesting question I talked at length once because I was interested in before I told this to my wife but the neurosurgeon also didn't want to do it I talked to several neurosurgeons about the idea of doing this myself where I would identify REMT in functional imaging and then have a little burr hole and then you put a little electrode in because it's a really interesting question what do you see? there's several possibilities one is that if you put an electrode in the brain you actually see motion without anything that's a bit difficult to imagine it's a bit weird I'll show you in a second a motion after fact where in fact you see exactly that you see motion in the absence of change so it's not inconceivable on the other hand when Newsom did this without any motion stimulus the animal did not respond so if there was no motion stimulus he injected current it's not that the animal told them there was something was moving so probably the current that he used well, I don't know, in this trial I feel it's more likely to be right to the left and it depends also it depends on the highly sensitive way you inject so if you inject current in one particular column you might only bias in this direction and in a neighboring column you only bias neurons in that direction as you would expect from this layout no, this doesn't work it's horrible interlaced I wanted to finish with REMT because you can also get it active for illusionary motion with motion after effect but I don't think this is going to work let's see, I know this so there's this additional motion signal due to the LCD it works it's not as powerful as the color after effect we can also see this again this interlaced problem yeah, I see it a little bit it works very well on the old technology if you have a turntable none of you will have a turntable but if you still can, it works very well particularly if you look at something like your hand or friend's face so it seems to exploded you and you see activity and people have done this experiment Dave Higa was probably the best version where you can see that even the absence of a physical motion similar to REMT is active so in monkeys there's a nice relationship between the firing of these neurons and the behavior in a motion task if you stimulate it you get a very specific effect in a motion task it is active this area is active in for motion as I showed you, that's how you define it in humans, remember first I showed you the picture of the human subject when he looks at motion it's also active, albeit somewhat weaker for illusionary motion lastly, remember I talked about the essential node in the first, in the second lecture I think so there's this famous patient in fact there's two patients, one patient in 1918 in Germany after the war and this is another patient much more recently she just died, Zeal is a neurologist in Munich and he described the patient with a bilateral stroke that included REMT it's not just limited to REMT but included REMT on both sides, bilateral and as you can see here this lady was unable to perceive motion so she could see things I accept that things were very slow what is possible and we have correspondence with the neurologist in fact she never saw motion but if things were slow she could judge motion because things were far away and then close by and you can judge by relative parallax or other cues, you can judge well it must have moved because it was far away and now close by she never gave any appearance of actually seeing motion so what this tells you this relates to the early model I drew that the existence of essential the clinical existence of essential of essential nodes implies that the neuronal correlate for this aspect of consciousness in this case motion has to be localized and if you have a stroke in that area you lose the conscious perception of motion you don't lose all visual perception she had fine stereo and she could do lots of other visual tasks she just couldn't see consciously motion anymore and so this really seems to imply this one area one of the best examples where we have pretty good evidence that this area is involved in motion processing not only in motion processing there's also stereo selectivity there like most areas of the brain it's not this sort of that one area one function but it seems to be heavily involved in motion and if you have lesions in that area particular large lesions you will have permanent deficits in motion perception okay let's leave it at that