 Uh, everyone welcome Mark. Hi. Everyone, hi. My name's Mark, and I'm a neuroscientist. This presents a small problem for me when I am in bars or at cocktail parties, because people ask me what I do. And the most accurate thing to say is, I'm a neuroscientist. But people really don't know what to do with that. They say, you're a neuroscientist? Is that like a urologist? Or they say, oh, is you going to be a brain doctor? You want to operate on my brain? That's not that kind of doctor. So usually what I say is I read minds, because that actually is the conversation that I want to have about, come on, what do you mean? And I say, well, I use fMRI to measure the mental state of people while they're doing things. But as Rick pointed out, brain reading has gotten a bit of a bad rap of late, because there have been a lot of studies with a little bit overblown conclusions, like looking for the neural correlates of maternal and romantic love, and looking for lies in the brain, trying to do lie detection via fMRI and General Lair, who, well, General Lair. And then in response to all of these, there have been a lot of people that have said, this is all bunk. This is baloney. We need to be very skeptical about all these claims made by these neuroscientists. And there's this rise of neuroscepticism and head talks about how to detect neuro bunk. And my personal favorite is this one, which is the seductive appeal of mindless neuroscience. It's such a wonderful pun. So is this really so different than something that happened about 150 years ago? Is this just phrenology reborn? Is this trying to assign functions to bits of the brain based on the shape of the skull, which is what they originally did? Of course, we have a little bit better measuring them now. But this guy, Gaul, tried to locate particular spots in the brain that were associated with particular personality traits based on a couple of people he looked at that had bumps on particular parts of their head who demonstrated a lot of a particular quality, for example, destructiveness or inhibitedness or benevolence. And well, that just didn't work. And it ended up being more than a little racist because he decided to look at criminals and people that he didn't like and draw conclusions about brain functions based on those people's brains and how they were different people he did like. But we're not going to do that. The point is, is what we're doing now any different or not. And the only way that any of you really have to make that judgment for yourselves is to have more information about exactly what we're measuring or measuring the brain. So for the entire talk tonight, I'm going to take you at your words that you actually are nerds and get extremely nerdy about methods and about fMRI and what exactly you're measuring. So hang on tight, it's going to get nerdy. So I'm going to tell you about one particular way to measure the brain, which is functional magnetic resonance imaging. And I'm going to try and get at what each of those individual letters in the fMRI mean in the next few slides. So first of all, we're not going to go into order. We're going to go with magnetic first because it's kind of a weird thing to use magnets to measure a brain. You don't think that what's going on in your brain is actually magnetic. It is electric. And you know that electricity and magnetism are related if you take a physics class. But it's not even that satisfying a relationship. You're actually measuring the magnetic properties of different bits of tissue when you're doing an MRI scan. And you're measuring the concentration of protons if you're doing a standard structural fMRI. How many of you have had an fMRI scan or an MRI scan? Raise your hands. Wow, it's like a third of you, half of you. That's good. So you know what fMRI sound like. You know that it's kind of this tube that they stick you in. And I've got a picture of that later. And Rick had a picture of one already. But what that machine is looking for are differences in the magnetic properties of different bits of tissue in your brain. So some of them have more water and some of them have less water. And this is the presence of water determines the magnetic properties or to some extent determines the magnetic properties of different bits of your brain. And so if there's a difference, if there's a lot of water or a little water, you can read that out by just reading out magnetic properties. You can make an image that looks like this, which is actually my brain, which looks kind of alarming like a submarine sandwich. It's a little, it's like in the long end. It's just kind of, you know, people have different shorter, longer brains. Mine's kind of longer. But what you really want to measure if you're trying to read your mind, thank you. Thank you. Thank you. What you really want to measure if you're going to try and read your mind is what's happening from moment to moment. And so this measure of how much water there is in the given bit of tissue is really mostly determined by whether it's cell bodies or connective tissue, which is not going to change from minute to minute. The anatomy, it's pretty static. So you need a measure of function. And one measure of function is how much blood there is in tissue. Isn't that a great image? That's going on in such a head right now. But the point is that your circulatory system, this is maybe a little known fact, it operates at about a third of its actual capacity, which is why when you go into shock, you faint because your blood vessels all dilate and the blood goes down and you fall over because there's no blood in your brain. But you can see the effects of this when you're exercising because your skin flushes. The blood vessels in your skin dilate and you need to cool down so the blood goes to your skin and water evaporates off your skin and your blood cools down and your body cools down. And the same thing happens in your brain. When your brain's active, the bits of it that are active need more blood and so blood gets channeled to those areas. So this is not a terribly satisfying measure of neural function because it's not actually measuring electricity, which is sort of the currency of neural communication, but it's correlated with it, which is kind of good enough if your measurement is slow enough. And that's where I is. So what you get is this measure of whether there's more blood or less blood in a given little bit of tissue over the course of your experiment. And then it's up to you to have something interesting happen during that experiment so you can make some conclusions about what that bit of the brain is doing and why the signal went up or why it went down. So this is what our scanner looks like. And have you, remember a third of you have had an MRI so you know that it sounds something like this? An MRI scanner. That is a bunch of people I think at MIT that decided to program their gradients in the MRI to make exactly that noise. Which is just great. But the point of these scanners is that they are extremely large magnets. They're very, very strong magnets. And I'm gonna show you exactly how strong right here. So the magnet that we use at Berkeley is a three Tesla magnet. The jump yard magnets that pick up cars are one Tesla. So this is three times as strong as those jump yard magnets that pick up cars. And this is what you're going into. Oh, I get an MRI. I can see a lot of open mouths and raised eyebrows. That's good. So the force we're reading on that guy right now just at that part of the magnet is plus 300, 280. That's like what a wrench would weigh on Jupiter. Yeah. So the point is this is a very, very strong magnet. And the best illustration of this if I can move my mouth forward which I'm not sure I can. Yeah. Yeah, I think I can. So we're gonna watch all of this. So this actually was done at Berkeley. And this was done when Berkeley changed over from the four Tesla MRI coil to the three Tesla MRI coil. So the guy that walked into that shot is actually one of our MRI technicians. And these are the awesome people that I work with. The last time I went to the MRI scanner and needed to change the MRI protocol I was working with, I wanted to change exactly the way I was scanning. I needed a dummy to test that on. And usually you test on plastic bottles. But then it's just so cool that I walked in and said, then I need to do the scanatomical scan. Do you have something I can do that actually has some variations such as the plastic bottle? And he said, yeah, hang on. And he came up with a whale brain, you know, bag. And it was, you know, I had like formaldehyde or something in it. And we stuck that in the scanner, not this scanner, the new one. And scan that to test our protocol because these guys are just great. So that is a chair. And the upshot of this is that that chair ends up weighing 2,000 pounds. And there's a really good shot of it kind of breaking. We are at 700 in this case. Yeah. 700 pounds. Oh, yeah. So the closer it gets to the center of the magnet, the more it weighs because the magnetic field is so strong. Yeah, it jumps. So this is what you're in. So that ends up weighing up to 2,000 and pretty much flat lining up on the ground. But we're going to skip ahead on that. Oh, sorry. The rest of the slides are just as good, I promise. So you have this incredibly strong magnetic field because you want to measure magnetic properties of the tissue. And you can't just measure properties of things in the Earth's magnetic fields. Earth's magnetic fields are very weak and you don't get much magnetization of the stuff in your head if you're just standing around it or it's a magnetic field. So you have this extremely strong magnetic field and then what an MRI scanner does is creates variation across that magnetic field. And this is really the great good of fMRI is that it can measure this property of your entire head, the magnetism of your whole head without actually opening up your head and getting into it. So this is totally different from any other kind of, well, almost any other kind of imaging. Head imaging, you have to have an injection of some radioactive stuff that you really don't want in your body. You don't want to do it more than once or twice. I have been in maybe a hundred MRI scans and if it turns out that it is bad for you, I'm pretty much screwed. But so far the evidence says it's just fine. It's just like getting your toenails clipped. There's no problem. So the point is how do you actually get information about a little itty bitty bit of your brain when you're measuring the whole brain at once? And that is what I've spent a whole bunch of slides here, a whole bunch of ever trying to convey to you. So it can be one of what frequency is. Okay, so you're all familiar with high frequency sounds and low frequency sounds. High frequency sounds are like this, like my fiance's voice is up there. And low frequency sounds are like these, they're all those dark matter. And so it turns out that frequency can operate across space as well as across time. And so if you have a sound wave and a high frequency, it sounds high pitched. And if you have a sound wave, it's low frequency, it's low pitched. But you can also have spatial frequency, which is variation across an image or across something that you're looking at. And low frequency things are blurry and high frequency things are really edgy. So these are spatial frequencies that I have matched up with increasing the high sound frequencies. So this is an incredibly high frequency variation in space. And the earlier ones are low frequency variations in space. The point is that any image can be quantified according to those frequencies. You can take an image such as this handsome gentleman here and you can take only the low frequency components. And you can take slightly higher and slightly higher and slightly higher frequency components. And you can get basically every frequency component up to every single pixel being different. And all those make up that image. And this decomposition of an image into its constituent frequencies is called a Fourier transform. This is a bit of mathematics that was invented by the gentleman who was the ambassador to Egypt for Napoleon, who was also the first guy to predict that global warming might be happening because maybe the sun is progressively heating the earth because of some effect of the atmosphere. This is a very interesting man. Anyway, it turns out Fourier analysis is used in all kinds of things, including image compression, such as the one that prevented you from seeing Betty Page getting naked last talk. And I'm really sorry about that. I'm really sorry about that. So anyway, the point is that you can take an image and transform it into what's called frequency space. And these are all the frequencies that make up that image with the low frequencies in the middle. So the really low frequency things are there and these are progressively higher frequencies going out from the center of that image. And what this image in the center is, is basically a recipe saying, look, I need, you know, the brightness is the amount. I need this much of a low frequency and I need progressively less and less of each of these other frequencies. And if I combine those all together, I get Ryan Gosling. And that in the middle, that is Ryan Gosling. I swear, that's Ryan Gosling. It doesn't look like it, but you know. This is exactly how fMRI works. Is it collects the magnetic signature of your whole brain, but it collected at different frequencies and it collects them over time. And so that actually is my brain and I know it doesn't look like it, but that is, that's what I'm thinking. That's like my brain right now. That's like a two seconds of my brain. So this is how you collect an fMRI image. You manipulate the magnetic field around someone's head. And this, by the way, is gonna be the most hand-wavy part of my talk and I really am sorry for that, but I just could not squeeze into 20 minutes without this. You establish some spatial frequency across the brain. You basically ensure that the magnetic field, but that your signal that you're measuring, only is coming from this particular grid across the brain. And then you collect the measurement. And then you change the grid and you collect another measurement. So you do this to the brain, you read it out in an image like that and you transform that image into something like that. And so what I'm gonna show you now is a video of what it looks like as you progressively collect a single image in fMRI and it looks like this. Yeah. Okay, so the point is that you don't actually collect it pixel by pixel. This is really weird and really cool. And it is only because we don't have to actually measure a single spot at once that we can even make these measurements. This math, this like 200 year old math, 150 year old math is what allows it to happen because we measure frequencies. We measure one frequency and if you stop it partway through it, you get something that looks like that. Looks like shmoo. I mean you don't have like one individual little pixel of brain, you just have shmoo. So anyway, this math gives me very excited and I had to really restrain myself from putting the equation up here but I'm not gonna bother you with that. Oh, thank you. So what I hope you learned from these last couple of slides is that functional means we're measuring over time, we're measuring, so by the way, I didn't mention this but this is actually sped up 30 times to how fast it happens. So this takes two seconds to unfold and in fMRI, we actually take the whole brain in two seconds, not just one slice of the brain like this. So that's fast and the magnetic fields are changing very, very quickly and they're changing so quickly that the overall magnetic field exerts some force as they're changing and that's what makes them vibrate and that's what gives you the awesome soundtrack. So when you're hearing the eh, eh, eh or whatever your MRI sounds like, it's because the magnetic field's changing so quickly the wires are just vibrating at its frequency so fast that they're sound frequencies. Yeah. So it's magnetic because you're measuring magnetic properties of tissue, by the way, I didn't mention this but the magnetic property is that there is iron in blood and that that iron has a magnetic signature that you can measure and you can measure how much that there is from second to second. So it is magnetic imaging, it is resonance imaging because you pick your particular frequency and you measure that frequency, you basically set up this frequency, you zap the brain and you listen for what happens and whatever resonates, that allows you over the course of two seconds to eventually recreate an entire image of the brain. Okay, fine but what can you use this for is what you really wanna know. I should mention before I go on actually that there are a couple of really important limits placed on how much you can measure a brain by the technique. This takes two seconds, two seconds is forever in brain time. So if you think about really reading a mind, what you want is every little individual bit of the brain, every little element that's doing something you wanna measurement of that because you have no idea what's important. And so what you want are a hundred billion measurements and you want them a thousand times a second. That's gonna break your computer really hard for one. And two, there's no way we can actually measure that. So what we're stuck with is either measuring a single neuron or a single individual bit of the brain, a single cell and what it's doing or maybe a couple hundred cells if you're really lucky and you have some fancy post-docs in your lab. Or you can measure the whole brain as you measure an fMRI and you can measure it slowly. And if you wanna measure humans, fMRI is pretty much all you've got because to get into humans' brains you have to cut open the skull and people don't really like that. So fMRI is what we've got if we care about humans and if we care about human cognition, which I do because I like reading brains that have something interesting going on in them. So now the question becomes, well, what can you read out with this kind of technique? And a really important consideration is this really boring first one that this doesn't actually measure any absolute quantity of the brain. It measures how much magnetization there is, how much blood there is. What's zero on that scale? So the scale here goes from two to negative two because there is no scale, it's arbitrary. You don't have a zero. And so your contrasts that you do if you say that this area is the God center of the brain or the love center of the brain or the lying center of the brain, you can only get that by making a measurement when they're lying and making a measurement when they're not lying and subtracting them. So every time you read an fMRI study, what you should be looking at is how creative were these people in determining what two things they're contrasting? Or if they did something even more creative and weren't just contrasting things that they actually made some kind of measurements. Oh, no, that's what our lab does, but that's what we're gonna talk. So the point is pay attention to comparisons because a lot of times, for example, lying, people, they're told to lie, does that satisfy you as a real lie? I mean, I tell you, tell me a lie, is that the same thing as, no, I'm not cheating on my wife. I mean, come on. It's not the same thing as, these are not the toys you're looking for, it's just different. There are, you guys aren't here, that's good. So what fMRI can be used for and can usually be used for is mapping studies. So if you've got something that's going on in the brain over the course of seconds, something that's not your decision of where to move your eye, your eyes move like three times a second, that's just, whatever computation is going into that, it's just too fast to measure with fMRI. If you ever hear an fMRI study talking about the influence of one area on another, don't believe it. Two seconds, there's like 2,000 things that happen in two seconds in the brain, but you can't make causal conclusions when there's 2,000 things that might have been going on. So anyway, mapping is what fMRI does reasonably well. fMRI makes map of the different parts of your brain that are doing somewhat different things. A really famous example from early fMRI, from a woman that I've actually collaborated with, who's a wonderful woman, is that there are some parts of the brain that are sensitive to faces that are particularly responsive to faces of other people if you show them visually to the person in the scanner and that some parts of the brain will respond really strongly if there's a face and not so much if there's not a face. So that contrast, I mean, that's a pretty clear contrast, that you can believe. But look, it's a face where it's not a face. It's a really responsive area. And look, every time I show a face, I show Jennifer Aniston, I show Halle Berry, I show Joe, I show Jim, and there's a response. And so mapping things like that can pick apart the components of perception really well. It's when you start getting into higher cognitive functions you start having problems. And also you start having problems because the mapping, well, it's only as good as the resolution you've got. And when you're changing those gradients in fMRI, the problem is that the gradients can only change so fast. And if you're changing magnetic fields around someone's head and you change them really, really quickly, eventually you give them the seizure because the magnetic field changes so quickly will stimulate the neurons in your brain. And you can't do that. So there are fundamental limits to how fast you can take these measurements. And so because you can't go fast, you can't measure quickly, and you also can't get to too small of what we call voxels, which are volumetric pixels, which are just the smallest little bits that you can measure in fMRI. So we can't really get, we're almost, almost, just from hacks and clutches, tacked onto this technique. We're almost down to what's a really interesting resolution, which is the resolution of cortical columns. And I cheated there. So what else can it map besides visual function, besides what visual categories you're looking at? Well, people would use this productively to look at mapping of the body. And one of the interesting questions, and I only put this slide in because I knew that any page shot was before me, was the mapping of the penis. Because there had been some speculation that, okay, I don't know how many of you know this, but there's very famous, if you've taken a neuroscience class ever, or read a lot of popular neuroscience, you know that your body is represented in a map in your brain. That your finger is right next to your hand is right next to your arm is right next to your shoulder, in this little tiny representation of a man in your post-sensual sulcus, right there. And the opposite sides of your body, some of you are reading ahead, the opposite sides of your body are represented on opposite sides. And this was originally found out in epilepsy patients by stimulating little bits of the brain with a needle, with an electrode. And people would say you're touching my arm, stop touching my arm. And they say we're not touching our arm, we're touching your brain. And people could not tell the difference. But in those original studies, I guess they were a little skittish about getting too close to the chunk. So they somehow concluded that the representation of the penis was actually near the feet, which provided a lot of fuel for speculation as to why people might have put fetishes and things like this. But sometime in the mid-90s, the Germans decided to straighten this all out and this is just my very favorite method section ever. And I've got to read it to you in case you haven't read it yet. So in contrast to previous studies that applied electrophysiological sensory stimuli, techniques, the stimulus touch was chosen to present a physiological stimulus. Bear in mind, this is just someone who's lying in an fMRI scanner, such as you just saw. With an ultra-sensitive tube brush, Dr. Best Flex is sensitive, the one that's so sweet, the one that's super healthcare. We brushed the needle aspect of the left halux that's actually the foot, the left penile shaft, the left pre-pupus, the glands in subject five, a worm, and the left lateral abdominal wall. So they're basically just saying touch, touch, touch, touch. And it turns out that the chunk is exactly where you'd expect it to be. So this was actually something we didn't know before fMRI. So it turns out you can read out maps of sensory function extremely well. But if you're talking about higher cognitive function, then you have some more problems and it really depends on partly the creativity of the experimenters and partly they're just fundamental limits and that you can only measure these chunks of brain that are a couple of millimeters across and you can only measure them every couple of seconds. So if you've got confidence that whatever you're trying to measure is some ongoing state of brain activity, then you're fine. Then you can draw whatever conclusion you would like. But if your activity is something that is changing on a millisecond by millisecond basis, as many functions in the brain do, then fMRI is just not your technique and you're just gonna have to wait a little while. The other problem with trying to figure out where the love center or the God center or the lying center is in the brain is that many areas have multiple functions. In particular, lying seems to have something to do with some areas in your frontal cortex that are involved in keeping many things in mind. And it turns out that if you're lying or if you're remembering the position of say three or four dots on the screen, the same area of the brain is more active. So remember three dots versus two dots. You get some contrast and you get some area that shows up as being more active for three dots than two dots. And if you show lying versus not lying, it's just easier to not lie. And so some of the same areas of the brain show up. So if you have a lot of confidence that that is in fact the lying area. And it's a bigger problem when you get things like emotions because there's a particular part of the brain called the amygdala. Who's wrote the amygdala? Excellent. You guys are up on your popular science at least. It's fantastic. This is a short list of things that have been found to activate the amygdala. In other words, you show this and you show something else. I mean, amygdala is more active to fear than to not fearful stimuli. Or to anger, things that might make you angry, then things that might not make you angry. And again, things that might make you angry, how do you define that? I don't know, what do you think? Let's show people with angry faces. Let's show someone you got in a fight with last week. Let's show, I don't know. And that is the part of the method section that you should really try and read. And that responsible reporters will report in the story if you actually want to evaluate the conclusions of the study. And also, it seems to respond to animals just for fun. I don't know. So, the other thing fMRI is really good at is that it takes a whole ton of measurements at once. It doesn't just measure one or two or three or four places in the brain. It measures 300,000 of them. And that's great if you actually want to read out the whole state of a person. If you want to read out their entire perceptual system, not just one little handful of neurons. And so some work from our lab, which I'm actually not going to show you because I show it in every talk, and I'm sick of showing you it's not my work. It was the work that convinced me to join the lab. But look up gal at lab.org and look up brain reading, and you'll find our video about reconstructing your visual awareness or what you were seeing based on your brain activity. And because we have so many measurements, we can do things like that because vision is really well mapped. But the point is that the state of the brain at any measurement, at any of these two second chunks that you've measured, is some pattern of activity like this. And you can look for patterns from one second to the next second. And you can look at an individual measurement and you can say, what was going on at each time? And you can say, why was it high and why was it low? And this looks like an easy problem if you consider just one little squiggly line like this. But remember that we have 300,000 measurements. And what we have actually looks more like this. So this is the cortical surface. This is the outside part of your brain. And this is what I am particularly concerned with in my research. We think that what's happening here is a pretty good reflection of what is going on in terms of what you're seeing, what you're smelling, what you're hearing, what you're thinking, dare we say. And these second to second variations are exactly the thing that my research is trying to figure out. Why did all of these signals go up and why did they go down? You can see the different parts of the brain are doing massively different things. So just to bring you a little bit here, the part of the brain back there, I don't know if you followed that whole flattening sequence but that's basically the back right side of your brain. So that's like right around back here. And that is visual cortex. That's where the brain is responding to visual images that you saw. And that seems to vary the most or vary the most reliably across all these images because what the people are doing is watching a video. And the emotional content of it, well, what do you think? Do you think you could rely if we came down to why one of these little dots turned from red to blue? I think so, then please go to grad school and you can go to our scientist. So, fMRI by no means the only way to measure the brain, there are a lot of ways to measure the brain, many of them cannot be done in humans. There are a lot of really fascinating techniques for making neurons glow, to manipulate the genetics of mice and fruit flies and their animals, to have them express things that every time a neuron's active, it glows. I don't think you guys have heard of the, used to be called the brain activity map with the BAM project, now it's called the brain project. You've heard of this? This is money for neuroscience, this is money for me. This is a good idea. It's a little bit of a vague idea because people, the goals of the project are not as well articulated as for example the human genome project or the human connectome project. So, the intent of this project is to create or one of the intents of the project is to create new ways to measure the brain, probably in animals, that will eventually be able to be used in humans. But if we're talking about humans and human brain reading, which again is what I care about, then one of the promising areas of research is in electrocorticography, which is actually sticking electrodes on actual brains and recording the electricity, which gives you about the same spatial resolution as fMRI. It gives you these couple millimeter chunks that you've put a little electrode to record over, but it gives you much, much better time resolution and you can measure things that happen on a millisecond to millisecond time scale. Unfortunately, well, perhaps fortunately, you can only do this on people that have epilepsy and are having surgery for epilepsy and to localize the spot where they have epilepsy, you have to put an array like this across the surface of their brain to determine which part of the brain is responsible for their seizures and which part you want to then cut out or burn out. But it turns out that there are enough people that have the surgery done, which is a very successful surgery these days, that you can get data and you can get data across pre-broad stretches of the brain. And this is incredibly valuable data and many, many neuroscientists these days are trying to cozy up to the MDs to figure out whether they can get access to some of these patients. And the thing that you may actually see in terms of brain reading over the next, I don't know, five to 10 years are better applications of EEG, which is actually a 100-year-old technology is by no means the cutting edge. But it's getting good enough and our signal processing techniques are getting good enough that we can actually make use of EEG scans in things like video games. So there are a number of things, including these people at MOTIVE, that are trying to use EEGs that you can just buy for a couple hundred dollars, stick on your head and use this input to a video game. And you can do things like use the force if you calm down. And so if you're trying to calm down, don't blink because what EEG actually measures are things like blink to the eye and muscle contractions in your forehead. So it's not really the most fancy brain reading. That's why I don't do EEG research. So, so what can we read out of the brain? Well, we can read out whatever we're creative enough to come up with a contrast for. We can read out differences. We can read out large scale patterns. Are you in any danger of having anyone read your brain involuntarily? Absolutely not. The best way to mess up an MRI is to move two millimeters and do it regularly. So just swallow a lot or sneeze and then pretty much the whole thing is shot. So I guess I bring you a message of comfort. And I hope that I have taught you a little bit of skepticism. So the title of my talk was neophrenology or window-engaging movies of our minds. And that last was actually a quote that was attributed to my advisor after one of our brain reading papers came out. And it was attributed to him by the press office at UC Berkeley. Somebody wrote it and said, you said something like this, can we say this? And he was like, no, no, window-engaging movies of our minds, really corny and it's not really an actual reflection of our research. But people just want that, they want to believe. There's this vertigo, like looking at brain reading and saying, oh, I want to fall in love with you. I want to fall into this, this has got to be true. And we're, you know, and that's basically the end of my talk is we're coming here.