 on a train and it was quiet, fairly empty, and then at this one stop a bunch of people got on and as they were all sort of inching along trying to get to their seats they did a double take and exclaimed, you're Dr. Villaner S. Ramachandran. And the guy was like, yes, yes I am. So stoked. This guy is a neuroscientist. He asks fascinating questions and he's a phenomenal storyteller. He seemed a little bit surprised to be recognized and I guess this was before he was publishing New York Times bestsellers and things. I think he asked me if I was a student at the university where he does research. I'm like, no, I'm a programmer. And then I blurted out, I live with a synesthete. And then the encounter was over. So this is somewhat less of a non sequitur than you might imagine. Ramachandran did groundbreaking work on synesthesia. So synesthesia is a little bit of cross wiring that happens in the brain. A few people have it. It's kind of rare. It's genetically determined so you either have it or you don't. And what happens is that when you get a sensory perception that's triggered, the signal will bleed over into a different part of the brain and trigger a secondary perception. So you might activate a pain receptor and feel pain and that pain will have a very specific color. Or you'll hear a sound and it might have a visual texture or some particular taste. So some people with synesthesia can use this extra layer of perception to help orient themselves. And this is the case for Priscilla Dunstan. She's an Australian opera singer. She's thin aesthetic. And her synesthesia gives her an extraordinary memory for sounds. So about 15 years ago Priscilla Dunstan had a baby. Now I've never had a baby but from what I've heard, the first few weeks and months can be incredibly tough. The baby cries. You're sleep deprived. You have no idea what the baby needs. You feel helpless. You feel isolated. And people promise parents that they'll start recognizing the baby's cries and understand what that baby needs. And Dunstan, despite her extraordinary memory for sound, was not getting it. She wasn't hearing what these cries meant. She couldn't find any resources that helped figure it out. And finally one desperate morning, she decided to figure it out for herself. So she started making a log of all of her baby's cries and crying sounds. And slowly she came to the realization that she was right. Once the baby is crying, once it's wailing, it's just a cry. It's just one big upset. But she did discover that at the very beginning, before the baby is really wailing, there are some fussy pre-cry sounds that are extremely distinct. And so as she listened and experimented, she realized that I'm hungry sounded very different from I have gastrointestinal discomfort. And she eventually reliably assigned meaning to five different pre-cry sounds that her son made. And so this was pretty transformational. She was getting a lot more sleep. But she didn't really think much of it beyond kind of the sense of like, I'm really good at this. My son and I can communicate really well. And then she started getting out of the house more. And to her astonishment, she started hearing other babies that were hungry and tired and needed to be burped. And they seemed to be expressing this in ways that were remarkably similar. So she was hearing it and others were not. When we talk about expertise and mastery, we tend to talk about knowing facts and being able to explicitly verbalize concepts and perform sequences of steps. In the beginning, we know things and do things very deliberately, slowly, laboriously. And then over time, we tend to get faster and it becomes more effortless. And one part of expertise is this transfer from the slow, deliberate part of the brain to the automatic, effortless part of the brain. But an expert isn't just a faster novice. One of the characters in the Kingkiller Chronicles observes that playing music is a little bit like telling a joke. Anyone can remember the words. Anyone can repeat them. But making someone laugh requires something more than that. Telling a joke faster doesn't make it funnier. An expert seemed to have this magical thing. We call it insight, judgment, intuition, brilliance. And they get that way because of some unarticulated range of experiences. Practice maybe. Seasoning. The enigmatic passage of time. This mysterious quality of expertise is crucial in almost every field. Architecture, archaeology, nursing, electronic circuit design, power plant operations, it's a factor in almost every skill. So whether us experts are judging livestock or analyzing terrain based on aerial photographs, they're relying on this gut sense. This ability to make snap judgments. To just know. So in one experiment, researchers gave terrain analysts two minutes to look at an aerial photograph. It generally takes a few hours to analyze this. So two minutes is not a whole lot. And after two minutes, this one engineer, as he started his debrief, started with an offhand comment that whoever got sent to the area needed to be ready for certain types of bacterial infections. And the researcher was like, what? You can see bacteria on a picture taken from 40,000 feet. And the engineer was like, well, the picture shows a tropical climate. And the contour of the tree canopy, because the vegetation is mature, reflects the underlying soil. And since that soil layer would be relatively thin, that would also reflect the underlying bedrock, which appeared to be tilted, intubated limestone. And so that would also determine the patterns to the streams and the ponds. And there seemed to be this pond that didn't have a major distributary running away from it. So given the climate and the vegetation and the presence of stagnant water, bacteria was a sure bet. Experts have such a hard time explaining what they do or how they know they just do. Their skill is consistent, it's reliable, it's reproducible, and it's kind of mysterious. So if you can't explain it, and you don't even necessarily know what it is, how do you teach it? How do you teach seasoning and perspicacity and judgment? How do you teach intuition? Now traditionally we're pretty good at training the deliberate part of the brain. We write tutorials and coursework and we devise drills and practice problems and we assign homework and we're not so good at training the automatic part of the brain. We just kind of let it figure stuff out as a result of this wild, chaotic diet of signals that were fed our whole lives. Now it turns out there's a whole field of psychology that's dedicated to understanding the conditions under which the brain figures all of this stuff out. It's all started in the 1960s with a researcher named Eleanor Gibson. So she designed this delightful experiment that basically debunked a bunch of nonsense that scientists believed at the time about how the brain learns things. And the experiment illustrated unambiguously the fundamental building block of our ability to develop accurate SNAP judgments. So she'd start by showing research subjects a meaningless squiggle and then she'd tell them that she was going to show them a series of squiggles and would they please identify every squiggle that matched this reference squiggle exactly? And then she'd flip through them one by one and the research subjects would make their guesses and she wouldn't give them any feedback. After they'd gotten through the entire deck, she would start over. This is the reference squiggle. Please identify all of the squiggles that match. There's still no feedback. By the end of the third time through, every research subject had correctly identified every single target squiggle. So what the brain was doing was it was discovering different meaningful squiggle dimensions. So as they were shown the squiggles they'd suddenly notice that some went clockwise, others went counterclockwise. Some squiggles would have three or four or five spirals. Some were perfectly round, others were kind of squished. Our brains are constantly going through this process of differentiation, figuring out which characteristics matter and which don't. And as the brain discovers which dimensions are important, it'll start focusing in on those dimensions. It'll pay more attention. And we begin to make finer discriminations. Our perceptual resolution increases. So photographers gain a richer experience of light. Musicians gain a richer experience of sound. Industrial tasters gain the ability to evaluate mayonnaise along 14 different dimensions of flavor. And this field of study is called perceptual learning. So perceptual learning is typically broken down into two very broad categories, discovery effects and fluency effects. So each effect describes a way in which the perception of experts differs from that of novices. Discovery effects are about how we perceive and extract information. So for example, novice drummers cite, read music, note for note. They pay attention to how long each note lasts. Experienced drummers don't read note for note, they read beat for beat. Each beat ends up having a distinct rhythmic figure. And then there aren't all that many rhythmic figures that occur for any single beat. So the musician starts recognizing that figure as a single coherent idea. This is what the researchers call units. Novices see lots of low-level, unrelated pieces of data, and experts see chunks and patterns and higher-order relations. In video games, you'll often get like a heads-up display, and it'll flash and throw a bunch of data at you. Some of that data might be irrelevant, but if it's really in your face, you're going to instinctively react to it. As you play the game, as you get used to the game, your brain will start attenuating loud, irrelevant data and automatically focus in on some of the more subtle but relevant bits of detail. So the technical term for this is selectivity. Novices pay attention to both relevant and irrelevant data, and as we gain expertise, our brains amplify the relevant data and attenuate or completely filter out irrelevant ones. And this happens before the signals even reach the part of the brain where we're aware of perceiving things, which means that experts often don't even notice the irrelevant data. They can't even tell you that it's there. So those are the discovery effects. The fluency effects are not about how we extract information, but how efficiently we do so. And again, there are stark differences between experts and novices. So experienced pilots can determine aircraft attitude and situation with a bare glance at their instrument panel, and inexperienced pilots will read and cross-check their instruments very, very carefully one instrument at a time. This is called search type. Novices process things serially, whereas experts have a much greater tendency to process things in parallel. Experts' photographers will spend a lot less time trying to compose a shot than inexperienced photographers do. I talked to some photographers to figure out, like, why? And they were like, well, it's because you quickly find the focal point of the shot, and you figure out whether the composition balance is out, and also you realize that you can crop it later. So this is about speed. Novices process slowly, and experts extract information quickly. And whether we're talking about sight-reading drum notation or cross-checking aircraft instrument patient panels, pretty much any perceptual skill, novices are going to be drained after doing it. For experts, that effort will hardly register at all. And this is what they call a tension load. It requires a serious amount of cognitive resources for a novice to extract information, whereas experts do it effortlessly. So that's the basic science. Discovery effects are about patterns and filtering incoming signals, and fluency effects are about processing in parallel quickly without draining cognitive resources. Where this gets interesting is when you take these basic ideas and then figure out how to use them to explicitly train intuition. There's a cognitive scientist out of UCLA named Phillip Kelman who spent the past 25 years exploring this question. And one of the things that makes Kelman's work so fascinating is that he picks complex skills that address actual real-world problems. So some of his early stuff was inspired by the fact that every year you have pilots that land at the wrong airport or get lost flying cross-country. Now, this was in the early 1990s. You might imagine that technology would have completely solved this by now. Surely pilots have iPad apps that help them figure out where they need to be. I checked. And in fact, there are iPad apps. However, in 2014, the National Transportation Safety Board issued an advisory that basically reminded pilots to take care to land at the right airport. So apparently this is still a problem. There's a skill that pilots have, and the better they are at it, the less likely they are to get lost flying cross-country. The skill is visual navigation. And this is where you look out the cockpit window and you eyeball the terrain and then you look at a map and you figure out if you're in the right place. Pilots develop this skill over time through experience. But they don't actually necessarily learn it particularly well. So in Kelman's pre-trial assessment, he had, he tested pilots with between 525 hours of experience. He'd give them video of some terrain and then have them choose one of three locations on a map that corresponded to that terrain. And the pilots got the right answer 50% of the time. Not very good. Kelman then put each pilot through three hours of perceptual training. It was made up of brief interactive trials, 20 seconds of video, and then they were shown a map with three locations marked on it and they had to choose the right location. By the end of the experiment, pilots were getting the right answer about 80% of the time, up from 50, not too bad. Their accuracy also, it wasn't just their accuracy that improved their reaction time, also improved. So at the very beginning, they were taking over 30 seconds to choose a location and still getting it wrong 50% of the time. At the end of the experiment, it was taking them less than 15 seconds to choose a location. This is pretty dramatic. Kelman also had a second group of research experiments who were non-pilots. So the non-pilots got the same perceptual training for three hours as the pilots did. And these naive subjects, as they're called, after three hours were outperforming the pilots pre-training. So they got to 65% accuracy and they were taking less than 20 seconds to pick their location. Another interesting problem that Kelman and his team tackled was teaching fractions to middle schoolers. So teaching fractions is hard, they're hard to learn. We don't really give students a very good mental model for how they work or understanding, like, why they work. The kids are just kind of blindly accepting that there are rules and they seem kind of arbitrary and so I'm just going to pick one that seems like it might, you know, match the given problem. And it's a real mess. So a typical word problem might go something like this. Ten alley cats caught five sevenths of the mice in the neighborhood. If they caught 70 mice, how many mice were in the neighborhood? Here's another one. We ordered computers for ten classrooms. Five sevenths of the computers came with blue mice. How many mice were blue if there were 70 mice in all? The numbers are the same in the two problems. But the underlying structure of the problem is fundamentally different. The first question asks, find the hole. The second question asks the student to find the part. If you are not sure whether you're being asked to find the hole or find the part, you're going to have a hard time getting the right answer. Kelman's training module for middle school students was defined specifically to help them tell the difference between these two structures. So they designed interactive trials and those trials consisted of mapping a problem in one representation to the same problem in a different representation. So they might be given a word problem and then shown three different fraction strips and asked to pick the diagram that correctly represented the problem. So the students weren't asked to find an answer. They were just asked to recognize what problem they were being asked to solve. The students were given pre and post trial assessments where they were asked to solve problems. Before the interventions, the students were getting the right answer about 40% of the time. After the intervention, they were getting it right 60 to 70% of the time. And they then tested the students several months later and these scores held. This learning was permanent. So Kelman's work shows that not only can you deliberately train intuition, you can compress the learning that happens in real life over the course of months and years into a very, very short time frame. So based on his work, I have put together an amateur's guide to designing perceptual learning training materials. The basic component here is brief classification episodes. And by brief, I don't actually mean instantaneous. Like if you can puzzle something out logically and deliberately, then go ahead and give your brain the time to do so. Sometimes that's not going to be possible. For example, there are people who can reliably determine the gender of a day old chick. They can sex up to 1200 baby chickens in an hour with 97% accuracy. And they cannot tell you how they know. They just do. So for those of us without this skill, a chicken will have to be about six weeks old before we can determine whether it's a boy girl, a boy chicken or a girl chicken, which means that we've just spent six weeks feeding a boy chicken who's never going to lay any eggs. This gets expensive at an industrial scale. So the way to train new chick sexers is to pair an expert with a novice, have the novice guess, the boy is a girl, and then the expert tells them yes or no. That's it. You do it over and over and over again until the novice is getting 90% of their guess is right. And they're still feeling like they're kind of guessing. But they actually really kind of know. So it's crucial here that the learner make active judgments. Showing someone a chicken and telling them the gender is not going to teach them anything. Also, notice the role of feedback in all of these examples. The pilots were told the correct location on the map after they had made their choice. And in the case of the students, it was even more interesting. Not only did they get feedback immediately, but when they got something wrong, they'd get the correction and then their wrong answer would be turned into the next question, giving them three options in a different representation. So that's the basic format, short interactive trials. Now, for this to actually work, you need a really good data set, it needs to have a huge number of examples and there should be absolutely no duplicates. The brain needs a large amount of complex variation in order to be able to detect the underlying invariance or the diagnostic structures. This variation should include not just the relevant features, you want to systematically vary the noise and the distractors and the irrelevant characteristics because if you don't, the brain is going to accidentally correlate incidental features with the structure that you're trying to learn, which is basically the recipe for teaching people how to be biased. In order to help increase people's perceptual resolution, start with examples that are very different from each other and then slowly decrease the contrast, incorporating examples that display more subtle differences. So that's how you would deliberately design a perceptual learning training module. Now, to illustrate what this might look like in practice, let's go back to Priscilla Dunston, the synesthetic opera singer. She believes that all babies regardless of culture or genetics make the same pre-Christ sounds. She claims that this has to do with infant reflexes and what these different reflexes sound like when you add sound to them. So I was explaining this to one of my colleagues the other day, and he was like, you're going to make these sounds during the top, right? I was like, no, I'm not. He was like, yeah, no, you are. So here we go. One of the things is a baby who is hungry is going to have a swallow reflex. And so if you add a sound to the swallow reflex, it's going to sound something like this. Now, if they're uncomfortable, they're going to trigger a different reflex and sort of uncomfort reflex. Their clothing might be too tight or they might be too hot, and it's going to sound a lot more like this. And I can't even do the one where they're tired where it triggers the yawn reflex because I can't even recognize it yet. So there are five of these sounds. Dunston is an opera singer. She's not a scientist. There's absolutely no scientific basis for her claim. She has never tried to prove her hypothesis wrong. She did sign up like she was working with some university to do a study, and then she canceled it because she wanted to get all of this out to parents as soon as possible, which is heartbreaking. So assuming that it's true, we would need to create a data set that consists of audio and video of lots and lots of babies from all over the world, different ethnicities, different cultures. You might have the same baby fussing in lots of different situations, but you wouldn't get the same recording twice. You need a bunch of distractors and incidental features so the cries would be recorded in different environments, different quality. Ambient noises would vary. And then the video snippets, you could record the baby from lots of different angles under different lighting conditions. And then you could put together all of these interactive trials where you present someone with an audio or a video clip and then give themselves several choices to pick from. Now you might pick three different choices varying from those five different alternatives or you might always show the full set of five choices. So this is interesting, but only relevant to very few people here. So a more relevant question is how all of this applies to programming. A few months ago, one of the people I met her asked if I could help debug some JavaScript. He had followed this step-by-step tutorial to make a game in the browser. He explained that the game kind of worked, but mostly didn't. And he'd been trying to figure out what was wrong for two days. And so I suggested that we go back to the very beginning and just walk through the tutorial together, start checking stuff from the ground up. So he was like, show me your HTML. He was like, here, see, it's exactly the same. And I was like, right there, missing quotation marks. He was like, but I checked it out so carefully. So he added the missing quotation mark. I immediately pointed out another. He fixed it. And then I realized that I was seeing something that he wasn't. So I asked him to look at the colors at the top half of his screen and then had him delete the quotation mark that he had just added and then look at the colors again. And he was like, oh. And then he immediately found all the missing quotation marks in his HTML and the game was working. Newbies spend an inordinate amount of time stuck and flailing because of trivial syntax errors. So deliberate perceptual training could cut down on the painful weeks and months where you get stopped in your tracks by a missing brace. But the more interesting instincts, I think, are much more complex than this. And they're a lot harder to pin down. So some people have this uncanny ability to home directly in on a problem. Or they'll look at some legacy code and suggest an abstraction that's just perfect. It's one of those things that you'd never have discovered yourself, but which is obvious in hindsight. And some people have a particularly keen eye for code smells. And they know how to change the code safely. Or which design pattern might make sense in a given situation. And we have some literature about the patterns and diagnostic structures to look for, but for the most part, when people do these things, it feels a little bit mysterious. It's a great example of something that's heavily dependent on our perceptual ability is code review. And the amount of experience you have influences the type of things you point out. The programmers with less experience will tend to focus more on low level standalone nitpicky problems than tax and variable names and long methods and long classes. Programmers with more experience tend to focus on much larger patterns that are harder to detect and comprehend for inexperienced programmers. All of these things are complex. It would be exciting to develop a data set that can help us accelerate and optimize the learning process to develop this gut sense. But at the moment, it's not really clear what the patterns and the invariance and the diagnostic structures are. So instead of urging you to go curate or generate a good data set to help learn these things, I will simply ask you to be aware as you interact with code and with other humans, with developers who are more experienced than you or less experienced than you, notice when there is a perceptual component and ask yourself what that component might be. Thank you.