 All right. Hello, Duncan. Thanks so much for joining me. How are you doing today? I'm great. Thanks, Chris. Thanks for having me. Yeah, absolutely. I just finished reading the book for a second time. Absolutely love it. It has opened my eyes to a lot of things in this world and I'm going to talk to you a little bit about some nihilism that I've gotten in from reading your book, but for those who have yet to meet you, can you talk a little bit about what your field of research is? Sure. So my name is Duncan Watts. I am a professor at the University of Pennsylvania and I'm actually in three different departments there in the computer science department in the Annenberg School of Communication and also in the Wooden School of Business in a department called Operations, Information and Decisions. And so that's a little unusual to be in so many departments at a university. And the reason is that my field, which I usually refer to as computational social science, is an intrinsically interdisciplinary field and is something that I kind of stumbled on many years ago in the course of my research. I, my undergraduate degree was in physics. My PhD was in engineering. I was a sociology professor at Columbia back in the early to mid-2000s. And I realized at that point that many of the questions that I was interested in to do with social networks and how things spread and how people influence each other and how collective behavior emerges out of individual decisions and actions and interactions was going to be revolutionized by the web. Then, you know, at that time, you know, Facebook was still a startup company and, you know, blogs were still a thing. And, you know, there were, you know, email had really just sort of become ubiquitous. So there was an enormous kind of technological revolution going on in the outside world. And it occurred to me that that revolution was going to fundamentally change how we do social science, basically, because it was changing the kind of data that we had access to and therefore the methods that we would need to make sense of it. And so out of that disruption came this new field of computational social science. And, you know, as the name suggests, it is sort of intrinsically sits in between, you know, more traditional fields like computer science and sociology, economics, political science. And so that's really where I sit these days and how I identify. Yeah, yeah. And so it's so interesting, just all the research, you know, the title of the book, right? I love the title. I don't know if you specifically came up with it, but it's just, it's perfect, right? So everything is obvious once you know the answer. So, you know, as the title suggests, you know, there's a lot of people where, you know, we get a result, we look back and say, oh, of course that was going to happen. But with, you know, with you getting into all this and realizing how the internet was going to change things, when you started looking at all the data and the research, like were you, were you surprised at all by what you were finding with the way trends work and the way we kind of interact? Or did you see any kind of shift with the internet? I'm curious. Well, so the title of the book, thank you. I'm glad you like it. I ran through dozens of titles before I stumbled on that one. Really reflects something that I've experienced, you know, in my career, going from a background in physics and engineering to sociology, where, you know, when you're sort of studying mathematical models, people think that sounds like hard science and, you know, you must be very smart and how can you even think about those things? Once you start doing sociology, then almost anything that you find will sound like something that people thought they knew beforehand. You know, it's pretty unlikely actually that you will come up with a result that nobody had ever suggested, because, you know, human experience is so vast and we have a lot of it collectively. So, you know, everything sounds like something that you already know. But, you know, actually, you know, nothing is really obvious is sort of the alternative title that I could have had for the book, because, you know, in my experience doing research, you know, I always go into these projects with some kind of hypothesis with some sort of prior about what I think I'm going to find. And almost always, I don't find it, right? Almost always something is different from what I had imagined. Usually things are more complicated. Usually there's some sort of, you know, contextual dependencies. You know, usually the result is less striking than I had wanted it to be. You know, but it's not that you find something that wasn't what you would expect it and it's totally shocking and no one has ever imagined this. It's just like another thing that, oh, maybe if you would, you know, again, once you find the answer, you think, okay, sure, I can make sense of that also. And so, this ability to make sense of the outcome, independent of what the outcome is, right? You can do it equally for opposite outcomes in almost any circumstance. You know, it came interesting to me itself as a social phenomenon, right? Yeah. That, you know, it was initially it was irritating because I would do all this work to do some research. And I would find something that I thought was super interesting because it wasn't what I had expected. And then someone would say, well, that's obvious. Like, I could have told you that. And I would say, well, but you didn't. And also, you could have found something else. And it's still important to go measure things. You know, we don't, you know, in physics and biology and other kinds of, you know, science, you know, sciences that people accept as real sciences, you know, like everybody knows that light travels really fast. But you wouldn't say that it was pointless to go and measure the actual speed of light. That's something that we would want to know, right? Yeah. So lots of things in physics and lots of times in physics and biology and medicine, like measuring things is really important because, you know, it makes a difference whether something is, you know, you know, big or, you know, a thousand times bigger. So the same thing is true in the social sciences where measuring things is important because we want to be able to, you know, come up with theories and models that make predictions and have policy implications. But it just sort of never quite seems like the same thing because this is stuff that we all think we know. So this sort of sense of, you know, social science, not being rocket science is one that I, you know, found frustrating at first, but eventually realized, look, this is, itself is an important thing, right? The way we think about what we're doing when we're explaining the world can really interfere with what we're understanding, right? If you think you already know everything, then you don't have the incentive to go out and do a careful job of figuring it out. And so that's why I wrote the book is to really try to get people to sort of unlearn what they think they've learned about the world and start over again with a fresh set of eyes about, you know, what really is obvious and what's not. Yeah, I think what's interesting, one of the reasons I love the book, you know, I bring so many authors on here who research and, you know, do so many things. And like your book is something that we can all relate to because we all have that friend, right? Or multiple friends where we tell them something and they're like, oh, I could have told you that, right? Like those know it all type friends and your book explains, you know, how we do that, why we do that. There was one famous study that you talk about where you could you could probably explain a lot better than I can where they talked about like, oh, these people from like rural America are more likely to, you know, be better in war and stuff like that. And then they and they're like, oh, yeah, I knew that. And then they say, oh, actually, we were lying. It was the city people. Can you can you kind of describe what that research sure. So this this is a this actually was a book review of all things that was written by a very famous sociologist called Paul Isisfeld back in the 1950s. And the book that he was reviewing was called The American Soldier. And it was actually a report by a bunch of sociologists and anthropologists who had undertaken this enormous research project, where they had interviewed several hundred thousand members of the US Army and asked them about their experiences during World War Two. And this report was reporting on what they found. And Lazarusfeld is writing this review of the book in the American Journal of Sociology, and he uses this book review as a sort of excuse to make a larger conceptual point about social science. And the way he does it is very clever. He recounts, you know, he tells the reader, you know, here's some of you know, here's like half a dozen or a dozen or so results that that the authors found in the study. And then he steps back from the review and addresses the reader directly and says, you know, you may think that that this is obvious, right? But of course, you know, so one of the results was that that, you know, men from rural backgrounds did better in the army than men from cities. And he says, look, you know, this may sound obvious. Like, of course, people from rural backgrounds did better, you know, they're used to, you know, working outside and doing hard physical labor and, you know, you're getting up at dawn and going to bed at, you know, at the, when the sun sets and, and, you know, sleeping on the ground, all this sort of thing, like, why, why do I need this big expensive study to tell me what I could have figured out using my common sense. And then Liza Spell flips it on the reader and says, but actually all the results I told you were the exact opposite of what the study really found. It was in fact men from cities who did better in the army and not men from rural backgrounds. And of course, had you been told that answer, right? You also could have reconciled it with other things that you thought you knew. You could say, well, you know, of course men from cities did better, you know, they're used to working in large vertically integrated organizations with strict, you know, chains of command and hierarchies and they're used to, you know, wearing, you know, a suit every day, which is kind of like wearing a uniform, you know, once again, you could, you could reconcile that with what you imagine to be your common sense. And so then Liza Spell makes his real point, which is that when, when every answer and its opposite seem obvious, then there's a problem with obviousness. That's the real problem, right? And that I think is every bit as true today as it was when Liza Spell wrote that review and when I wrote the book, which is now 10 years ago, that this is really something that we struggle with. That we're so good at, you know, sort of after the fact, telling ourselves a story about why things happened that makes it seem like we've made sense of it, right? And then we're happy with that. And then we just kind of move on with our lives thinking that we've understood things that we really haven't understood. Yeah. And, you know, I'm curious because it's so funny as, as you were just retelling that, and even though I know the results, I know what it was all about, I feel my brain just filling in this story and painting a picture, right? These like rugged, like farmer guys, like, you know, being tough and tough and being ready. So, you know, I'm really, you know, interested in, you know, self-deception, right? And evolution of psychology. Why do you think our brains have this kind of natural tendency to just start filling in these gaps and make us feel like we already knew rather than, you know, this kind of curiosity? Because I also read books about how important curiosity is and how curiosity drives innovation and moves us forward as a species. But on the opposite side from what you're talking about with this research, we also have this kind of just this hubris that we know things and we already knew the answer. So why do you think that we're designed or wired to just do that naturally? So I have a couple, I don't know, is the short answer, right? I mean, you know, we can't go back to sort of, you know, 50,000 years ago on the Savannah or 200,000 years ago or whatever it was and, you know, do experiments on early humans. But I have a couple of thoughts. The first one is that believing that you know how the world works is actually pretty adaptive under a wide range of circumstances, right? For the following reason that if you, you know, if you feel like things make sense and you have a, you know, a sort of grip on them and that you know what's going to happen next, that gives you the confidence to, you know, go out and take risks and, you know, and invest in, you know, your labor and things, you know, feeling confident that everything isn't just going to fall apart, you know, the next second, right? And even if you're deluding yourself most of the time, that confidence does enable you to do things that you otherwise wouldn't do, right? So people often talk about how entrepreneurs systematically overestimate their probability of succeeding, that they in some sense have a very poor understanding of probability. And you might think that that would be a bad thing for an entrepreneur to have, but it turns out it's helpful because if they really understood how likely they were to succeed, they just wouldn't do it, right? Yeah. That no one would take that kind of crazy risk. And so you have to in some sense believe that you know more than you really know to go and do things. And even if most of them end up failing, some of them succeed and we all benefit from that, right? And so I think something like that is true of common sense that if we really internalized how little we understand about the world, probably most of us couldn't get out of bed in the morning, right? It would be just overwhelming, right? It would be terrifying. And so in some sense, you're fooling yourself to kind of, you know, give yourself the confidence to do things. And so I think in that sense, it is an extremely adaptive and useful mechanism, right? And, you know, you know, I'm not and in fact, in the book, I'm pretty clear about that. Like in general, I think it's, you know, we're fine. Like I'm not saying that there's anything fundamentally wrong with people. But what I am saying is that same confidence, which works so well in everyday life, can get us into trouble when the decisions that we're making are not decisions about everyday life, but the decisions about, you know, policy or business that are affecting, you know, millions or hundreds of millions of people, right? So that, you know, so that this sort of, you know, epistemic hubris that is very helpful to our, you know, survival can be harmful, you know, when it's taken out of its evolutionary context and, you know, used in the kinds of, you know, modern decision making situations that come up in government and business and so on. So that would be my first argument. And the second one, which is related, is something that I did know about when I wrote the book, but I've learned about subsequently. And it's a wonderful essay by this psychologist who maybe you've even had on the show, Alison Gopnik, who studies child development. And she has this fantastic paper called Explanation as Orgasm. And the premise of the essay is that our brains are causal inference machines, right? Particularly, so think of babies. I have a couple of young kids and it's been fascinating to watch them, you know, grow up because, you know, when they're born, they don't know how anything works at all, right? They don't understand what they are, they don't know what their limbs are for. They're always like whacking themselves in the head, which is why you have to, you have to swaddle them. They don't understand gravity. They don't understand like that, you know, the knife, you know, they're gonna stick themselves with a knife or jump in the oven. I mean, they'll do, you know, they have to constantly be prevented from killing themselves because they have no idea how anything works. But they figure it out because they're always experimenting, you know, on little kids, little babies, they get go through the stage where they love dropping things on the ground because they're just fascinated by the idea that, you know, they let something go and it falls, right? And so they're kind of developing these causal models of reality through experimentation. And they do it by sort of having these kind of hypotheses about what is going to happen and they go on, they test the hypothesis and then something different happens and they learn and they build better and better models. And that is such a useful thing that it sort of doesn't matter that it's imperfect, right? And so here's where the orgasm analogy comes in, right? Is that, you know, just like, you know, sex leads to reproduction. But, you know, sex and so orgasms are part of sex, but they're not the same thing as reproduction. It's possible to have reproduction without, you know, without some people having orgasms at least. And it's certainly possible to have orgasms without reproduction, right? So there's a pretty sort of loose, you know, correlation between the two. But they're, but at some level they're correlated, right? And so evolution has kind of worked out this trick where it makes reproduction fun because reproduction is useful, right? And so it's a sort of messy correlation and they don't always go hand in hand. But even, you know, over the long run, on average, orgasms lead to reproduction, right? And in the same way, we have this innate seemingly innate desire to explain things, right? And the way she talks about it is the ah-ha pair where you sort of something puzzling happens and you go, hmm, I wonder how that works. And then you go and investigate and you do a bunch of experimenting. And then you get this insight that makes you go, ah-ha, I got it. And as I'm sure you know, that's a great feeling. Like it's sort of like the equivalent of an orgasm, right? That you have this like wonderful moment where you're like, ah, it makes sense. I got it. It all comes together. And that drive is to causal inference and causal modeling what orgasms are to reproduction, right? It is this sort of loosely correlated experience, phenomenological experience that makes learning causal models fun, right? And little babies do it and you can see the delight when they just like figure out a new thing and they're just so pleased with themselves. But adults also experience this. I mean, there's so many stories of scientists, you know, having the ah-ha moment, eureka moment. This is why people become scientists because they just love the idea of being, you know, having this sort of moment of clarity where things come together. But what's important to realize is that that moment of clarity doesn't mean that what you just realized is true. Just because you had that experience of like everything making sense doesn't mean that what you just thought is true. In fact, conspiracy theories, the perfect example of that, where they're popular for exactly the reason that they make people feel like everything makes sense. But they're totally crazy, right? So there is just as with, you know, orgasms and reproduction, the ah-ha moment and truth are sort of, you know, not perfectly correlated, right? But over the long run and on average, wanting to figure out how things work is sufficiently correlated with actually figuring out how things work that is a useful adaptive thing. And so this is sort of another take on it, which might, which helped me make a lot of sense of things, right? Is that we, you know, we have these sort of built in innate drives that are extremely useful for generating causal knowledge about the world. But they also end up generating a lot of other knowledge that, you know, might just be total garbage, right? And from a revolutionary perspective, it's worth the trade off, right? Doesn't matter if we have a bunch of like spurious ideas because, hey, you know, we're able to survive, right? Yeah. And but once again, you know, we've kind of moved into this stage of, you know, human history where we're so far beyond anything that we evolved to do that that's a little bit irrelevant now. Like we really have to sort of think about, well, you know, which of our tendencies are adapted to the kind of world that we live in now and which of them are problematic. And so I think, you know, we definitely, you know, we can tell a story at least about why we have these tendencies, but and, you know, on average, on balance, they're fine, but and this is sort of really the big but of the book is we should be careful about what they, you know, what they can result in the kinds of misconceptions that they can result in when it comes to thinking about complex social systems. Yeah, yeah, absolutely. And so much, so much is that there's just I'm getting that kind of explanation is orgasm just hearing this, you know, that's one of the reasons I love to read so much because I just have all these questions and I'm like, I bet there's a book about it, right? For example, you know, in a second, we'll talk about some of the things that you've both had me look at life differently with, but I read so I can get these explanations. We hate like I come from a marketing background, right, and do content marketing. And there's something where they call it like open the loop, right, where you want somebody to click on an article or come in because we hate not knowing we want an explanation, but like you said, this could lead to conspiracies and some people just stop, which is another reason why I think I like to read a lot of books on skepticism. You know what I mean, where I stop and I question like, am I just stopping at the first answer I get or am I questioning it further and seeing if that makes sense. But, you know, with your first kind of theory about this, like from the, you know, evolutionary aspect of certainty helps us get out of bed and not questioning everything. That's one of the main reasons I wanted to have you on here, right. So, because this is, this is something I think about a lot. For example, one of the reasons I found your book and your research so interesting, like I've been a content creator, I've been in marketing, I've, you know, I've done so much just over my, my years, right. And I'll give you an example of YouTube, right. So when I, when I started on YouTube, I started looking at a bunch of different creators that help YouTubers and everything like that. And what they'll do is just massive survivorship bias that I see everywhere. They'll pick out the top, the top YouTubers and say, this is what they did, right. Here's what they did that made it successful. But then I started noticing it everywhere, like I write, I write books, I'll read books about writing, and they'll pick out the best examples. And they'll say, here's what they did. And I'm like, but you're picking out the best ones. And then you're working backwards to explain it. And it doesn't discuss all of these other things. And, you know, I think before we dive into that a little bit further, one of your, one of your famous pieces of research is the music study that you did. And I just really wish that everybody knew about this because it blew my mind. And I'm like, okay, nothing matters. Everything's random. But we'll talk about quality in a second. But can you kind of summarize what this study was that you did with participants and ranking music? Right. So, you know, it was motivated by, you know, the kind of experience that you've had where, you know, when people try to understand success, what they tend to do is they look at successful things. And then they just describe them in effect. Right. So if you, you know, why is Harry Potter successful? Well, it's, you know, in effect, because it's more like Harry Potter than anything else, right? Oh, people love stories about boy wizards. People love stories about, you know, England, right? Boarding schools, you know, and the explanation turns out to just be a description of the book itself, right? So it's not, it sounds like it telling you why something happened, but it's really just telling you what happened. It's like a circular reasoning. I think you described it. It's totally circular reasoning. Yes. And once you, once you, and I talk about this a lot in the book actually, actually use the example of the Mona Lisa as a sort of the main anchor. But once you see this patent of people saying, well, you know, X succeeded because X is more like X than anything else, you know, the Mona Lisa succeeded because it's more like the Mona Lisa than anything else, you really see it everywhere. Right. These are just descriptions masquerading as explanations. And people might say, well, what's wrong with that? Like in some sense, we have some evidence that that thing succeeded. So we might think that things like that succeeded. But this is what statisticians call selection on the dependent variable, right? That you're trying to explain, you know, something like success here. But if the only things that you look at are successes, then your data is biased, right? So what you need to do is look at things that are successes and things that are not successes and see which are different, right? So it may be true that successful people wake up at 4am, which is something that you, you know, every so many years, somebody writes an article about that, right? But actually, lots of people wake up at 4am, and most of them are not billionaires, right? So if you started waking up at 4am in the morning, you wouldn't suddenly become a billion, right? So it's sort of a, it's a, it's a, it's a, you know, not a very hopeful piece of advice. It's probably not even a necessary thing, but it's certainly not sufficient to become successful. So how would you actually figure that out, right? You would need to run experiments, right? You would need to, you know, go back in history and, you know, rerun history, you know, many times and see, you know, is the Mona Lisa always the most famous painting in the world? Is Harry Potter almost the most successful, you know, children's book in the world? And if you ran it a hundred times, and every time Harry Potter was the most successful, you might say, well, we still don't know exactly why, but we're pretty confident that there's something really special about Harry Potter, and it would succeed in any version of the universe. But if you ran history a hundred times, and the only universe in which Harry Potter was successful is this one, and every other time it was some other book, you might think, maybe it was just like, right? Maybe it's some, you know, complex thing going on, and it just got, you know, picked up by somebody and then got picked up by somebody else and, you know, there was a cascade of events where everybody was going to read it because other people were reading it. And in fact, we have lots of mathematical models in, you know, computational social science that have exactly that dynamic, right? And there's all kinds of names for it, like the Matthew effect or cumulative advantage or rich get richer phenomena. So there's, it's certainly theoretically possible for that to happen, that people are deciding what to read, or what to like, or what to, you know, which social network to join in part because of what other people have done before them. And once they start doing that, then it just, it tends to snowball in one direction. And, you know, you get these, these when it take all effects. So it's one thing to observe that about history. But you can't ever prove it because we do in fact only have one version of history. And it's also one thing to make a mathematical model about it. But then people say, well, this is just a stupid model. And it's not, that's not real life. And there's all sorts of other things going on. And you don't, you know, you made all these simplifying assumptions. And so how do you know that that's really what's happening? And so to, to try to get around that, we ran this experiment. And this was many years ago, back in the early 2000s. And it was in the relatively early days of, you know, online music sites like Napster and whatnot was maybe even before, what was Napster? It was sort of around that time. Yeah. So we started doing this. Yeah. So I think probably it was right after Napster. Yeah. So it was around 2003, we started doing this experiment. And we, at least we started, when we come up with the idea, the paper was published three years later. And the idea was that we recruited a bunch of people online, mostly teenagers from an early social networking site, which now no longer exists. That's one of the ones that didn't make it. And, and they came to this website that we had set up called Music Lab at Columbia University. And they saw a screen with a bunch of songs, 48 songs by bands that they had never heard of, but which we had screened to be like decent songs, right? So these days, the equivalent would be on Spotify, Fresh Finds, right? Not famous, you know, artists, not yet famous artists that Spotify has determined are actually pretty good and maybe you want to hear them, right? And so we did that. And we had these 48 songs and people came and listened to them. And some people were randomly assigned just to see the songs and the names of the songs and the names of the bands. And they could click on them and listen to them and give them a rating from one to five. And if they wanted to, they could download the music for free. Okay, so that was sort of the the hook to get people in. But then other people got randomized to a different condition, where in addition to all of that, they also saw how many times the song had been downloaded by people before them. So there was a very weak social signal that told people how popular the song was. And our hypothesis was that when people knew what other people liked, they would behave differently, right? It would affect their decision making. And sure enough, what we found is that when people know what other people like, popular things get more popular and unpopular things get less popular. So there was some intrinsic difference, right? It wasn't pure luck, right? It wasn't totally uniform, randomness, even in the world with no social information at all, where people were making decisions independently. Even in that world, some songs did better than others, on average, right? So you could call that quality or whatever you want, right? You could say some songs for that audience, for that particular population, they liked them better than others. So there was an intrinsic appeal that was not luck, right? But that difference got magnified in the presence of social influence, right? So when people, the inequality in the world, the intrinsic inequality got blown up by the presence of social influence. And the other thing that happened is that unpredictability also increased. So at the same time that the popular songs became more popular, it became increasingly difficult to predict which song would become the most popular. And the way we showed that is by actually creating different copies of the world, right? And in each world, you could only see the people ahead of you in that world. And so they were identical copies, exactly the sort of idea of rerunning, rewinding history to 1997 when J.K. Rowling was still sitting in a Starbucks in England, typing on her manuscript, like rewind back to them and run it forward and then do that again and again and again. And what we found is that, contrary to sort of the common sense notion that the best song would always win, different songs won in different worlds, right? And so inequality increased and unpredictability increased. And we found that the more the stronger the social influence signal, the bigger these effects were. So we did a separate, a second experiment where instead of just, so in the first experiment, we kind of put everything on a grid and you just had these numbers telling you how popular things were. But in the second experiment, we also ranked them for you, which is a more sort of conventional way to do it. And so we're, same information, but now we're just hopefully telling you which ones are at the top and the size of the effects jumped dramatically. So in the real world where you have all of the other things going on, all of the advertising and promotion and search ranking and everything happening, you can imagine that the effects are much bigger still. Yeah. So that's, this is where like, where, like I said, like I, as a creator, as somebody who makes stuff, you know, the podcast, you know, I write, I make YouTube videos, I become very nihilistic, like none of this matters, right? Because once I read your book, YouTube made more sense, right? Videos that have tens of thousands of views or millions of views, they're going to get more views, right? People aren't going to, and the way the algorithms work and everything like that. And, you know, it, it messes, it messes with my, my head too, because I also, I read a lot, you know, I read a lot of articles and everything. And I'm just kind of blown away that not everybody knows about this, right? Not every journalist, every analyst, or, or they do, and they just ignore it, right? Because whenever I read an article, right? So for example, I recently started writing on sub stack, and they send out like a little newsletter like, Hey, let's hear from a successful sub stack. And they, they highlighted this one guy a few weeks ago, and it's like, how I'm making like tens of thousands of dollars with thousands of subscribers. I'm like, okay, cool. I look into it for two seconds. This guy already had like 50,000 followers on Twitter and over here and everything. I'm like, Matthew effect. All right. He's already big. He already has people to promote this and everything. So I don't know, like you, you know, you, you, you work with morning, that's a business school, like, do, do people not know about your research or has it gone like ignored because people want certainty so badly, right? Because lastly, you know, going back to JK Rowling as an example, I love that example, because her manuscript was shut down multiple times, right? There are I think it was rejected by eight, eight children's publishers. Yeah. And who, by the way, have like lots of expertise and they have, they have skin in the game. Like they're like, you know, they're like really highly incentivized to pick the next bestseller. Yeah. And they, they don't, they, they don't know, right? And they're making millions every year. These people are making millions and millions of dollars because they know, they know what's going to be a hit, right? But movies are another great example because how many movies do they spend hundreds of millions of dollars on that completely flopped and nobody saw it coming. So anyways, from, from your experience, do people not know about your research or do they just purposely stay ignorant and kind of neglected because they want that kind of certainty? Well, so, you know, in addition to writing about the dynamics of cultural markets, I also participated in them because I tried to write some books and get them to sell and also basically discovered that nothing I did made any difference. That's the same thing. And, and, and, you know, and actually, and I, and, and, you know, and I try not to fall into the trap that I describe of like, you know, you know, inferring after the fact, you know, what it was about the book that made it not sell. But I think, you know, if you flip it around and say, if you're, if you're not already famous, right? So the advice I give to people is like the best way to succeed is to already have succeeded. And that's very much true for, that's very much true for book publishing, right? That, you know, best way to write a bestseller is to have previously written a bestseller. Because there's a whole machinery of book publishing, where people are making bets about like what they think is going to succeed. And they're very conservative. So, you know, they got to order, you know, they got to order some number of copies. And so if there's someone who's already a bestseller, they know that, that book has a built in audience. And so they order a bunch of it and then shows up in bookstores and it gets promoted on places and it gets advertised. And so like all of the levers are pulled, right? Yeah. And they don't know exactly which ones when it works, they just pull them all, right? And, and, and probably it does pretty well, right? And, you know, at least our research suggests that like the best, the best prediction you can make, you know, how many books copies is your book going to sell is like it's going to make, it's going to sell about the number of copies that your last book sold, like that's probably about the best prediction you can, you can make. So if you're not in that category, like just think about all the things that have to go right for you in order to succeed, right? If you're just a nobody writing a first book, there's thousands of books coming out every year, maybe tens of thousands. You know, everybody's, you know, everyone's on Twitter, everyone's doing, you know, you know, I mean, I have lots of friends who are authors and, you know, every time someone writes a book, they're all doing the same thing. They're like, okay, I got to get on Twitter, can you please retweet my book? And I'm just like, all right, I'm going to do it. But like, it's not going to work, you know, because guess what, everyone else is doing the same thing. So you're all just kind of compete each other back into equilibrium again. So to kind of burst through all of that and become a bestseller just requires an alignment of stars that I think is not, is sort of fundamentally not predictable. But the other thing that I would say is that when people do succeed, they are highly incentivized to believe that it was there. Right? That, you know, that, you know, they say, you know, it's hard to get a man to understand a thing when his job depends on understanding it, right? Who's that Menken? I was just looking up the quote the other day. It's all Menken or something. But I think that's, you know, and actually my friend and colleague at Cornell, Bob Frank, who's written a number of... He was on here talking about success and luck. Yeah, he has this wonderful book about success and luck, where he talks about exactly this point that, that, you know, people who are successful are actually sort of systematically less likely to acknowledge this point, right? Because they're very invested in the idea that they did something, right? And the thing is, they did do something, right? Like, we sort of have this idea that like luck means I was just lying around, you know, drinking a margarita on the beach and like lightning hit me and then I was a billionaire, right? And they're like, no, no, no, that didn't happen. I worked hard every day. I, you know, I had plans. I took advantage of my opportunities. Like, other people were, were lazy, but I was the one who was, you know, upstudying every night. And damn it, I earned my success, right? And to a point, that's true. And most people who are, and Bob admits this too, like most people who are successful are generally smart and hardworking. And it's not like they don't deserve to be successful. It's just that they don't deserve to be successful more than, you know, a thousand other people who also worked hard and did everything, but just like didn't have the particular sequence of opportunities. So it, but it's very hard to appreciate, right? And, you know, and Bob Merton, another Bob famous sociologist, colleague of Lazisfeld back in the, in the mid 20th century, wrote about this in his, in his article called the Matthew effect, right? Where he's talking about actually scientific careers. Yeah. And, and it turns out in, you know, an industry after industry that, that, you know, people who look very, very similar early on in their careers, you know, sort of roughly the same, you know, level of intelligence and hard work and potential tend to diverge over the course of their careers to, to yield very different outcomes. And it happens in a, in a sort of sequential way where, you know, like academia is particularly prone to this where your first job out of grad school. And in fact, which grad school you go to, right? So it even begins before that, right? We know that faculty hiring is extremely biased towards the top department. So if you don't get your PhD from a top department, right there, you're probably not going to succeed, right? And even if you do, you're probably not going to get a job in another top department. And, and if you don't, it's going to be very hard for you to, to catch up later on, because the top departments have more resources, you have better graduate students, you're more likely to get grants, you're more likely to have your papers published. And so the cumulative advantage kind of begins very, very early on. And somebody who goes to a, you know, to a, a volunteer department, they have to teach more. So they don't have as much time to do research. They, their students aren't as good. So they're less likely to publish papers with them. They don't have as many resources. They don't have as much time. They can't travel to go to conferences. So identical people put in different departments can experience very different career outcomes. But 20, 30 years down the track, you look at these two people, and their CVs look totally different, right? One of them has a string of awards and, and, and honors and, you know, and top papers in top journals and, you know, students who have gone on to do other things. And the other one has just been, you know, teaching, you know, introsociology for 20 years. And you think they can't be the same, right? There's no way this is like these differences are too large, right? But because the luck is compounding over time, right? Because these, this cumulative advantage process is compounding over time. It takes things that are in fact luck and makes them look not like life. So it's, it is genuinely a hard thing to understand. And I'm sure you'll get people calling in or I don't know if people call into podcasts commenting on it saying, yeah, this is just wrong. You know, like that's not how the world works because that's always people just hate this message, right? They're like, no, no, no, no, no, no, you're, you're, you're making things seem like they don't matter, right? But it's especially true for the people who have benefited from that process. Yeah, I, yeah, I stumbled across your book around the same time I came into Robert Frank's success of luck because I, I kind of fell into this weird kind of depressive state, right? Because I was doing on paper everything I was doing was right, right? I was working hard. I was doing quality content. I was doing all these things. I was writing so many things. And I read your book. I read Robert Frank's book. I read a bunch of these books. I started really getting into books about the myth of meritocracy. Michael Sandel just came out with a book not too long ago. And, you know, as, as depressing and random as it can seem, it actually gave me some comfort like, okay, I'm not insane, right? Because it seems like we, we, some of us start beating ourselves up, right? Like, you know, like you said, when you send out book proposals, and it gets turned down, you start working backwards and saying, Oh, well, I must have screwed this up. I must have done something terrible. But we don't take into account those, those luck factors. But, you know, just yesterday, my Twitter was blowing up, because I don't know if you saw New York Times wrote this article about what they're doing in California. They're talking about taking away like the gifted programs, right? Because there's no such thing as a naturally gifted kid. And while genetics are a component, I wrote a piece like, people don't understand all the other variables that go in such as the household, like my backgrounds in mental health and addiction, like I've worked with a lot of people from terrible backgrounds. But like you said, and like Robert Frank says, people who have succeeded, they don't want to hear that, right? And there's that, that just world fallacy, like this world is just good people get what they deserve and bad people. So if you fail, that must have been because you didn't do something. And from that evolutionary aspect, that makes sense, right? We have to believe that this happens. But also for all the millions of people out there who are working their, their asses off. They have to understand that sometimes it doesn't work. But, you know, one, one thing I wanted to ask is, is I have a little bit more of your time, like, like, you're not my therapist, Duncan, but I need you to help me out here. How do you, how do you balance this, right? Like I say, like what I do, and this is something I actually talked with Robert about when he was on the podcast, I stay grateful for my successes. I realized that certain things have to line up. Like I got sober in 2012. I have to look back and realize how many lucky things happen for me to be alive today, right? So I practice gratitude for my successes. But with you researching this and knowing the amount of randomness and how quality isn't always the measure and how the Matthew effect rocket propels people ahead. How do you stay sane and not just be like, you know, what's the point? Why am I doing this? Why am I putting in so much effort when there's so much randomness? That's something I just, I think about far too often. Yeah, my mom asked me that a long time ago. It's like, how do you sleep at night? And you know, it's funny. It's never really bothered me. I mean, look, I mean, personally, I get bothered by things all the time, right? Like, you know, like everybody, you know, I failed many more times that I've succeeded. And, you know, every time I feel aggrieved and, and, and, and, you know, harshly and unfairly treated. But I also like you, you know, practice gratitude because I know that, you know, I have, you know, because of the world I was born into and the family that I had and the, you know, the schools that I went to and the mentors that I had. And, you know, the fact that I'm a tall white male with an Australian accent, you know, like these are, these are all like hugely important factors in driving success. And I also have benefited from the Matthew effect where I got some lucky breaks when I was very young and they gave me the opportunity and the resources to invest in other things that have also, you know, paid off at least sometimes. And so I often feel uncomfortable about my own success because, you know, I know how many other people are, are, you know, just as smart and just as hardworking and they haven't done as well. And so I think, you know, on the one hand, what I, what I tried to do is to just be aware of that and to, and to, you know, try to help other people, you know, to sort of narrow the gap a little bit, just as, you know, if you're rich, you should pay taxes because you got, you got to benefit from the world, right? Like, even if you worked hard and you did all the things that you think you did, the fact is, like, you know, in a state of nature, there are no billionaires, right? The only way you can be a billionaire is if you live in a world that is, you know, a highly differentiated, you know, market society. And you got to benefit from that. And so other people didn't and there's nothing wrong with you giving some back, right? It's not, you know, just to be comfortable with that, I think is a, would be a big help for all of us, right? If you could just sort of say, you know, continue to be pleased with yourself, right? Continue to be proud. Like, I also believe that I worked very hard and took advantage of my opportunities. And I also know that I could have done all of that and still not been successful, right? And so I think that I try to keep that in my mind and, you know, in order to sort of be more generous to other people, right? But the other thing that, and so it's, Bob Franks has something similar to this in that, you know, he is a paradox here, right? Because if you, going back to the sort of getting out of bed in the morning, if you, if you, if you sort of, as a kid, sort of internalized this message and said, well, everything's random, you know, there's no real, you know, there's no real meaning in the differences in success. So why would I even bother, right? They would certainly not be successful, right? So there is a, there is an extent to which it is under your control, right? That, you know, you sort of have to, I mean, it's not entirely true. There are plenty of people who are born into such privilege that they really have to do something terrible to screw it up, right? But for most of us, we do have to kind of work hard and hustle and take advantage of things to do well. And you want to teach your kids to do that, right? You want to say, look, you know, you want to kind of basically make them feel like their destiny is within their own control because you want them to work hard, because you want them to take advantage of opportunities, because you want them to be hopeful, right? And then once they become successful, they have to forget that, right? They have to completely forget that and say, oh, no, no, it was all just luck. And trying to reconcile these two, I think this is what trips people up so much is because they seem like totally opposite. Well, they are totally opposite views. One of them says, it's up to you, it's under your control, go out and make things happen. And the other one says, good for you, you've been successful, just remember it was all luck, right? And those are in fact contradictory pieces of advice, but they can both be true and they both are true, right, of the world. And, you know, another great quote, oh, is it who said, you know, the mark of a true intellect is to be able to hold two opposing ideas in your head and not go crazy, right? You know, probably Mark Twain or somebody like that, right? But like, this is your opportunity to do that, right? To be intellectual about it and say, I am proud of what I've done. I believe that it was good work and that I you know, and that I worked hard to get there and I know that I struggled and I know that I felt and I picked myself up a hundred times and finally I succeeded and good for me. But still, I was lucky. And other people were not as lucky. And so it's incumbent on me to show some generosity back to the system that produced me, right? And that produced the kind of success that I have. I mean, I think about myself personally, I'm a computational social scientist, right? I mean, what the hell is that? Like, most people have no idea what a computational social scientist is. And for most of human history, I would have been totally useless, right? No one would have any use for me. You know, the kings and queens of England would have me executed or, you know, sent off to like, you know, run around with a sword or something, because like, you know, or whole buckets of something, because like, computational social science would have been no use to them, right? So the fact that I get to live at a time when somebody like me is valued and very well compensated by a great university and given all kinds of resources to do the kind of thing that I love doing, that's lucky. Right? That is like the greatest luck of all, right? Forget about all like the, you know, ins and outs of my life, which also were lucky. The fact that I live in a world that is able to reward me for being me and for doing what I love doing, that is the greatest luck of all, right? And it wouldn't have been true in most countries for most of history, right? So the fact that that has happened to me means I should be grateful. I just should always be grateful, right? And I shouldn't be sort of worrying about like, my, you know, top marginal tax rate is too high, right? Because like, because, oh, I worked hard for my money. It's like, I would never have gotten the money if the world weren't organized the way that it is, right? So I think it is possible to be proud of your accomplishments and to believe that you worked hard to get them and to know your whole narrative and your story and still also feel profoundly lucky to be able to have that story. And those two things can go together. So that doesn't really bother me. I mean, not that I don't struggle with things a lot, but I don't struggle with that thing, right? Yeah, yeah, absolutely. And I love all that. I actually just finished rereading the stakes were made, but not by me this morning, which is all about cognitive dissonance, different ideas. But yeah, like that paradox that Robert taught us about, it is, look, it's like standing in one place, looking in the future and then also looking in the past, right? I got to work hard for the future, but I also got to be grateful for the past and all that. And, you know, what I've noticed talking with you, talking with, while we're talking with a lot of other authors is people who are grateful and they do try to give back and narrow that gap. Like I'm always honored when someone like yourself takes the time to come on here and chat, you know, and things like that, because, you know, maybe it's also top of mind because I just got back from a 12 step meeting this morning where their whole program is based on the idea of if you stay sober, help somebody else, you know, because something happened that was lucky for you. But real quick, last question, I'm not sure if you're familiar with the work of Richard Wiseman, but he does like the psychology of luck. But my question for you, because something he talks about is like the numbers game, if somebody's listening right now and they want to like succeed or create, would you say it's more of a numbers game, like just shoot your shot as many times as you can and cross your fingers and hope for the best? Is that kind of a strategy that you think might work or should you focus more on quality because it's that quality quantity debate? Well, you know, the answer to every question in social science is it depends. And the interesting question is on what does it depend? And so I definitely have, you know, I think I get the intuition of that, that, you know, that, you know, certainly in my own career and, you know, talking to all of my colleagues, like, all of us fail many more times than we succeed, right? All of us get our papers rejected, you know, no one, no one lives in a world where like every single thing they do is we're welcomed by their, by their colleagues. And so everyone has their, their stories about, you know, this, this favorite project of theirs that was harshly treated by referees or some prize that they didn't win that they should have won or some job that they didn't get that they should have gotten. But in the end, like no one sees those things, right? Those, those failures get hidden and the world only sees your successes. So, you know, if you fail 50 times and you win big once, that's what everybody sees, right? And so to some extent, yes, it is a numbers game, right? You've got to like, you know, if you give up the very first time that something goes wrong, then you're, you're, you're clearly done. So you have to, there's a lot of kind of picking yourself up and dusting yourself off and, and, and that's easier for some people than others, right? I mean, already it's unequal, right? But, you know, that some people have, you know, you know, family commitments, they have other things going on in their lives, other burdens that prevent them from just, you know, doing things, you know, an infinite number of times. So, so I think that, you know, that's already inequitable. But, but yes, to some extent, it's a numbers game. And to some extent, quality also matters, right? Like, you know, when I, I definitely have this discussion with my students and my, my junior collaborators who are very, very, this is, this is, this is still their future, right, that we're talking about, right? There's, they're really worried about, about, you know, what do I need to do to succeed? And they look at and they see other people who are insanely productive and who are, you know, writing tons of papers and they're sort of, you know, writing one or two papers a year. And I really think that, you know, just having a lot of papers, just doing a lot of low quality things does not work as well, right? And if you think about most famous people, here I am selecting on famous people again, but just sort of, most people, at least in, you know, in my profession, and they think probably in other professions as well, are known for like one thing, right? Or maybe they're known for a couple of things. Early on in my career, somebody said, if you have one good idea in your whole career, you're already ahead of like 90% of everyone else. If you have two good ideas, you're in the top 1% and if you have three good ideas, you're Einstein, right? So, you know, you can publish hundreds of papers in your life, but probably everybody knows you for like a couple of things, right? And so, and so if you have a couple of good things, right, you're good, right? And the problem is that you don't know ahead of time which things are going to be the good things, right? So, if you could only do like two really good papers, you know, pre-tenure, then maybe that would work. But because there's uncertainty about what the world will regard as good, everybody's kind of gotten incentive on the margin to like write one more paper. And then we're all kind of locked into that. Bob has written a lot about this, you know, competitive consumption, but the same is true in production, right? Where we're all, you know, everybody is competing to write more papers and then everybody writes more papers and now that's the new baseline, right? So, I think like most things, there's some trade-off between trying to have, you know, multiple bites of the cherry and also packing as much as you can into each bite because, you know, in the end a few high quality, you know, a few big hits, but just like in book sales, right? One best seller is worth any number of average sellers, right? So, what's the magic formula? I don't know, right? We don't know. If only I could tell my students, like, this is the sweet spot between, you know, optimal trade-off between, you know, number of strikes and a number of attempts and, you know, quality per attempt, then I would tell them, but I think, you know, in my own life and, you know, in my collaborations with others, we just sort of think through it all on a case-by-case basis and we do the best. I certainly wish we lived in a world where we could all spend more time on each thing and do fewer things, but that's not, unfortunately, is not the world we live in. Yeah, yeah, absolutely. And, you know, that is a great way to close because that brings me comfort because if Dunkin' Watts can't give you the sweet spot, then nobody knows the answer, so I'm good, but yeah, this conversation was great. I've been, we've been trying to link up for a while now and I'm so glad that you were able to come on. So, for everybody out there, first off, I need to go get your book, which is going to be linked down below, but where can they find you? Keep up to date with what you're working on, projects, your research, and lastly, is there another book in the pipeline so I can read more of your stuff? There, so the Computational Social Science Lab at Penn is my research group and that's the best place to come and find out about what I'm working on. You know, I have a thought for a book, which will be even less appealing than the last one to most people. So, I mean, no rush to write it. And so, there's nothing, there's nothing in the works just yet, but there's an idea that is germinating and I'm teaching a course on it right now and, you know, we'll see what happens. Beautiful. Well, I'll be, I'll be staying tuned and if that book does ever come to fruition, I'll be the first one to review it. But yeah, Duncan, thanks so much for your time and yeah, hopefully we'll have the opportunity to do this again. Thank you, Chris. It was really fun.