 At least if you're aware of these things and you're aware of some of the possibilities for these biases You can make you know ad hoc types of decisions to try to Maybe down weight certain things and try to avoid overconfidence in your model from same-same markets This is the odd will try 100 points swing on the doubt we can talk any human performance problem Definitely a part of a work-life balance, but it's also just maximizing your ability. Hello and welcome to the artful trader I'm Michael McCarthy chief market strategist at CMC markets Asia Pacific in our third season We talked to the experts in their own fields to uncover what gives them the confidence to succeed We uncovered confidence Unlocking the secrets behind resilience preparation and growth and how it can make you a better trader Today we're speaking to Harry Crane Harry is associate professor and Chancellor's excellent scholar in statistics at Rutgers University He focuses on probability statistics logic and the foundations of complex data analysis Harry talks to us today about both the mathematical side and the emotional side of any probabilistic situation Revealing how intuition is just as important as hard numbers and perfect models That's a big job title Harry I guess the important part of what that means I guess is so I'm affiliated with statistics and a little bit with philosophy, but my main my main work is in probability and statistics particularly in Trying to understand how probability applies in complex settings and in how to use statistics to analyze data coming from complex systems and If you go into a kind of a stat 101 introductory course a lot of the initial examples the toy examples that you're presented with somehow involve coin tossing computing expected values these perfect Idealized models, but you know as anyone who knows and the traders that you work with and you would know very well That's a very small and very limited part of the big picture One of the areas of concern about the application of analysis and statistics has been in political polling And we've seen a number of very important votes over the last few years go against what the analysis predicted President Trump election was a surprise to many people the vote in Britain to exit the EU was a surprise to many people What's going wrong here? Or is anything going wrong? Should we expect outlying results? There's always going to be outlying results whether these whether there's something wrong in this or not I actually think it's probably a little bit of both But I do think that there was something kind of fundamentally wrong in some of the in some of these analysis the one that I'm most familiar with is in the The US election the presidential election This is a good example for where we both get a piece of information But it turns out that we both got that information from the same place and likely for the same reasons You know, there were a lot of people who when Trump was elected were in disbelief. They were actually shocked They had no idea that this could even be possible. This was inconceivable and you know, what's the reason for that? Well, one reason is You know a lot of information coming through the news was saying you know was explaining every single day pretty much how You know Improbable or unlikely this was they were given probabilities to Clinton in the 90 to 95 99.9 percent range making it very unlikely that Trump was going to win And so it seemed like all this information was mounting to make it such that well Everybody on TV is saying that this is unlikely the statistics are saying it's unlikely and then it happened the other way Well, how could that possibly be and I think that part of it is you know Those those people who were talking on TV are not independent replicates of each other. They're not all giving an independent opinion They're all influenced by each other They all probably come from similar backgrounds and share similar beliefs themselves, so they would have similar biases People who only saw on their Twitter feed pro Clinton things would tend to believe that Everybody in the world must be pro Clinton. Similarly, if you were pro Trump, you're tending to be around people Who probably are pro Trump, so you you don't see the other side, so I think a lot of it is the tendency to kind of see the quantity of well all of these instances of Evidence that seems to point in one direction and not realizing that these are correlated events. These are things that don't represent a Lot, you know a big signal in one direction it represents maybe one signal that's being kind of propagated and being kind of Repeated, you know throughout throughout this this network, you know on the other hand, it's important to say there really isn't right now any Kind of go-to way to get around that. I mean there's no Develop theory or methods to handle this complex dependency. That's part of my part of the work that I'm trying to you know do that's research that's kind of on the cutting edge, but At least if you're aware of these things and you're aware of some of the possibilities for these biases You can make you know ad hoc types of decisions to try to Maybe down weight certain things and try to avoid overconfidence in your model So traders and people in looking at probabilistic situations We have to know when to break the rules, you know the technical analysis is in a lot of ways the least important part sometimes You know the practical side first Understanding and being able to assess and spot opportunities, you know, you come across these opportunities I'm sure it's the same in trading, you know, is this worth pursuing is this even worth Analyzing there's a lot of situations where I'm sure after you have so much experience You can see right off the bat. Well, this is obviously a good opportunity or this is obviously a bad opportunity I'm not going to waste my time on that and so that's the You know, that's the practical side and then if you get to the technical side if you have time for it That's that's all well and good, but then there's the the You know the whole emotional and psychological aspect of Risk which is well, you can have the perfect model and you can have exactly the perfect strategy But there's always going to be deviations From what you expect and so you know a lot of times, you know The first thing you talk about is whether something's a positive expected value Opportunity and that's kind of the first order thing to look at look for but then of course, there's The second order effects there's going to be volatility and there's going to be variance And so then there's of course immediate considerations one Can your bankroll handle those fluctuations and can you emotionally and psychologically handle those and that's so That's those are things that don't get taught in the classroom that you know those are not academic aspects of Probability, but those are very important and arguably you know more important I would say and so I try to Emphasize those and I have an example of that There's actually also ethical considerations as well, which is that when you're in the real world and you're taking risk You're actually taking risk and that you can if it if it goes one way you win if it goes the other way You actually lose and so I was teaching a class once This is what this was actually that one of the first classes I ever taught which was at the University of Chicago and I was doing an example an illustration of something called the st. Petersburg paradox I don't know if you've ever heard of that, but this is a coin tossing example where I'm going to toss a coin until it comes up heads So it could take me 10 tosses take me five tosses But every time I toss a tail the amount I have to pay you double So we started a dollar and then if I toss a tail I owe you two dollars if I toss another one I owe you four dollars and so on and so on and so quickly the stakes can go pretty high and so I did this example in class and I actually tossed eight tails in a row and so I ended up Having to pay out the stake of the game was one dollar and one of the students in the class. I guess one 256 dollars off of me and So that was a real-life example, but where you kind of have to deal with that volatility But the interesting thing to me that the thing that I didn't even expect Was that after we did this example the kids in the class didn't even expect me to pay out They thought well, this was obviously just an in-class demonstration. You're not gonna actually pay the person Which I did right on the right on the spot in the classroom So, you know, it kind of dawned on me that there's there's a lot of there's a lot that goes into this that really doesn't quite You know can't quite be conveyed in a classroom setting and so that's something that I've You know, I've thought I've tried very hard to stay aware of those things through my outside kind of work and more practical work and more applied work and I also try to convey those things whenever I Teach them. We've spoken with John netto the protein trader a number of times for the artful trader podcast And he speaks exactly to that that he does rely on his intuition to determine his trade size and some of his most successful trades have Come when he was uncomfortable, but felt compelled to increase his position size This is sort of going against the rules of trading, you know rule number one in trading is stay in the game It's about protecting capital and yet John who's been very successful breaks that rule regularly Well, I guess if it's working, you know, I'm not like who am I to say that it's you know That it's a mistake, right? But you know, I guess that that's just the kind of thing I was going to actually make a comment on the in the other direction, which is that, you know mathematically You know, there's expected value you can talk about something being a positive Plus cv positive expected value play and the idea being that you should bet if it's positive and not bet otherwise But of course any professional gambler knows that that that's not actually the case There are other considerations and I assume, you know gamblers have known this for hundreds of years, you know Instinctively, they've probably they've known something like the Kelly criterion Intuitively you don't place big bets on small edges that are unlikely to come through Or on edges that have a very high variance and that's kind of what the Kelly criterion formalized, right? It gave a theoretical justification for something that you know, everybody, you know Anyone who was who was good or who, you know knew what they were doing was doing already This example of a trader going the other direction. I mean, that's kind of on another level, right? I guess that's an intuition that I guess you have to respect it for sure But as of now, yeah, I don't have an explanation for no Well, it's a possibly insoluble conundrum about any duke another guest on the artful trade-up talked about resulting And being fooled by our results and it's an issue you've you've raised, you know for sure you just like you don't want to become Kind of discouraged by a method that might be losing in the short term. You also don't want to be fooled by You know short-term gains that are really just a result of luck. I mean, that's also part of it I mean the flip side of getting discouraged and having the emotional breakdown of losing is You know thinking that you're invincible just because you want a few times when you probably shouldn't have or that you're just You know running, you know above your expected value. So that's where I think Even though I think the the intuitive side is in a lot of ways the most valuable But where you're going to get your edge is how to use those tools and have in a unique way of looking at in a creative way of using them, but you know if anything where those technical Tools might be most useful is in being able to assess You know assess whether or not your assumptions are correct and assess whether or not your Your methods are doing what you think they're doing If after a long period of time you've been applying a method and you're you're running either well above your Expectation or well below it then obviously it means there's something wrong with the with the assumption It doesn't mean that you should stop using the met the model necessarily or that it's a bad strategy, but it at least Gives an indication right that there's something to look into and maybe a way to improve the model the analysis plays a Sort of fundamental role in building the picture, but success requires more well sure I mean the first thing you said was that you know the first rule of trading was to stay in the game And so that's something that you know I saw personally When I was playing poker and I had a lot of friends who played at that time It was very popular and I was playing you know I was admittedly you know I'm playing games that I probably shouldn't you know at that time I was playing What was the highest stakes on the internet which was much lower than what's available now? But you have people who are very good and you know very good at Know what they're doing, but they couldn't keep their money right they didn't have good bankroll management, so That's the first rule and then you know so it takes a lot to be successful and it takes It takes a lot to know what's worth your time on any given night you might you can walk through a casino and find Money-making opportunities at any given game But you're not going to have in your head a thousand strategies for a thousand different games When on any given night none of those circumstances comes up So that's where a lot of intuition comes into play as well, right you you see something It looks good based on your experience You have a vague intuition for how to take advantage of it you don't have time to go and do the math or calculate anything and Based on what you know you feel like you know you should either go for it or you should pass and So that's something that I think you can probably only learn through experience That's again not something you could ever learn in a classroom or a textbook And so there really is no substitute for the experience and also you know You're going to have your ups and downs and and losses at first When I first read the notes about this interview I was surprised about your emphasis on intuition and and it's been one of the features of this Series is that some of the top traders that we spoke to Dave Floyd John Neto Yourself these are all people at the top of the game now as a young trader The idea of trading off your gut was beaten out of me and and and that was the consensus 20 years ago that We could we were heading towards a more scientific approach It's clear from what you've said and what others have said that at the cutting edge of an understanding of Markets at the moment the human factor is gaining in importance Yeah Well, so what you just mentioned is something that maybe it's actually something that I didn't say You know, I talked to a lot or a little bit about overconfidence And it's easy to also have false confidence in your methods, but I didn't talk much about underconfidence, which You know in some ways is maybe not as bad because you probably won't get ruined But you also might and that you're also not going to reach kind of your full potential there In a trading or risk-taking situation, but I think that these are these are probably risks that are The risk of overconfidence or the risk of underconfidence I mean that probably come into play at different points in somebody's trading career, right? You mentioned as a young trader You're not you're taught not to use your your gut or your instinct because well, it's expected that you don't really have any instinct I assume this is something that takes takes time to develop right and so I guess How do you go about developing that and you know, how do you have the you know? the confidence at first to even go into Make your first trade or do you know even you know step foot into this arena at first you're probably taught some kind of system You have some kind of process and there's people who teach you and they mentor you and so on and so forth And there's a trust that you have in that process and that's something that I think is also important to have You know, it's it's just as important to know when to trust your gut as when not to trust it And this is an obvious situation where if you came in day one and were overconfident and arrogant in In your approach that you probably wouldn't last too long either right, but over over the years as you enjoy success Then you have the possibility of overconfidence because you've been using these methods that have been proven for such a long period of time And that's why I emphasize kind of staying sharp and trying to stay Kind of aware of the many limitations and to see different applications of a method that you might be using That actually failed or that where different considerations might pop up in other applications They're good to keep in mind as well in this whole conversation. Yeah, Harry I don't know if you're familiar with Richard Taylor's work in the field of behavioral economics a little bit He analyzed the television show deal or no deal 50 suitcases are on a board they contain varying amounts between $1 and $50,000 and Suitcases are removed and the aim being to end up with the $50,000 suitcase But along the way, there's an offer regularly made to the player They can take money or they can continue playing so it's a one of those rare examples of where we can see people making financial decisions under pressure and One of the key findings is that people regularly make irrational choices I raised this issue because it's one of the challenges in analyzing markets These are complex systems with a huge number of variables and the way each of those People reacts will not always be predictable. Does this speak to your work with complex systems at all? well somewhat I guess this idea that people tend to make irrational decisions and Particularly in the deal or no deal example. This is this is maybe a perfect example. I mean, I think it's a flawed premise It's a bad analysis for all the reasons. We just mentioned which is When you are making these decisions as a trader you don't just see that something's positive expected value and then put You know half of your half of your bankroll on it, right? There's bankroll considerations in any of these things So I don't know if he's done this or not But the first thing that you know, I would want to think about in this case is whether of whether somebody's making a good decision Is you know, how much money do they have? You know, if you're being offered? $250,000 for something that should technically maybe be worth $270,000 or something like that You know, you're being asked to gamble something that's like $20,000 for in exchange for some probabilistic event, right? And that very well may be a bad gamble If you you know, maybe you're you're broke you only have a few thousand dollars So of course $250,000 is is is not irrational. I would say so I think there's actually a lot more to it than Some of the behavioral economics literature Makes it out to be and that's I think because there is an emphasis in that literature and in this type of analysis on this first order expected value type of calculation when You know anyone, you know in finance or any type of risk-taking Situation knows that there's second order third order considerations about variance skewness and you know What is the distribution of the payouts and then beyond just the probability distributions? Of course, there's the consideration of outside factors You know one being how much money you have but others being you know, how much money do you owe? You know, what are your expenditures going to be for the next month? year and so on and so forth so All of those things are very complicated and too complicated in some way to account for all at once, but It does get at this idea that there's complexity in this decision and that when these contestants make that decision they're somehow, you know in the heat of the moment and you know In a very short and pressure pressure filled situation They're they're making those decisions and I mean, I don't know how irrational they are by the by the Calculations or by the analysis in these papers, but my guess would be if you actually looked at it. They're not actually that bad Hey, thank you very much. There's a lot of insights there for our for me and for our listeners So I really appreciate you taking the time to talk to us today. Yeah. Thanks a lot enjoyed it That was Harry Crane on the artful trader confidence uncovered If you liked what you heard in this episode, you might want to head back to episode two of confidence uncovered called Confidence in your worst-case scenario with good friends and even better traders John Neto and Dave Floyd I sat down with John and Dave to reveal the secrets to their success It's a confidence that doesn't lead to recklessness because confidence that leads to recklessness Can be a major risk management issue. I think it's been positioned in the market as if you just do these three simple things That's all you're going to need to be successful But that's on top of many many other things that are required that aren't so simple Make sure you subscribe on your podcast player to hear Dave Floyd and John Neto and all of our interviews Or listen at the artful trader podcast.com Tune in next time