 So Matt, today we're reviewing the master algorithm by Pedro Dominguez. So like all great modern data science books, there's about 10 pages of confusion of whether the author is talking about machine learning or AI. It turns out Pedro Dominguez is a professor of CS at the University of Washington, and he is in fact talking about machine learning. Now the core principle behind the book is that there is some master algorithm out there that could unite all schools of machine learning we have today and could bring us into a golden age of science and technology. Yes, so Dominguez kind of impresses on us early the importance of machine learning as if we didn't know it already. You know, things like recommendation systems that's being used in medical research, self-driving cars, you name it, machine learning is going to affect our lives in some way. But as Dominguez points out, right now there are a ton of different algorithms that are being used. You have things like neural nets, Bayesian networks, all kinds of things with different pros and cons, and no one has really found a way to sort of unite the best of all of them into a single algorithm. So you can kind of think of this process as analogous to, you know, Einstein's theory of relativity, whereby that led to the standard model of physics. And a universal algorithm could basically take all the best of deep learning, Bayesian statistics, what have you, and create the one true algorithm for global law. Absolutely. And this gets to the core hypothesis of the book, which is that all knowledge past, present, and future can be derived from data by a single universal learning algorithm. Yes, this algorithm seems like something that could be quite useful. Were you able to find it? But in the meantime, Dominguez kind of divides the current state of knowledge in machine learning into five distinct schools of thought. There's five schools of thought. Yes, there's one more machine learning school of thought than there are houses in Hogwarts. And I don't want to bore everyone with the details too much. I do think it's helpful to kind of give a general overview of where Dominguez is coming from of these. So school number one is the symbolists. Now the symbolists are super traditional learners. They use things like inverse deduction, they like writing proofs. And if you've ever heard, you know, the classic deduction of Socrates is a mortal, you'll sort of know where they're coming from with their style of learning. Now school number two, much more modern than the symbolists, it's the connectionists. Connectionists see the best way to learn as looking at the human brain. Now they're not necessarily dissecting people to do this, but they are trying to kind of replicate the known structure of the brain in the learning algorithms they create. So they initially did something called the perceptron way back in the day. And since then they've moved on to things like neural networks, which are super prevalent in things like deep learning, and really kind of the frontier of ML in a lot of ways. Now the third school we're getting more into statistics here is the Bayseans. Now Bayseans look at things from sort of a probabilistic perspective compared to the other groups. So they'll not just only think about the data they have on hand for a certain problem, but they'll also try and combine information about their prior beliefs, you know, about the issue or anything else they know about the world. Yeah, that's where you would find modern day demigod, Nate Silver. Yes, when Nate is cooking up those predictions, you know, combining those poles together, he's generally learning some using some kind of Bayesian methodology. Now school number four is the evolutionaries. They're a little bit different to the others. And they really see things like survival of the fittest and Darwinism as being the best way to progress and learn. And so they try and incorporate the concepts of evolution into their algorithms, things like genetic algorithms are really kind of, you know, their expertise. Now the fifth and final school is the analogizers. Now this is probably the most general I would say the hardest to define of the five. But essentially, as the name suggests, analogizers are interested in learning from analogy. So this is something that humans are generally pretty good at and computers can really struggle with. So an example here is something like a small child looking at a dog. Now they'll be able to figure out that the dog is not just this one single thing, the term dog is sort of a generalizable concept that applies to poodles and golden retrievers. But teaching a computer to do something like this can actually be really, really difficult. And so analogizers are really focused on these kinds of problems. Yeah, Adam, I cannot wait to find where the sorting hat places me, but which of these schools do you identify the most with? So I will probably put myself in partly in the connectionist camp and partly in the Bayesian camp just based on kind of big data work and statistics background and all these different kinds of things. But you know, unlike Hogwarts, where you know, you really don't want to go to Slyverin unless you're evil, there's really like no bad option with these schools. I mean, they each have kind of their own pros and cons, the things they figured out, you know, the reasons for being, there's really a bad one. Yeah, I think something that, you know, just reading the book you may not get on the surface is that there is a lot of overlap between these particular areas. It's not like data scientists are out there fighting about, you know, what can't they fall into and there's some war going on. You just need to use all of these if you're a data scientist today. So perhaps to the reader who's thinking, Oh, yeah, these guys are definitely not getting along. That's not true. Yes, I've never gone up to someone who said, you're a symbolist, like I'm never going to talk to you, because we just fundamentally disagree on everything. No, but Dominguez I think is writing to a pretty general audience with the master algorithm. For that reason, like he's created these segments just to make things more memorable and easy to understand, but they really shouldn't be taken, you know, absolutely literally. But with these segments, I think Dominguez does a good job of kind of explaining the core concepts of each of the five schools. I mean, the bulk of this book is just going through these five and kind of explaining the things that they're working on, the kinds of algorithms that use that sort of thing. And so in the Bayesian chapter, for example, I mean, you really get kind of like your classic Bayesian intro one on one. I mean, we have a, you know, your coin flipping example, which by law, you know, every Bayesian text must include. You also have your your drug trial example, whereby you get tested to see if you have a particular disease. And based on the result of a test as well as the prevalence of a disease in the population, you can figure out the probability that you actually have the disease. So there's always kinds of classic Bayesian examples in here. Having said that, like sometimes for me, it was hard to follow. I mean, there's just so much information coming at you. Something like MCMC is in here, which is a Bayesian concept for basically sampling from a posterior distribution. Now, going into this book, I felt like I had a pretty good grasp on what MCMC was. But when I read the Bayesian chapter and I just went through the description and it's so quick and there aren't really any kind of visual aids for you, it became like pretty difficult to follow. And so, you know, especially for schools where maybe you're not as versed in them, I found that the book was just fast paced and a little hard to keep track of at times. Yeah, you know, the book does a great job of explaining everything with analogy and with real world examples. There's very little math in the book. I almost think none at all. And because of this, I wondered if it struggled a bit at time to explain some of these more difficult concepts. I felt exactly like you did, which is there were lots of things I thought I understood really well and I wasn't sure if it was just the presentation or just a different perspective that made it more difficult for me to understand them. So I definitely agree on that point. We should also say this book is really popular with people who are not in this field. So Bill Gates, Xi Jinping, leader of China also loves this book. And so it obviously has widespread, you know, appeal. And I wonder how some of those people perceived these topics. Well, they are probably enticed by the very catchy title of, you know, the master algorithm. It sounds quite impressive. And so you're probably watching this wondering if, you know, Pedro does actually end up finding a master algorithm at the end of this book. And the answer is sort of, I would say, kind of. He proposed us something called a Markov logic network, which has kind of been part of his research. And this sort of aims to combine some of the concepts from different schools. But to me, Gates himself admits that, you know, it's not necessarily ready for primetime at this point. I mean, it can't handle big data. There's a lot of kind of knobs that need to be turned to tune it correctly. There's a lot of data prep work that goes into it beforehand. And so he sort of says that, theoretically, this kind of thing could become the must algorithm. But it's not really there yet. And, you know, you're probably just better off running a neural net at this point. Yeah. Instead of talking about the five schools, which Pedro Dominguez himself explains very well in a series of videos online, I think we should talk about the two core concepts that kind of book in the five schools. The first is the philosophy behind analytics, data science, etc. And the second is the future implications is such an algorithm is developed. So first, let's talk about the philosophy behind this book. So Dominguez says there's kind of two core camps that people fall into. The first is an empiricist and the second is a rationalist. Now an empiricist wants to look at the world, make some observations and decide from that what the true knowledge is. A rationalist wants to come up with a theory, have that represent knowledge, and understand that oftentimes the data has a bit of noise and won't match their theory. Now he goes into a really detailed comparison of the two. But Adam, where do you see yourself in this dichotomy? So I've always seen myself as an empiricist. I want to get all that data. I've got to model it. I've got to believe in the power of data. Good data beats a good algorithm every day of the week. But I did really enjoy kind of a historical perspective that Dominguez brought on this. I always enjoy reading about Hume, the philosopher, who's kind of like the wet blanket of philosophers in many ways. And everyone else is coming up with these amazing discoveries and theories and ideas. And then Hume sort of comes in and says, well, it's never really possible to sort of know anything. You can never really learn from generalization, just because one thing is true today, just because the sun rose today, you don't know that it's going to rise tomorrow. You can never truly know anything like that. Yeah, there's the perfect example of the Bertrand Russell story about the turkey. Do you want to go into that? Yeah. So Bertrand Russell wrote the history of Western philosophy, very well known figure. And he has this famous story about a turkey where each day, the turkey is fed. It's fed at a particular time, 9am, let's say. And so a data scientist, say studying this phenomenon, would soon come up with an extremely confident prediction that every single day this turkey is going to be fed at 9am. And this model, this insight would work great. You take it to everyone you know, you present it to conferences, and things will be working really, really well until say Christmas Eve, when instead of getting fed, the turkey has its head chopped off. And so at that point, your model isn't working so well. Your assumption that this 9am feeding would go on forever has been broken. And this is essentially what someone like Hume is getting at with the idea. But just because it's happened however many times beforehand, you could never truly know something like that. Yeah, and I'd say this is an unresolved question in the book. Dominguez actually even calls this out that we don't really have an answer to this, which is, you know, kind of going back to your point, good data beats a good model every time. So I don't even know with the master algorithm, if this kind of problem will be solved, because you can imagine, okay, well now if we have data on holidays, maybe the turkey will get a little bit more knowledge and will be saved. But in reality, if this is random, or this is a more of a noise component, we're never going to be able to perfectly predict the turkey's outcome. And so I kind of see this as a big challenge, even given a master algorithm. Yes. So the core concept of the master algorithm is that it could learn any knowledge that is possible to be known from data. Now that like from data is I think a very kind of important quantifier here. And that you know, I think at best, a master algorithm would be able to say perform better on any holdout data set for you know, any input training data set than any other known algorithm could. You know, you could look at, you know, all the common machine learning data sets in the world, apply the master algorithm to it and it would essentially, you know, win every Kaggle competition imaginable. But to make us kind of get into this with human other parts of the book, but there is kind of a limitation on what is possible that we could ever, you know, actually know. And the master algorithm isn't going to be able to take us beyond that is just going to be able to make better use of the data we've collected to begin with. Yeah, that's why I really see the focus of this book as probably being slightly misguided, which is, you know, even if we get a better master algorithm, what I really want is better data, right? If we could know, you know, for example, what people are thinking, what people are feeling, that's much more valuable, even given today's models, than a more advanced model that can sort of get two or 3% more accurate than what a neural net can give you today. Yes, I think there's kind of transitions into kind of an extra discussion point of talking about what is the future, you know, if a master algorithm is created. Because I think you could immediately envision, you know, just kind of picking up this book, looking at the phrase master algorithm, basically saying it'd be some sort of Skynet, you know, well dominant type thing that would take over absolutely everything. You know, it's kind of reminiscent of a discussion we saw in Super Intelligence by Nick Bostrom, where Bostrom is really trying to hypothesize about, you know, as the frontiers of AI are pushed further and further out, as AI becomes much, much smarter than any human, kind of what would happen. And, you know, his futures are pretty dystopian in terms of kind of mind control and, you know, human type farming matrix style outcomes that you might end up with. Yeah, I'm still in the Nick Bostrom camp. I know Adam, you probably found yourself a little bit more identifying with Pedro Domingos in this book, where he sees it as a pretty rosy outlook, because if I'm interpreting this right, he said, actually, a computer is only going to do what you ask it, and it's not nefarious. And so, as long as we ask things correctly, it'll be fine. I think the limits of machine learning in terms of innovation are probably slightly overhyped. Machine learning can help you with something like a self-driving car, let's say, where you can take an existing product, you know, a car which is driven by a human, and you can automate that all the way, and you can make it incredibly efficient, so maybe we never get stuck in traffic, but you're essentially just improving an existing task that has already been created. Like true innovation, I don't think could be, you know, derived from a master algorithm. You know, a master algorithm could never say, you know, create something like the internet, just, you know, off top of its head. It could never create something like a Google, the search engine, just off top of its head. If you'd applied the master algorithm, you know, to instead of PageRank, you probably could have gotten better search results. But the initial spark of creativity and, you know, focusing on that problem, kind of determining the parameters of that problem, you know, a very much a human thing, but I don't think a master algorithm is going to get the ability to do it. Yeah, of course now, if we believe the master algorithm is all powerful, there's no reason to think we couldn't put some of those human elements in the algorithm itself, but it definitely made me think a little bit about the, you know, super intelligence point of view that this could be super dangerous. But I think one of the parts that is not called out enough is that even when we make a, you know, perfect ask of the computer, if you will, it can still do things we really don't like. So the classic example being, you know, I want you to make paper clips, but we don't tell it how many. So it turns the entire universe into a paperclip making factory. Yeah, Domingo sort of reassures us of master algorithm by saying that, you know, it's just going to operate within the constraints we set. So like everything's cool guys, nothing's going to go wrong. But obviously setting those constraints is difficult, especially if you want the thing to actually be useful. And so in that regard, you know, there are these problems that could arise. I mean, we've talked about this before, that you kind of negative side effects of algorithms, things like the compass for cynicism, algorithm, predictive policing, you know, creating negative feedback loops, there's all kinds of examples where algorithms that are implemented do not kind of lead to beneficial outcomes. Yeah, the other topic that's not really discussed is the NP complete are hard to solve problems in some given amount of time. I think one of the things that's ignored here is that even if the master algorithm is perfect, and even if we have perfect data, it has to be able to compute that in some small amount of time. It doesn't do any good to have a perfect prediction of what'll happen a week from now if it takes two weeks to calculate that. Yeah, you've seen that with, you know, Domingo's own research with Markov logic networks where in principle, they're a really, really cool idea. But he's kind of admitted himself, you know, computationally, they're not the most, you know, efficient or effective. Another clever concept that's brought up in the book is the idea of a data collective or almost a data union. Now, this is really cool for two reasons. One, Domingo's almost says that in the future, people will want to own their own data source, they won't want to give it to, you know, external companies, but they will want some way of leveraging the power of this information. And they could do so through a data collective. The interesting idea that he proposes that I hadn't really heard before was you would almost have a virtual avatar of digital self, a model of you online that would allow different companies to interact with this model of you. And instead of, you know, really worrying about your presence online, you'd actually worry about your avatar's presence online. Yeah, Domingo's, you know, produces this really great sounding example of your digital self, basically taking care of all the tedious elements of your life, where you go and just, you know, relax on the beach instead. So, you know, if you were, you know, applying to like graduate school or something, and they wanted to interview you, you wouldn't actually have to go do that interview. It's far too tiresome time consuming. Your digital self would basically go in your stead and it would know everything that you do and be able to give kind of an accurate representation of your answers to all these questions. Now this sounds very much kind of science fiction, perhaps a dystopian future depending on how you think about it. But Domingo's is kind of thinking big in terms of how super advanced machine learning would affect our lives. Yeah, now the next concept that comes up, and I think one we're going to expand on in the future is this idea of automation. What automation will mean for society? So one of the things that Domingo's talks about that I thought was really cool was this idea of employment rate, that in the future people won't talk about unemployment, they'll talk about their employment rate. So a country like the United States might say, actually, our employment rate is way too high. We need to lower this because we're getting beaten by other countries in the world. Yeah, there'll be a certain amount of cash aid to lowering your employment rate, because that would show how much you've been able to, you know, automate a way that would mean your loss of society. So I'm kind of giving away my own opinions here a little bit, but I don't really see this type of automation occurring anytime soon. Not only are there many kind of service sector jobs that I just think, you know, automating a haircut is going to be much harder when people think it's going to be basically my point here. But I also have to work to get this, but I also think that there is going to be a lot of pushback against automating jobs, because people don't want to lose that kind of sense of purpose and drive and things to do every day. And I think you would see a lot of negative societal consequences of all our jobs automated away, even if you had sort of a universal basic income to try and replace that. But having said that, that'll be our discussion for next time. This has been Random Talkers. Thank you very much for watching. Make sure to subscribe on YouTube, and we'll see you next time.