 Boom, what's up everyone, welcome to Simulation, I'm your host, Alan Sokian. Very excited to be talking about safe, artificial, general intelligence that benefits everyone. We have Greg Brockman joining us on the show. Hello. Hey. Got beer. Thanks for coming on. Really appreciate it. Love your work. You're the chief technical officer and co-founder of OpenAI, which is doing incredible work in artificial intelligence, safety, security, building artificial general intelligence and previously at Stripe for five years prior to that, doing a lot of work at Harvard and MIT, building up the math and computer science skills that you now have today. So this is a very pressing time, probably like the most pressing time that we have is building out passing the torch from the biological to non-biological intelligence. What is your current take on the state of humanity? Also, I guess the way that I think about it, I don't think about it as there's a torch to be passed here, right, is that the purpose of technology is to enhance human lives. You think about the march of progress, even starting with like a steam engine and all of the wonderful technologies we've created over the past hundred years. Think about the computer. Think about the internet. Why do we like these technologies? I think that when technology serves people and enhances the human experience, helps us create a world of more plenty, a more equitable world, it's good. But it doesn't always work out that way, right? Sometimes you look at more recent technology and I think that we've introduced new societal questions that now just someone with an idea can generate such wealth, such value that you end up with a very inequitable system. And so looking forward, if you can actually build smart systems, if you can really build machines that can automate human intellectual labor, which is the kind of thing we're talking about when we say artificial general intelligence, that all of those questions that we've seen are just going to be amplified, that there's this potential for this huge benefit, right? You think about like what are the problems that humanity faces that we're just not really seeing answers on the horizon. You think about health care, cheap health care for everyone that really works. You think about human doctors today, we're addressing symptoms. We can't go and do root cause analysis, right? There's no human doctor who can go and look at your genome and say that these are the specific risks that you have and so starting today at this age that you should start eating these foods or exercising in this way or you should do these preventative things. There's no human doctor who can see every single health record of every patient across the entire world and be able to correlate all of these treatments with outcomes. Those kinds of questions, how are we supposed to solve that? We're just so many steps away. Right now we're just stuck in this world of trying to figure out how we can get our medical records transferred between systems. So I think that we have this huge opportunity, right? We're on the cusp of having technology that can really enhance the human experience beyond anything we've seen to date and we're excited about that. Yeah, interesting. So when we think about it in this big history perspective, we've always been developing technology that has enhanced human life and this is another one of those technological advancements that we're aiming to make as enhancing of human life as possible, as benevolent and benefiting as possible to everyone, instead of thinking about it as a passing of a torch. That's right, absolutely. Interesting. Okay, okay. And then, so then how did you get to this point of being here five years now developing, you've went and given, you've delivered statements to congressional committees on the importance of artificial intelligence. So tell us about how you became Greg Brockman that we see today. Well, I actually started out as a very different Greg Brockman. For a long time growing up, I thought I was going to be a mathematician. And I read these stories of Galois and Gauss. And they were operating on this 100, 200 year time horizon, right? That Galois was this young mathematician. He actually ended up dying in a duel when he was 21. I didn't want to emulate that part. But the part that I did want to emulate is that almost out of nowhere, he developed Galois theory. And it really took another 100 years for mathematicians to build up the machinery where they really should have been able to develop Galois theory. And so that was how I thought. I was like, I want to do abstract mathematics where it's never used in my lifetime. If anyone uses it, it means that I wasn't thinking deeply enough. I wasn't abstract enough. Because I just love the idea of that deep impact. The impact that's going to last hundreds of years. And really, that's going to help set the future for humanity. And that was kind of the best way I could imagine to contribute. And after high school, I took a year off. I was writing a chemistry textbook because I'd happened to come up with some very mathematical way of thinking about chemistry that I wanted to immortalize. One of my friends said, you're never going to get this thing published. So I learned to program to build a website. And the thing about programming is that you do a very similar process to mathematics where you think hard about a problem. You write it down in an obscure form rather than a proof. It's a program. But then suddenly, anyone benefits from it immediately. That that feedback loop, the idea that you could build this castle in your mind. And then suddenly, anyone can live in that castle. That for me was just like, why didn't anyone tell me about this before? And so this multi-hundred year timeline horizon. It's an instant deployment to billions of people. Absolutely. It's such a powerful thing. It's such a powerful thing. I think we still don't really appreciate how beautiful that is. And so at that point, I said, I'm just going to focus on building. I want to build software. I want to become as good at programming as I can. And so I showed up at Harvard for undergrad for freshman year. And I discovered the Harvard Computer Society, which was a group of people who just want to build services for the Harvard community. And got really involved in that sophomore year. Suddenly, I was running the club. The two seniors who knew what they were doing, who had spent all freshman year teaching us things. They graduated. And I was the one who was supposed to teach people things. And I was like, I'm not ready. I'm just a sophomore. I kept looking down the street at MIT where there were all these people who I wanted to learn from. And so I ended up transferring. I was there for another year. And throughout all this, I was working on the thing that I felt was going to have the highest impact, which were startups. And so I tried a bunch of different things. It never went anywhere. But from each one, I learned another thing not to do. And I ended up meeting some people working on this payments company. It was like two or three people at the time out in Palo Alto. And when I met them, I realized these are the people that I want to work with. These are the people I've been looking for the whole time. And so I was in the middle of a semester. I had this whole path charted out for myself. And I said, never mind. I'm going to drop out and go work on this. I didn't think it would succeed. I expected that we'd work on it for a year. And we'd fail. And I'd probably be back in school. And I'd have to retake my classes. But I just knew that these were the right people. And so this. What was it about them that made them the right people? Yeah. For me, it was just we really clicked in a way that I think is very rare. And I think that we had very complimentary skill sets, but also overlap in terms of the way that we wanted to affect the world. I think one thing for me was I was 21, 22. And I had this belief that as a 21, 22-year-old, you can't really do real things yet. You have to be in school. You have to go to grad school. Come with something cool there. Turn it to a startup. If that's the route you want to go. But 25, 26, you can do stuff not yet when you were my age. And these people were my age. And they were already out here. They had already started to build a company and had raised some seed around. And I was like, one of the two of us is wrong. One of the two of us has completely wrong. I just want to know who it is. And so in some ways, that questing to probe the different systems in the world and to see how can you have impact and how can you have outside leverage, that was something that really drove me. Yeah, that's so, so important to get to greater local maximums of intelligence and drive to make new systems that obsolete old ones that maximize our society's potential and a great place to do that at Silicon Valley and a great place to do that is with people that are building something like Stripe, like something regarding payment deployments. That's a big idea. So then you guys wrapped on that. You wrapped on that big idea. Stripe is still roaring. Stripe was purchased, right? No, no, no. I'm still independent. OK, Stripe's still independent. And then what was the transition from Stripe to OpenAI? So for me, I think in that transition period where I was going from thinking on this 100, 200-year time frame of lay some mathematical foundations and have impact that way to you can actually build stuff. I discovered Turing's 1950 paper on the Turing test. And it's this wonderful paper. Have you read it? I haven't read the exact paper. I highly recommend it. And there's really two big ideas in it, one of which everyone knows and the other, I think, does not get played. The one that everyone knows is the Turing test, right? The idea of can a machine think? Turing's like, I don't know what that means. So instead, I'm going to devise a concrete test that seems just about as good, which is, can a computer fool a human into thinking that it's a human as well? That it thinks, yeah. That's right. And so it's like, thinking is not really operational. Everyone has a different definition of it. But this test is very concrete. And if you can actually operate, you can answer all the questions, and I can over some text channel, I try to teach this other being calculus. And it learns calculus like, sounds like thinking, right? So that's the idea that everyone's familiar with. The idea that people aren't, that really captivated me, was Turing said, how are we going to pass this? How are you actually going to build a system that can pass my test? He said, it's going to be too hard to program an answer. So instead, we're going to have to learn an answer. And he says that, imagine if you could build a machine that learns just like a human child, that you could program in whatever rules you need to have that learning process. And then you've got to grow up. You've got to experience the world. You've got to have 20 years of experience in an interesting environment. Like, this is a way that you could actually get all of the crazy things that are required to pass that test into a system. And for me, what I realized was that, here was the key. You think about programming, it's limited by what I, the human programmer, can understand. Like, I have to understand the process for how a domain works. You think about building a payment processor, right? You have to understand all these details about where you're going to move the money in, like all the different error codes that come back. You write those all out. And then you have an automated system that executes it. But that core understanding of the process has to come from a human. But with learning, I, the human programmer, can specify the goal and the details as to how it actually works. That's something the system can figure out for itself. And to some extent, the sounds abstract, but we experience it all the time. You think about children, right? And how do children learn and grow? And that as a teacher, you don't get to program in all the rules. You don't get to say, here's how you have to think. But if you're a good teacher, if you're doing a good job, you can help that child learn the skills that they need and the right ways of interacting with society and interacting with others and doing good things for the world. And so I think that that idea is a really core powerful one. Is that both of the pieces of that? Those are the two pieces. So here's this starting task. But the thing that I think is undersung is the way we're going to pass it is through learning. Oh, OK, OK. So then that the thinking machine would need to learn in order to pass along the way. That's right. And so the idea of human programmers, we can only push our system so far. There's a limit to the problems we're going to be able to solve. When we do great things, don't get me wrong. You look at all the software systems around us today. It's pretty amazing. Imagine describing that to someone in 1900. We're going to have all that. We're going to have the internet. No one would believe you. So this is a build up of thinking capacity that is really important for machines and artificial intelligences. That's right. And in a more general, potentially, the better at reasoning in a human society that is also very general and complex. Yep, exactly. And the way that I would think about it, just to add to that, is that you can think about the systems we can build today are very brittle, very narrow. We can just, the systems we're all used to, you have a computer, and if anything goes wrong, it crashes. And you get a blue screen of death. We kind of hopefully move beyond blue screen of death in particular, but that same idea of that there's just some programmer who forgot to add some if statement and now it's all broken. It's not really able to handle new circumstances. And so the way you can think about it is that the computer systems that we can build, which really do benefit us, that there are huge benefits from these technologies we've already built, are just such a narrow slice of all the computer programs that could be out there. And so the question is, are there new technologies that we can add that increase the space of these beneficial programs we can build? And that's where I think this really powerful idea of having the machine learn the process, have the human specify the goals, we should be in the driver's seat, but why should it be that I have to spend all my time really looking through the details of how all these little nitty gritty piece of the system should work? Specifying the goals and then having the thinking slowly become better and better over time. So it's constantly running iterations of thinking until it gets closer and closer to the goal, and that's how it's building itself. Well, and it might be instructive to look at today's AI technology and how it works. So if you look at the history of the field, starting in 1950s is when the term AI was coined and when people really started to set out on this journey of trying to build machines that can be smart, that can pass the Turing test, that can automate intellectual labor in some way. And there were two big camps. There was the camp of, this is a little bit of a caricature, but let's say the symbolic systems camp, which was, let's try to write down the rules, right? And they ended up building the expert systems, that there were various other approaches of just have machines that can kind of search through lots of things, try to encode some human knowledge in there. If you want to learn about language, well, let's encode how parse trees work. And then there was the learning camp. And the learning camp in some ways ended up really focusing on neural networks. And neural networks are this simple, simple idea that as time has gone on, and as we've gotten more computational power, have proven to scale much better than any other technique. And so the first neural net probably dates back to 1940s even before we even had computers. It was a mathematical model of a neuron. And in 1959, this person named Rosenblatt released the perceptron, which was really the first implemented neural network. It was a hardware device in addition to a learning rule. And it was able to learn some very basic stuff. And in the New York Times, it was announced that the Navy, in front of this project, and they said that the Navy says that one day perceptrons will recognize people, will call out their names, will instantly translate speech between languages. And basically, the other camp of AI people looked at this, and they said, your machine can't do any of those things. It's over a height. This is false. You're misleading everyone. And the funny thing is, you fast forward to today. And neural networks are doing all of those things. Neural networks are the cutting edge of artificial intelligence technology. That's right. So in the history, the recent history, is that in 2012, so neural nets were kind of this idea that existed in the 60s, that actually the Marvin Minsky and other researchers who were very big names on the other side of the fence, spent a long time really trying to discredit that approach. They didn't believe in it. They thought, look, you said you could do all those things. You can't do any of it. That you're kind of sucking all the oxygen out of the room. You're getting all the funding. We need to put an end to it. 1969, published a book that actually did curtail research for about the next 10 or 20 years. So that was the beginning of the first AI winter. Fast forward to the 80s, the democratization of compute hardware caused this resurgence of neural nets. And suddenly, all these people were trying these different things. They developed a better learning rule called back propagation. They developed a lot of really awesome things. But once again, somehow that there ended up being a winter where people got really excited. They weren't able to make the progress that they were hoping. And so now you fast forward another 20 years, 30 years to 2012. And in 2012, something really magical happened. So two grad students at University of Toronto, so one of them is my co-founder, Ilya Satskover. The other is Alex Prochevsky and their professor, Jeff Hinton, were able to train a neural network on two GPUs, which are just these gamer cards that they literally developed so that gamers can have really fast graphics. They're able to train deep neural network to recognize images. So is there a cat or is there a dog in this image? Better than anything else, by a huge margin. So you imagine what's been going on is that for the past 40 years, all these computer vision researchers have been trying to figure out how can I recognize if there's a cat in this picture? And so they come up with all these rules of like, oh, maybe there's an edge here and if there's a whisker and a whisker, but maybe one of the whiskers are coded so you could try to make all those rules work and they built these very brittle systems that just didn't really work. And then two grad students with just, like $2,000 worth of hardware are able to blow all that out of the water with a system that just started out knowing nothing and you just showed it examples and you just keep showing the examples and something gets really good at this task. And so that was 2012, everyone was like, whoa, this is a wake-up moment and- This is supervised. This is supervised learning with neural networks. Because there is a decision of, yes, you identified correctly or not that there's a cat. That's right, yes. And so the dataset that you have has a bunch of labels and in fact the big dataset that everyone uses as you use the benchmark is called ImageNet. And when ImageNet came out in 2010, it was just viewed as intractable. It was like millions of images that were pulled from my Google image search and a bunch of other things and it's like these pretty high resolution photos and they have a thousand different possible things that are in there. All sorts of varieties of dogs and cats and motorcycle and a thousand different categories of things. And so all these computer vision researchers using the traditional non-neural net-based stuff were like this thing is totally impossible. We're gonna be working on this for the next 10 hundred years. And two years later, so I think 2010 is, I think about when ImageNet, I think they first started running the competition then it might be slightly off in terms of the years. Two years later, this neural network comes out from these two grad students and they're able to just push so far. They move the state of the art to like, I think it was maybe like 30% of the time there would be a mistake with the old stuff and then 15% of the time there would be a mistake with the new stuff. And that gap is actually very, very significant because especially as you basically those, the ones that you don't make mistakes on are the easy ones, right? It's now going to the pretty difficult ones. And everyone's like, all right, we've got another 20 years worth of research to go from 15% to really solving this data set. But actually what happened was that every year from there there was this exponential decrease. And now we've surpassed human level on this data set probably 2015, 2016. And now it's basically this particular task is viewed as obsolete. It's solved, right? And the funny thing is we haven't, we still have more to do, but this data set is totally saturated. Interesting, so there's a certain amount of human level tasks that are already become obsolete by artificial intelligence. And so there's a little asterisk too, right? And so here's where things start to become very subtle and you really have to look at the details. And I think one of the hard things about figuring out what's really going on in AI today is that simultaneously the details tell a very interesting story, but when you really zoom out that the narrative looks very different. So when you really look at the details we have not solved computer vision. Yeah, yeah. Otherwise we would have fantastic vehicles that could drive themselves. That's right. All this other stuff. That's right. That's exactly right. And these neural nets make mistakes that a human child never would. That it maybe misidentifies an object as a human. Exactly, right? And just like you think about like even a small child is able to never make a mistake of thinking that a human is a cheetah or something, right? It just never happens. And so we have to solve that problem. But at the same time if you look at the contrast as to where we were it's night and day before we couldn't even, we didn't even have a place to start. And so I think that if you zoom out that the story again is a really telling one. So first we had computer vision and basically everyone else in AI looked at that and said, cute trick, worked for your field, my field, way too complex. These neural nets that start out with no knowledge and just learn the answer, never gonna work. Especially for machine translation. Like can you imagine for machine translation you think about what you need to understand, you need to understand languages and syntax and like all these things. And actually it turned out 2015 that neural nets had surpassed every other technique in machine translation, right? And it was like there was this march of computer vision speech recognition and a whole host of others. And so this is general. This is generality, right? This is if you have a labeled data set, so you have your inputs, maybe their images, maybe their sentences in one language and you have your outputs, maybe there is there a cat in this image, maybe it's the same sentence in a different language, we now have the technology that can learn to map one to the other. Yeah, that's massive. Okay, so any data set that has to be structured, it can be unstructured. So yes, so here's, so that advanced that I refer to is that supervised learning. There it has to be structured. And the thing that people have been saying is that, well, if you look at what data's out there, there's huge amounts of unstructured data on the internet, huge, huge amounts, but we can't use it for anything. The only data that's useful for these systems is labeled data. Is a labeled data. That's right. Okay, okay, and that would be very interesting to figure out how to train artificial intelligence to maybe label data. Yeah. That's interesting. So, okay, so now you, so basically with any, right now we're trying to make it so that with any input structured data and a goal that in mind, we can get outputs through a neural network. Yep. And so how did you and Ilya go like, let's start open AI. Let's have this charter as well. Let's be very ethical, heart-centric. Let's really be thinking about this on a benefiting everyone around the world level. Yep. Yeah. Exactly, so when I was, so in 2015 is kind of when I felt like I'd gotten Stripe into a very good place and I could really start to think about how can I have the kind of broad impact that I want? That I think within Stripe, pretty amazing. Today it's a $20 billion company and I could contribute to that. But I think what I really like to do is try to find problems that I think will not work, will not play out in the right way without my involvement, where I can really try to steer them from one place to another. And AI had always been the thing that I felt was the most important problem. It was just a question of when. And after three years, 2012 was when Ilya had that breakthrough and three years later is when we were starting to think about this. We actually had this dinner with Sam Altman and Elon Musk and others. And I think that the question there was really is it too late to start a company trying to make AI play out rather than in a negative way in a positive way? Is this something that you can actually do from scratch? And the conclusion from the dinner was it's not obviously impossible. But you're competing against the top talent at Google and in China. This is very, very tough stuff. Apple, Facebook, Microsoft, Amazon, it's just, and they can pay $250, $500 a million a year for these positions. So how does one come up into the rankings? These were the questions that we had to ask, right? And really, I think that AI development is not like a normal science in some ways. You almost think of as very underfunded and very kind of backwater, right? And it's like neural nets, in fact, were this for a very long time. The only reason that we have what we have today is because this Canadian Moonshot organization was willing to fund a couple of labs to go and work on this stuff. No one else was willing to do it. And today it's very different. As you said, that lots of corporate labs are involved and that there's a large number of people who are working in this field. And so to have a differential impact, it's pretty not obvious. But we felt that this technology is just so important, that what I described is supervised learning, but if you can really make unsupervised learning work, you can really learn from all the data around us. Like that only unlocks a whole host of problems. And if you can really build an AGI, if you can really build machines that can automate meaningful amounts of all of human intellectual labor, it's gonna affect the world more than any technology we've seen. And so if you believe that that might be possible, it's really hard to sit by idly. Yeah, you wanna dedicate your life to it. That's right. And so it became a question of tactics. And it's like we have a mission that we knew was important. We wanna make sure that AGI benefits everyone, that the world that we deliver, we in the sense of humanity, in the sense of any of us as individuals, to the next generation to the post-AGI generation is one that is equitable, that is a better world than the one that we're in today. And I think it's very achievable. Like I think that AGI can be the most unofficial thing ever, right? I think that, again, you think about the amazing applications. And how many people were saying, oh, we'll be weary about planes, or be weary about antibiotics, or be weary about all of these advancements. And so it's equally as important to push through the adversities because it does end up catalyzing a lot of reduction of suffering and increase of health. That's right. And I think it's really important to be realistic, right? If you're developing a transformative technology, you gotta think about the ways that go wrong. And you gotta think about the ways that go right. And then your job becomes trying to sift those out and trying to figure out what path do we take that minimizes these downsides, maximizes the upsides. Yeah, yeah. Okay, so then how do you figure out how to minimize downsides, maximize upsides? You made the charter. And the charter is a, would you say, kind of like an ethos of open AI? That's right, yeah. And so it took us some time. We started out with a team that had done all the recruiting for that and we brought together a team of about 10 people to start. Which is a really weird thing to do as a new company. Normally it's just you and a co-founder and you got some time to build a product and get some users. But here we needed a critical mass of these people to push forward AI technology. And so we spent about two years really trying to figure out how do we wanna operate? What is our strategy? What is our path to actually making the future be better than it would be without us? And the open AI charter is a lot of ways and embodiment of that thinking. And so there's four sections. It's very short, it's on our website. Anyone can read it. And I think that there's a few points that are pretty not obvious. The first one is that the core mission of open AI is to ensure that AGI benefits everyone. And that in terms of how do you actually implement that? We can talk about that in a bit. But I think it's just like AGI is just going to be so transformative and it's going to be so impactful that it shouldn't be something that's 100x more concentrating of wealth than anything we've seen before, 1000x, 10,000x, whatever the number's going to be. It should be. That creates a lot of instability and civilizations if that's the case. That's not a good world. I don't want to live in that world. Doesn't matter which position I sit in. I don't think that's a world that's good. There's a lot of beautiful creativity in people's minds that can be unleashed, that we would love to see unleashed. That's right, that's right. And so there's a second point which is as we talk about safety and security and what's it going to take to get to that good world? Well, one thing we're very concerned about is first of all, people not investing in safety research. And the more that there is an arms race to get to the end, you have a bunch of different companies or countries or other actors that are trying to build a transformative technology, what's the first thing that's gonna go out the window? The safety. Exactly. And so we actually have built into our charter a pretty weird provision, which is that if someone else is pretty close to succeeding at building AGI and they're value aligned so they kind of want the same thing that we do, we'll stop competing with them. We'll actually help them deliver on that. Because the core, like one thing that I think is really important about our mission is that we don't actually have to be the primary actors. We don't have to be the ones to make AGI benefit everyone, for AGI to benefit everyone. And that's something I think really sets us apart from other structures from other companies. Yeah, well every company is trying to figure out how to maximize their vision, but they don't think most often about it like if another company is doing that same vision and they're doing it potentially better, we can maybe merge with them and help them. Yep. Yeah, that's, yeah. Yeah, and so one thing that did, yeah, it's a pretty non-typical thing. Yeah, yeah. Yes, okay, so then there's that safety component, of course, and then there's fourth as well. That's right. Also we have, I think it might not be too important to break out all of the privilege here. But the technical, the fact that you have technical professionals doing it as well. Yes, and this is the core strategy, right? The core strategy for open AI is to push forward AI development on that exponential until we successfully build safe AGI, which is, there's really three components that go into that. There's a technical piece of how do you actually make this technology work? And that's hard. And how do you actually figure out how to build a general intelligence? Yeah, this is the hard problem, right? And so for us, a lot of what we do is we have a number of different teams that work across different domains. And to some extent, the actual problems we work on are secondary. Because what we try to do for any problem is we're trying to develop general purpose technologies that can solve a problem that no one's been able to solve before, but are applicable across different domains. I have a question. Yes. If your intelligence that you're building can maybe switch between intelligent tasks that are considered narrow intelligence, would you consider that a step towards a more general intelligence if it's able to do two or more narrow? I think this is a step, and I think that there's this core technical problem called transfer learning that people haven't quite figured out yet. Which is, how can you use knowledge from one task or domain in a new task? Humans are great at this. I read a book on Paris. I go to Paris. Suddenly I'm better at Paris related tasks. Our AIs don't do anything like that right now. But we're actually starting to see with some of the models that we've been working on recently. We're starting to see inklings of this. So one example is this model called GPT2, which is a model that was trained on internet data to just predict the next word in text. So you show it the beginning of some snippet of text. You say, what's the next word? It just plays that game. It gets really, really good at that task. It's like an n-gram now? So it's an n-gram for a very large value of n. So the context here are like thousands of words. And so it turns out this model is able to, first of all, you can ask it to generate text for you because you just show it some snippet and then you say, what word should come next here? What word should come next here? You keep feeding its output back to itself. And so it's able to write text and the text that it writes ends up being coherent paragraphs, coherent pages on any topic you want. You can literally, we have this example that I think was very visceral where we prompt it to basically say, recycling is good for the world. No, you cannot be more wrong and then let the AI complete it from there. And the AI race is a very convincing essay on why recycling is actually not that good. This was very interesting, yeah. And also one of the things that when I read that that came to my mind was that maybe it could potentially be that the idea is that we're not thinking about it from a first principles perspective. Like you should think about it from a first principles perspective of the way that you transfer goods and packages should not even have a component that goes into a recycling in the first place. That's right, yeah. I mean, it generates very good discourse, right? You know, now when I see recycling, I'm like, hmm, why are we generating all this waste in the first place? Yeah, yeah. And the thing that's amazing about this model is that's an instance of transfer. It used this knowledge from this task to predict the next word, and it can suddenly apply it to a story. To a story, to an essay, to a debate, to, and you know that here's where things get a little bit unfortunate to generating fake news, to generating abusive content. And so this was the first model where we explicitly said, we're not sure that this should be released. Yeah, yeah. So now, okay, this is an important point. There's periods of research and development that occur at companies when they realize that, like, okay, we may have invested lots of time and effort into something, but we can see that there is potentially more malevolence that could come out of this than benevolence, so what we need to do is we need to, how do we package that in a way that prevents other people from leveraging it to make bad acting decisions on the planet? That's right. And so the thing with GPT-2 is it's borderline. When we said this, it was very controversial. There were a bunch of people who said it was so obvious you should have just released it. There's a bunch of people who said it's so obvious you should not have released it, which given the fact that there are these two camps, I think definitionally it's correct to be a little bit hesitant and to wait a little bit and to make sure you're doing the right thing because taking action that might be negative, it's kinda hard to roll back. And so I think that where we are with AI today, and this is one part of our charter as well, is saying that today we publish most things we do, but in the future we're gonna have to decrease that due to safety and security concerns. And this was one concrete example of that. And if you look at the security community and computer security for hack me into systems, it took them a long time, maybe there was like a 10 year period, maybe longer, to really figure out a process for responsible disclosure, right? So you're familiar with responsible disclosure and security. Yeah, so it's a really interesting concept of, let's say that you're a hacker, you're not trying to do anything, unless you're just kind of playing around with things and you find a security vulnerability in a website. What do you do? Who do you call? You email them? Right, so you email them and let's say that they ignore you, they don't fix it. What do you do now? Do we wanna use the only 30,000 days we have to live to continuously invest days of time and to try to help them with the... Exactly, right? So this is a question. So maybe one reasonable thing to do is just to announce this exploit publicly, right? If you just say, here's this thing, they're gonna fix it. Yeah, that's interesting. Okay, so where is this good, where is this going? So where this is going is that there's real questions in the security community about how do you, when you have something that's kind of malicious, you don't want bad things to happen, you want it to be fixed, but it's not being fixed, it's not being addressed, no one's helping. How do you have a process where you do the right thing and what the community converged on is this idea of responsible disclosure, which is you email the company, if you don't get a response, if they're bad about it in various ways, then there's a time, there's some time window at which you can then go public. And if this actually is accepted, you do that, company gets upset at you, the whole community will say nope, you did the right thing, right? This is the acceptable standard. And I think that developing community standards like that, in AI, we haven't done it, right? The thing I just described for security, it took us a decade to do it, it's not obvious at all. For AI, we're developing these systems very, very quickly. And so, you know. Doesn't that also expose it to malevolent actors online? So, yeah. That's right, and you gotta be careful, but if you don't announce it, the malevolent actors are gonna find it anyway. Anyway. And it's never gonna get fixed. Interesting. So you hope that maybe the public can push an action faster. That's right. And just time and time again, this has turned out to be true. Interesting. This seems like a pressing aspect of AI safety and security when people discover these issues. And you wanna have a convention in place before you build the model that you're sure if it gets released is going to be harmful. Yeah, yeah. Interesting. Now, okay. We're kind of hinting at this earlier. I wanna see if we can maybe understand this from a more technical perspective from you that this, and whatever you can teach us, like whatever you guys are comfortable with teaching about this process, but so what happens when you start, you were talking about this issue with transfer, with like transferring intelligence from a narrow task to another narrow task, but then you realize that you guys could write out where you could write a whole story. And so you're starting to play around with creating an artificial general intelligence. What have been some of these wide awakening moments from you of like really good steps to take to continue moving that process forward faster? Yep. So again, in terms of how we structure ourselves, so we have a number of different teams. We have a, one of the teams that I led is to build bots for the very popular video game, Dota 2. Yeah, you guys are competing against the world champions in what is considered to be one of the hardest games to balance strategy, teamwork, resource management. When you're a gamer, you really get these things about how difficult this for an archeologist to be able to beat them is, yeah. And famously no one can program in AI for these games. Until. That's right. And so we have a system that has learned how to play and the way that it learns is that it doesn't know anything about the game when it starts. And a human sees the game as a bunch of pixels. You have this image and you have all this knowledge about knowledge from our ancestral environments about animals and various things. You know that there are these heroes that are running around. You know that if you hurt your opponents, that's good. There's all sorts of things like that. This AI starts out knowing nothing. And that it sees the world as just a big list of 20,000 numbers. That's all it sees. It just sees 20,000 numbers. It somehow represents a game. So this is not a supervised model. This is a reinforcement model. That's right. So I think that, yeah, so supervised learning, I think then kind of bleeds into reinforcement learning and reinforcement learning. You still have a label, but the label is you took a bunch of actions and then you got a point. Now you gotta figure out what did I do to get that point? What did I do that I really deserve that point for? And in the case of Dota, one thing that we do is we put in some knowledge about you should get a point when you successfully defeat an enemy unit, when you go and you get this kind of item, that there's some level of rewards, which kind of provides some high level structure to what's happening. But we don't tell the agent what it's supposed to do. And the way that it learns what to do is through a very powerful idea called self play. Yeah, yeah. This is crazy and it can play against itself at millions and millions of times. That's right. And create, as it does, it can iterate on it's, oh, like why don't we have this version play against? It's just, this is, yeah. That's right. And so imagine if you play against a perfect copy of yourself. Yeah. It's actually the best thing for learning, right? Yeah, yeah. You try something new that you've never done before. It'd be like, imagine if Alan could sit across from Alan and Alan would be literally challenging me to become a better interviewer. Exactly. In all the different aspects of how to become a better interviewer all the time. And I was literally sitting here 24 seven running this process. Yes, yes. And it's exactly that. And so it's a really powerful idea. It builds these really amazing systems. And now, the product that we built, like the end result of this research of this two year development phase of developing the algorithm and scaling them up and all those things is a general purpose system, which we then took and applied to a different problem in robotics. And so we have this physical robot hand, which no one can program. No one can do the old fashioned, like write down all the rules. Like these hands have existed for 20 years. When's the last time you saw a robot hand in operation? Intuitive surgical, but that's not even, I don't know if that's even a hand, right? That's right. And so it's just no one can use these. But we took this system that was developed for a video game and that we successfully applied it to the robot hand to manipulate a block. That went through a process of reinforcement. It's reinforcement learning. To know how to have a strong dexterity when it comes to gripping something. And it develops these grasps that, there's this whole taxonomy of grasps out there. And it develops these grasps that are very recognizable. It's got the pinch grasps, it's got a variety of other ones. And that's all learned, totally from scratch. Well, yeah, that's very human for something that's not human. And I think, and here's where the entry and stuff really is. It's like these questions of you build these systems that can learn where you set the goal, it figures out the process, it figures out the details. Like in some ways, that is the most empowering thing that we can hope for, right? That humans specify the goals, don't have to spend their time trying to figure out like, okay, exactly how to program, like if you're at this angle with this digit, then you've got to live here. And it lets us build systems that can do things like, think about elderly care robots. Those are gonna have to be able to handle totally weird and unfamiliar situations that no one can possibly anticipate and program in. They've got to be really trustworthy, but they're also gonna be so beneficial, right? You think about, we've got this demographic time button that people talk about, right? That we're all getting older and how are we supposed to take care of you and me in 50 years? And I think that this kind of technology, it gives us a path. And so I think there's something amazing there, but the thing that's really important along the way is that the impacts that we see, they help us be more human rather than feel threatened. And then, so one of the open AI teams, there's a hundred of you now. That's right. And one of the teams is doing these reinforcement learning models for Dota 2 and then applying that to other, like you said, this robotic hand, then the other aspects of open AI as well, the security, figuring out how to handle things on a governance and a geopolitical level. So it teaches about that one and the other ones. Yep, so it's something as a technologist just to focus on the technical problems. So we've got this hard technical problem of can we make systems that are smart at all? There's a second hard technical problem of how do we make these actually do what we wanted? We all have heard various sci-fi stuff, but I think that there's really like three classes of risk that we're worried about. The first one is systems that pursue misspecified goals, a little bit of a careful what you wish for scenario. Second class of risk are systems that can be subverted by malicious humans. So that's, if you can hack into an AI and make it do bad things, how do we make sure that that's not possible? But there's a third class of risk, which I think that people really underpay attention to, which is it's possible to solve those problems. We build these trustworthy systems that do what the operator wants, but somehow the resulting economy, the resulting world is not one that results in broad benefit to human lives. That somehow we have these systems that are doing their thing, but it just kind of hasn't made people's lives better. And that one, that's very pernicious. We don't want that. And so we have a safety team whose job it is to think about parts of these questions, the technical parts of these questions, but we also have a policy team whose job it is to think about not just how do we make sure it pursues the goals that it's given, but it's a question of what should those goals be? Like who has a say? How do you make sure that these systems that the world that we build is representative, right? That it really has broad buying that it actually is something that is able to have government involved in the right way, that policymakers know what's going on and are able to put in the right regulations, and all of those questions, if you can solve the technical stuff, are going to become the most important questions to answer. Yeah, the nuance that you just explained safety and security risk mitigation with is very interesting. There's a lot of different ways to get into and to make sure that we're taking care of all the different possibilities. Yeah. And then, I mean, it's just taking us to the beginning where you just said that this is not a passing of a torches, is we're trying to enhance humanity with this technology just like we have with every single other, with other tech that we've built. It also does feel as though that there is a potential to build an artificial intelligence that does not have consciousness embedded in it. There's like this merge scenario that is one that is very popular, that we merge the biological beings with the digital beings with the digital play space that we're making right now. What are your thoughts around the transhumanist movement and towards that digital space and what do you think is potentially best? I think it's really hard to predict. All right, you think about how can you even describe Uber to someone in 1900? Yeah. Like, how do you do it? Yeah, yeah, yeah. Yeah, how would you explain airplanes to someone in 1500? I guess that wouldn't make a little more sense. It's like, we'd basically be the bird. That's right, that's right. Exactly, right? And the mechanism would make sense and there's a lot of concepts there. But I think that, and I think one thing that really distinguishes information technologies from a lot of the physical technologies that we're used to is almost this degree of like it's so hard to anticipate and understand. Even the internet. The internet in some ways, like if you described it to me, I'd be like, I don't really know why I need one of those, right? It's like a system that lets people connect like, I don't know, we already have like the post, like what's the problem, right? And, but suddenly you get these capabilities and they change the world in surprising ways, right? You get the capability of, you know, I can basically instantly, psychically communicate with anyone on the planet. It's called the text message, right? You know, I have to like type something with my little fingers. But like, it's not really any, you know, if I actually literally had some top of the built-in, I don't know, it'd be any better. And so I think that the technology we've built has already given us superpowers in very, very surprising ways. Definitely. And so I suspect that what's gonna happen when we have even more powerful technology, it's just gonna be hard to anticipate. I like how you give the hefty dose of humility that's needed. It's just, there's so much conversation that's happening about what's happening in 10 or 20 or 50 years and all this type of stuff. And yeah, it's also equally important to aim to extrapolate what we do have today with this cutting edge neural network technology and see where it can be applied to maximize human potential incrementally along the way. And the medical field is one of the big ones and preventative health cares, also with the medical imagery making better decisions there. I'm curious to hear, what are your thoughts around the newest protocols with decentralization technologies being at play with artificial intelligence? Because you guys also do have a new aspect of open AI that's enabling you to kind of gain, it's called the LP. That's right, yeah. And so in some ways, maybe two parts to that question, what is open AI LP? And in some ways, open AI LP is an implementation detail of the open AI charter. Right, so we have this charter, it says here's what we're about, we care about the broad benefit, we know we're gonna have to raise huge amounts of resources, but we're gonna make sure that that mission is served. And it has a cap like you said. That's right. And the way that we think about this cap, right, so we kind of felt that, so we started as a nonprofit because we have this mission, it seemed like the most obvious implementation of the mission. How are you supposed to have a for-profit company that's going to benefit everyone? It just felt almost incompatible. And we spent really a long time trying to figure out is there a legal structure that gives us what we need where our primary fiduciary duty can be to the charter rather than to narrowly making money for investors. I don't think that there's anything wrong with returns for investors, I think that people will take a risk. There should be incentive for them to do it, but I think that this technology that we're talking about is just so powerful that we gotta make sure that we have a focus on making sure it goes well. And I don't think those things should come into conflict, but it's really important that your priorities are clear. An inclusive stakeholder environment. That's right. And so we basically looked at every legal structure out there and concluded it wasn't quite the right one. And so we ended up custom writing rules for an LP and to find what we call this cap profit. So the way that it works is that investors or employees have a similar sort of like stock option granted to them have a cap to return. So they put in some money today, they get a return, but if the company happens to generate more value than that cap, that value doesn't belong to any of the humans at OpenAI. It doesn't belong to the AGI either. It belongs to the nonprofit organization. That's very interesting. And so you wrote your own code for an LP that yeah, because there was no available option to it before for that before. And so, and then from there, and so it's possible that the way that how should AGI benefit everyone isn't in the form of cheap or free AGI services, the AGI doctor, maybe, right? And if that is, then we have the flexibility to do that or is it in the form of something that somehow generates capital and that you return that to individuals in like actual distributions, also possible, right? But the point is we're not locked in to any of these outcomes, but the thing we are locked into, that we are committed to is the mission that AGI will benefit everyone. Yeah, yeah, and then this is where the decentralization component is kind of interesting is that where do you also balance out a centralized OpenAI 100 employees with an open protocol for other people from around the world to help augment? Yeah, so I think that first of all, I think that there's a lot of really interesting things going on in the blockchain space and in particular having these decentralized technologies. I'm actually on the board of Stellar, which is a nonprofit crypto system which has the mission of giving financial access to the whole world. And that was a project that I helped launch in 2013, 2014, I've been really involved since. And there's some really interesting lessons from what we've seen with Stellar. In fact, when we initially launched, so there's this coin built into the network that we just gave away to people for free. That's part of the mission. You wanna give everyone financial access, give everyone a stake in this network. And it was literally the case that if you signed up with a Facebook account, you'd get this Stellar coin and it was worth about $10 at the time that we launched, maybe $15. And so it was literally, you have a Facebook account, you go, you sign up, you get $10 to $15 worth of coin and you can actually literally sell it the day that you got it. It's kind of this amazing experiment in a lot of ways, right? Just like if you just let anyone in the world sign up for this thing, what happens? And that we saw this real exponential growth that grew faster than anything I've ever seen in terms of people claiming these coins, maybe not unsurprisingly. But the thing that was very surprising was we looked at where the signups were coming from. And there was this very massive growing base of people in countries like Vietnam or the Philippines. I started to dig in to figure out how do these people even hear about it? What are they doing with Stellar? And we started to find these forum posts where it turns out that in a lot of these countries there are these businesses that create legit looking Facebook accounts. And so there's a lot of individuals who have a lot of Facebook accounts that are fake. And at first, I was furious, right? It was we were trying to give financial access to everyone in the world and these people were stealing. They were taking that away from others. But then you start to read these forum posts and you see these people would get their Stellar coins, they would sell them and they'd buy a goat for their village. Interesting. And you're like, this is amazing. This is like almost a wealth transfer from these people, these crypto speculators in the first world to these people who just want to go for their village. Like, look at that impact. That's awesome, right? I didn't mean for that to happen, but kind of cool that it did. But then you fast forward about three or four years to the, do you remember the height of the crypto bubble of I guess early 2018, late 2017? We were just like everything was skyrocketing and it was all very high at that peak. Those Stellar that we gave to people was $10, $15. It was worth about $4,500. Wow. And suddenly you realize that this system resulted in a goat for this village, but resulted in these like cars for the crypto speculators. And the thing that I think is really interesting, like I don't think there's any like, you know, it's just interesting to look at the system. Like I don't think that, you know, it's us about kind of the, you know, the morality of it, but it's really more about the like, there's this tendency for capital to accumulate more capital, right? The people who already had the capital were able to speculate, therefore we're able to reap the benefits of this platform succeeding. And so I think that the same is important to look at with the future of AI, AGI, right? I think it's not enough to solve the technical problem of giving everyone access at a technical level, giving everyone a key, that sort of thing. Yeah, malevolent actors can take advantage of that. And the thing about the stellar story, so it's definitely malevolent, you gotta watch out for it, right? But the thing about the stellar story is no one's really malevolent, right? You know, there's the people in Vietnam who have these, you know, Facebook accounts that weren't legit, but I don't really view like the transactions that happened to be illegitimate, right? You know, the crypto speculators, it was like a fair transaction, right? They did get the goat, they took the risk, you know, all those things, but somehow that system was set up in a way where it didn't result in the equality that we were hoping for. And so with OpenAI, I think it's really important, and with AGI and these other technologies we're building to learn from that and to think about how do you set up the system? Not, you know, again, like the technology is almost a servant, you know, it's a servant to the impact, right? It's a vehicle, it's not the most important thing. The most important thing is what happens to human lives? And so that's the thing that I really focus on, and I think that's a really hard and really important problem. Yeah, and what is a skill that children should learn as we enter into the automation AI age? I mean, so start with programming, right? Everyone should learn to program, I think that's- Everyone. I think so, I think so. I think it's a really important way to understand how technology works and how these software systems that we all interact with, how they operate. Yeah, yeah, interesting. Okay, wow, this has been super enlightening. We have a couple of quick questions that we usually ask on a show to ask you, Greg. Yep, yeah, go right ahead. All right, first question is, are we alone in the cosmos? Not if we succeed at building a GI. And unpack that a little more for us. No, I think it actually stands alone. Yeah? Yeah, I think that, you know, I have real hopes for the what happens if we succeed at building these really smart technologies and I think that we can build systems that will enhance human lives, but these systems will also be different from us in real meaningful ways. And so I think that there's something really amazing to discover there. So that AGI ends up helping us discover other? No, no, no, I was looking for AGI. More so that AGI is its own. I think there's a chance. In some ways it will be an other, and in some ways it will be like us, and I think we get to control that. Okay, okay. And then are we in a simulation? Hard to predict. I'm not sure what I would do differently if we were in one versus not. Yeah, you'd keep leveling up regardless. Yeah, and I think that it's, I like to separate out the kind of concrete things that I can have scientific intuitions about. I think that for this question in some ways it feels like a non-scientific question in the same way that just like, is there a flying teapot out there between Mars and Jupiter, there might be out of no way of disproving it. But I'm also not sure what changes in my life for this one. Yeah, but that one is a little bit more, you're just kind of like throwing that, like a more of like maybe a meaningless question out. This one is probably a more meaningful question that can actually be probed with science. Mm-hmm, yeah. Potentially, but I think I would love, I would love for someone to try. You'll maybe in the couple of decades with AGI we'll be able to run the simulation of a big bang and then see ourselves 13.8 billion years later. Yeah. It's hard to rule out anything. Yeah, yeah, yeah. Last question, Greg, what's the most beautiful thing in the world? My girlfriend. This has been so enlightening. Yeah, there's, and we were very grateful that you've banned together some of the greatest minds out there to make sure that we do AGI right. And that helps for our generation, also the older generations, but especially the younger generations to see your charter, your principles, your first, your values that you're building this with and that helps them also want to be driven towards similar first principles and values and that's very powerful, very powerful. Thanks, Greg. Thank you. Thanks for coming on the show. I really appreciate it. We greatly appreciate you all for tuning in. Thank you very much. We'd love to hear your thoughts in the comments below about what was discussed here. Go and talk to more people about artificial general intelligence, what exactly goes into this, the power of neural networks. Everyone get programming. That was Greg's take away, get programming. Understand that this is a major part of our future and that everyone should be at least basically fluent in it. And huge shout out to Ron Vagus. Thank you very much for producing and directing. We love you very much. Everyone, support the artists and entrepreneurs that you believe in. All of the open AI's links are below. Support them, simulation. All our links are below. Support us. Support the ones in your community. Get them growing and flourishing. Go and build the future. Manifest your dreams into the world. We love you very much. Thank you for tuning in and we will see you soon, everyone. Peace.