 Welcome to this town hall about how to trust technology, particularly in an age of AI. I added that last little bit, but unsurprisingly, AI has been a major focus both here in Davos and in the world generally, and I suspect that will be the case for, well, basically forever. So this town hall is designed to be a forum for discussion and an opportunity for transparency. To that end, the primary driver of the discussion is the audience, so both you here in the room and also online. After initial opening statements from our panelists who I will introduce in a moment, we will open the floor to questions. Those of you in the room, you can simply raise your hand, we'll bring a mic to you, you can ask your question. Those of you online, you can use this app called Slido, which you can access at a link on the website for this session. As a quick introduction to Slido and as a way to sort of ground the discussion, there are two poll questions that you can answer right now. The two are first in the past year, has your trust in technology increased or decreased? And second in the past year, has your trust in organizations in general, not just in tech increased or decreased? We will come back to the results in a bit. Please note that you can also vote on the questions other people submit online to surface the best choices. For now though, let me introduce the panelists who will be answering your questions. Over here on the right, we have Dr. Ayanna Howard, is the Dean of Engineering at The Ohio State University. She holds a faculty appointment in the College's Department of Electrical and Computer Engineering, as well as Computer Science and Engineering. Before Ohio State, Dr. Howard was at Georgia Tech, and before that NASA, where she worked on designing advanced technologies for future Mars rover missions. Dr. Howard's research focuses on AI, assistive technologies and robotics, and I think all of those are certainly very pertinent to the broader discussion and the discussion for this panel. Here on my right is Mustafa Suleiman. Mustafa is co-founder and CEO of Inflection AI, an AI-first consumer software company. Before that, he co-founded DeepMind, a leading AI company that was acquired by Google and is integral to sort of Google's AI efforts. As a part of Google, Mustafa was responsible for integrating the company's technology across a wide range of Google's products. Mustafa is also a newly published author of a book called The Coming Wave, published by Crown in the US, September 2023. I think you can all guess what the book is about. So with that, I would like to pass it on to our panelists who give sort of a brief opening statement about the topic in general, about what they're working on, and so Dr. Howard. All right, so he said brief, but you know I'm an academic, so brief is all relative. So when I think about technology and trust, I think about the research that I do. So back in 2011, my research was focused on how do you evaluate trust, human trust, with respect to robotics in high critical, time sensitive scenarios. And so we picked emergency evacuation. And so we had done scenarios where we would have, like this room, and we have people interact and the fire alarms would go off and what would you do? You exit. As you exited, the building would be filled with smoke and there would be a robot that would guide you to the exit. And we intentionally made the robot not go toward an exit. So you could see the exit signs and we would have people go to other places. And what we found out over and over and over again, even when we introduced mistakes, even when the robot had bad behavior, people would follow the robot. And so when we looked at this to say what's going on, we actually found that people actually trust or over-trust technology in that they believe it works most of the time. And when it doesn't, that's when they then swing to the oh, like airplanes should always fly. If they crash, okay, what's going on? It's the developers, it's the companies. We need a ban and make sure that no one ever flies again. And so we have this over-reaction. And so really thinking about technology and trust is how do we deal with the mistakes? Not necessarily with the fact that we under-trust, we tend to over-trust in essence. Yeah, I think this is a critical topic for LLMs because we're still at the very earliest stages of developing these models. I mean, they are inherently probabilistic models. So as an intuitive grasp compared to the past wave of technology, it's very important to separate what we had software do in the past, which was you input some data to a database and then you make a call of that database and you collect pretty much the same information out of that database. Whereas here, of course, with LLMs, you can ask the same question of the model three or four or five times and get a slightly different response. And many people have referred to this as hallucinations. I actually think about it more as creativity. This is one of the strengths of these models that they produce a wide range of different responses that you never anticipated before. And that's exactly the magic that we've always wanted software to produce. The downside is that our past mental model of default trusting technology, as you said, doesn't really apply in this situation. And so I think at this moment, we have to be incredibly critical, skeptical, doubtful, ask tough questions of our technology. And we can talk about the ways to do that today. I think there's two elements that will drive trust in this moment as we develop more consumer-facing applications. The first is, of course, the extent to which models are factual. It's IQ, right? And that's actually something that we can formally measure. So there are lots and lots of benchmarks today that try to evaluate the freshness and the factuality of models. And what we've seen in the last three or four years or so is that there's been a steady and quite predictable increase in the accuracy of the models as they get larger. The second component is the emotional connection that you have with a model. What is its EQ? Like, how fluent and conversational is it? How kind and respectful is it? Does it reflect your values or does it antagonize you? Is it formulaic or is it adaptive? And of course, many of us like to think that we're rational actors that make decisions based on knowledge and facts all the time. But in fact, we largely make decisions using our limbic system. And it's our gut that actually drives a lot of our decision-making. And that's gonna be the same in technology, especially now that LLMs are these dynamic interactive modes. So we have to think very carefully about what values are in the emotional side of the model, who gets to oversee them, how are they programmed, to whom are they transparent and accountable. And those are the kind of skeptical and critical questions that we should be asking of all new consumer technologies. Dr. Howard, I'm actually quite curious. Because you've worked in this field for so long from robotics to AI. And this shift that Mustafa's referring to, going from being deterministic to probabilistic, how has that shifted or changed the way you thought about sort of these questions for both AI and robotics? Yeah, so I actually think around the human aspect. And so I just wanna ask a question. How many of you have used chat GPT or BARD or some equivalent of that? Okay, that's I would say 100%. How many of you have used it in any form of function? Like to actually do a job, do a work, do, okay. It hasn't changed, right? We know that there's mistakes. We know that it's not perfect. We know that lawyers put in briefs that are written by chat GPT and they get slammed by judges because it's incorrect. It really hasn't changed. I think what's changed is that we now understand the risks. We haven't yet figured out how do we address that as a society because it is so useful. It is so valuable. So when it's right, it really does make our work life better. But when it's wrong and we aren't using our human EQ to correct it, we have things that go bad. And my thing is, is like, how do we think about the human EQ aspect and blend that into the tools? I actually think the robots in AI should say, you know what, you've been using me too long. We're done today. I actually believe that. From your perspective, how has this communicated? Like how do you, you know, Dr. Howard is concerned that people are too trusting. You know, in our poll questions, we have the increase or decrease in trust and technology. And I actually asked to add in the also what's your increase and decrease relative to general institutions because it does feel like you're sort of getting at this point of revealed versus stated preferences. People will talk about, oh, tech is bad, big tech, X, Y, Z. People use it all the time. And to your point, they really do implicitly trust it to a great extent. Is this something that needs to be communicated to people or is it just on technologists to get it to a state where that's okay and it'll be right sufficiently enough at the time? I think both are true, right? So on the one, so we create an AI called Pi which stands for personal intelligence. I'm a big believer that everybody in the world will have their own personal AI in the future and it will become your aid, your chief of staff, your teacher, your support as you navigate through life. And we made a couple of important decisions related to what Yana said. Number one is at every screen, we leave in place a system reminder that you shouldn't trust what the AI says, that you should be skeptical and ask critical questions. It's a small thing, but it's a constant, ever-present reminder. It changes form and structure in UI so hopefully people don't get desensitized to it. The second thing is Pi itself will remind you after a 30-minute interaction that it's been pretty long now. Like, how do you feel about getting back to the real world, right? And it would gently ask you in a very non-judgmental and polite, respectful, kind way to start thinking about the world around you. And the cool thing about it is that instead of just having a system notification, it actually comes from the source of the interaction. So I think both of those things are really important but at the same time, people get fixated on the technical progress that we have at this moment in time and don't really pay attention to the curve that we're on. We're on an unbelievable trajectory. It is truly magical. And even though everyone's played with chat GPT and spent a year now talking about AI, I still feel like we're not fully internalizing what's actually happening. So maybe just one possible intuition. Everyone's used an LLM of some sort, maybe for a real world practical use case. The latest models are approximately human level performance across a wide range of knowledge-based tasks, right? They can produce very high quality content. Two generations ago, so GPT-3 and then GPT-2, that was a 100x less compute. So each generation is 10 times more computation. So it's significantly larger. And 100x ago, these models were completely incoherent. They produced totally in-factual, completely made up, not even a proper sentence, let alone factually accurate. So you have to sort of try to extrapolate when we train the next two generations what capabilities are going to emerge, right? And my bet is a couple of things. One, we're gonna largely eliminate hallucinations. The factual inaccuracies are gonna go from today when they're hovering around sort of 80, 85%, all the way up to 99 to 99.9%, right? An order of magnitude improvement over the next three years. The second thing that I think in terms of capabilities that are gonna emerge is that at the moment, these are one-shot question-answer engines. You ask something, it gives you an output. It's kind of like a one-shot prediction that's accurate. In the next two orders of magnitude, the model will be able to produce an entire string of accurate predictions in sequence, some of which are code, some of which are images, some of which is text, and that is gonna produce what is much more like a project plan or an entire analyst's briefing on a complex topic or a series of API calls to a whole range of different third-party services, and that's gonna be truly transformational. That's when the AI will be able to take actions on our behalf. And I think there'll be a whole different trust question in that environment. One more question, then, prepare your questions, I will hand it off. But one aspect of an exponential curve is, yes, we are thinking about sort of the steep part, and it's steep now, and it's going to get steeper. But before the steep part, it's very flat for a very, very long time. And I'm just curious, we were sort of talking before the panel, Dr. Howard, about what has it been like to sort of over the last year in particular, over the last few years, suddenly everyone is an AI expert, and it's sort of been sort of like hacking away at this question for such a very long time. Like, what's it like to, like, oh, is everyone welcome to the party, or what does that feel like? Well, some aspects, you become the cool kid on the block, but the other aspects, it does worry me. And I think about, I'm gonna turn the dial back. Back in the days when electricity was born, we actually had a lot of inventors that were like, oh, we can do this, we can do incandescent lamps, and light bulbs actually exploded in people's houses. Like, this was a thing. And everyone was like, oh, but we have light, but yes, there's danger. And it took a while before the rules and the regulations and UL and CE came about. This is, okay, if you are an inventor in this space, there are some rules of how you do this. You have to have some certification, you have to have some validation. We don't have that in AI. And so basically anyone can participate and create and hook it up to a machine. And it's like, oh, this is good. And we have people who don't know what they're doing that are selling to consumers that also are trusting. Like, oh, well, this company, it's got a VC investor. Yeah, let's bring it in. That's what worries me. We do have our initial pull results in the past year as your trust in technology increase or decrease, stay the same, high. That's what I voted for personally. Decreased, however, on its heels of 33%, increased 70% and hard to say 18. Do we have the second question by chance? We might have to wait to get to that one, but it is interesting to consider what that might be relative speaking for you too. Has the advent of these large language models and I'm going to limit it to the chat GPT era because I think that's what is sort of awoke, the, you know, everyone else woke up to it. You two have been working on it for a very long time. From your perception on the inside has, where would you answer this question? This is the organization one, but your trust in technology. I'm one of the weirdos who have been working on this for 13 years whilst it was flat. And we were making like very modest progress and you had to get really over excited about a very tiny, relatively tiny breakthrough, right? And so to me, I trust them more and more, like way more than I thought I would two years ago. When you look at the quality, I mean, clearly talking to one of these models, like is magic, I mean, provides you with access to knowledge in a fluent conversational style and iterative back and forth. I'm using it all the time to ask questions that I didn't even know I wanted to know about. Now it's sort of because I've reduced the barrier to entry to access information in my mind, I've started to kind of like subconsciously condition myself to ask more things than I would have asked a year ago or two years ago because I subconsciously would have thought, oh, I'm going to have to Google it, look through 10 blue links, go to the webpage. And it was like the bar's quite high to asking a question there now. So that's made me more trusting of it, actually, because I can now ask more wide-ranging, interesting questions. More or less trust? Of technology? Yes. I'm actually more trustful. And I would say that one, because I can query it, I actually know what it's doing. And I'm actually more positive of what's in the black box because I know exactly what's going on. So more of it's because I'm not worried about it explaining itself. And I understand the developers that are part of this because we've all grown up together. Organizations, that's a whole nother question. Are there any questions in the audience? Yes, we can start here in the corner. Unfortunately. What I still can't understand, and by the way, I loved your book, but I still cannot understand it. Why is Silicon Valley so obsessed with AGI, especially the big companies? I mean, if that same level of obsession could be maybe put on solving climate change or getting manufacturing fully automated so we could see that finally, the promise of productivity deliver, wouldn't that be more useful? Yeah, it's a great question. And I hear and share some of your skepticism, but let me make the bull case for you so that, you know, to answer your question. So look, I think what is amazing about us as a species and unique about us, as humans, is that we can absorb vast amounts of very abstract and very strange and contradictory pieces of information. We can digest it, process it, we can reason over it, we can generate new examples of that and imagine and be creative and inventive, and we use that to produce tools. And that is the essence of civilization. So the AGI quest is really a game to capture the essence of that learning process and apply it to all of the big challenges that we face this century, from food to climate to transportation to education and health. I mean, that's really what's at the heart of the motivation, I think, of a lot of people. There is a group of people who are just, you know, slightly evangelical kind of transhumanists who think that there's a natural evolution. I'm absolutely against that. I'm very much a humanist. And I think this is going to be a moment when we have to make some decisions as a civilization about what kinds of agents, with what sort of capabilities and powers we allow in society, right? Because very soon, in the next three to five years, there will be AIs that are really equivalent to digital people. I mean, they will have the smarts that we have, they'll be able to speak to you and I just as we're having this conversation right now. And many people will ascribe self-consciousness or awareness to those models. And those models may sometimes say that they suffer. So many humans are going to feel very empathetic to those claims that this AI is suffering and therefore we shouldn't switch it off. Well, therefore maybe we should give it certain rights and protections that humans have. And I think that's going to, if we just treat that as a philosophical debate, that's going to be very, very hard for the humanists to win. And I think that we should really draw a line in the sand and say, look, you know, society today is for humans only. And that's how it should remain. Just as we say this year that AI shouldn't participate in elections. Why do we say that? Sort of an arbitrary choice. At one level, an AI could be really useful in election. It could actually provide very factual information to everybody. But I think it's a principled decision to say, democracy for all of its flaws is for humans. It should remain for humans. AI shouldn't campaign or participate or electioneer or persuade. And we just should deal with the consequences of human weaknesses in the electoral process for exactly this reason about the future of AGI. Actually, I'll answer a few because I think I'm a little bit of a pessimist. My feeling is this, I'm outside of Silicon Valley, but my feeling are two. One is that the organization that creates a true AGI will control the world unless we have some regulations and rules. So that's one. And so if you think about just the philosophy of creating, it's like first to the ship, gets on the boat and wins. The second is that as humans, we fundamentally wanna create. And AGI represents that ability to not just create our physical, but also create our mind. And so I think if you think about the humanists, it's just the natural evolution of we as people, of creating something that grows, that learns from us. But that is the next rendition of what we can be. And so those are kind of the two things that I've seen as philosophy. That's a great point. One question I have though, just to jump in, particularly with your background in robotics, it seems to me, is there a line between the digital world and the physical world? One of the questions we have here is are we on to lose our jobs? And I think particularly for anyone that is in a digital space, that is a very sort of pressing concern. But at the same time, it seems to me from the outside, I'm no robotics expert. That's why we have you here. Is there a much further runway between, say, an AGI that operates in digital space versus one that operates in the physical world? I think in terms of the function, yes, but I will say the worlds are starting to converge quite rapidly, amazingly rapidly. It used to be I was a roboticist and AI folks. Those were software people that actually didn't know hardware. And they would say, oh, your hardware, your robots are really stupid. And now we have a nice blend, especially with being able to connect to the cloud and actually learn almost in real time. So at some point, it's just not gonna be a digital persona. It is going to be a physical. And we don't necessarily have the skills or tools to really think about what that world looks like. But there is the constraint, though, that you have to actually manufacture robots. It's not like digital information where endlessly duplicate or something on those lines. I mean, what's your perspective? Yeah, no, I think you're right. That is a constraint, which means that robotics is sort of gonna remain behind for quite a while. But to my honest point, these fields are converging in a way that, during this flat period of the exponential, they've been very separate. Like they've sort of been enemies. It's like, robotics does symbolic reasoning. And if this, then that rules. And AI is trying to build learning systems. And now you're actually seeing them converge. So I agree with your server motors and physical infrastructure is gonna always be a constraint. And that's going back to what we were saying about building on the shoulders of all of the cloud investments that have been made over the last decade and all the devices we have. All of that distribution infrastructure enables this rapid deployment of AI. I mean, I think it's gonna be the fastest proliferating technology in the history of all technologies. Let's take another question from that. Yes, right up here. We're one of the biggest risk takers in cyber insurance. And I think one of the reasons you trust AI is because you're good people trying to do good things. There's a lot of bad people who are gonna try to do bad things. And legislation and rules and regulation aren't gonna prevent them from doing it. So my question is what is it that you would like to build in or what are the risks that you see that we don't see in how to deploy this, not for the benefit of society? That's a great question. The reality is that the more centralized the model, the easier it is for some number of regulators to provide oversight. If it's completely open in five to 10 years time when these models are four generations ahead, it's clearly empowering everybody good or bad. So the fundamental debate in the community at the moment is there's absolutely no harm, I think, today being caused by open source. And we should be accelerating it, encouraging it. It's been the backbone of software development for as long as software has been around. At the same time, there's a very legitimate question around when that point arises. And I think that's the dichotomy. Even in the centralized providers, APIs and so on, there are still gonna be a lot of hard questions about how to interrogate these models to limit their capabilities, to restrict them. I mean, it's almost like the kind of social media moderation question all over again, but in an even more abstract way because these threats are described in time series data rather than in language and words. Is there an aspect of the solution is actually going to be other AIs? I worry when people say that because I think it's the silver bullet that people always say. And I think we shouldn't be too complacent on that. Of course, there are gonna be AI systems that help. And we already have all kinds of pattern matching systems that are detecting insurance fraud, credit fraud. To the cyber security point, that's sort of the game as it is today. Definitely, it is part of that. But I think it is, realistically, it's also about experimental deployment. Like you have to put things into production, expose them to the world to really identify their flaws and weaknesses. And I think that's actually been one of the great achievements of the AI community in the last 12 months that everybody's got a chance to play with these LLMs, poke holes in them, demonstrate their weaknesses, publish papers on them, try and deploy them in certain applications that fail. Like that's the correct model. And I think that we certainly can't slow down on the deployment or integration side. I think in cyber, there is a move, for example, because you can't eradicate 100%. And so there's a move towards zero trust. Like assume that you're gonna get hacked. Assume that there's bad actors. And so how do you design your processes, your frameworks to deal with that? In AI, there's really no standard of saying, okay, let's just assume the AI is bad. How do we design our interactions such that if we assume it's bad, what do we need to do on the human side? Or what do we need to do with the hardware? What do we need to do with XYZ? There isn't an equivalent movement I haven't heard of. I think you're right, there isn't. Although if you look at it, the standard that we expect from an AI system is much higher than what we'd expect from a human performance, right? So in clinical care and diagnostics, we already have models that can detect all kinds of radiology questions, predict all kinds of acute kidney injuries, sepsis from real world data at human level performance, but they don't get deployed because they have a much higher standard. Same with self-driving cars, right? The AI car has to be much more safe and reliable. So in a way, we do have built-in skepticism of these models, but we certainly don't have zero trust. Like that's for sure. I'll take a question from online. How can we trust technology if there are no policies behind it? Or if we can't trust the policymakers? To what extent, I think it is easy for people to reach to government, reach to regulation. Is that the answer or is there a better model or a companion approach to that sort of, which is itself its own centralization of a sort? So I wear two hats because I actually dabble in policy at least with respect to AI and regulations. And I think we'll take out the trusting policy or policymakers, like let's take that part out because it varies by country. But I think one of the things around policies and regulations is that it allows you to at least start off with equal footing of what is the expectation. And if companies or other governments violate that, there is a ramification. Now, some companies can pay, so it's like, oh, whatever, but there's still some concept of a ramification if you violate a policy or regulation. Right now, we have so many things going on. You have things in the EU, you have things in the US, you have things in Japan that are like sort of combined, but they're all slightly different. In the United States, we even have states that vary in regular. California just released one versus a federal. That is the problem in terms of trusting the policy because we don't have a uniform thinking process of what does it mean when we talk about regulations or policies and use of AI for the good of humanity? Yeah, I think that on the policy side, the curious thing about the models is that as they get larger, they get easier to control. And the policy that drives their behavior is also expressed in words instead of in code. So that's quite a significant shift compared to the history of previous software that you're trying to control. Should all these companies, should they be exposing their full prompts just for transparency reasons? Yeah, I mean, the prompt isn't the only way to control it, like it's actually an entire process of learning from feedback and so on, but I think there's a pretty good case for that. I think the other thing is that you could actually look at the outputs, test the outputs, because you can ask a whole battery of test questions and evaluations. And I think what I'm starting to see is rather than a formal legislative approach today, a lot of the governments are getting behind new evaluations for bias and fairness or for increasing the risk of biohazards, for example, coaching somebody to create a bomb or something like that. Which to be fair, you can look up on the internet right now. Which you can look up on the internet. Or your public library. Yeah, exactly. So the question is whether it makes that easier and reduces the barrier to entry in some dramatic way. But the good news is you can actually just stress test these models. And so there's gonna be a battery of like automated questions or attacks on the models, which I think will help give people a lot of reassurance. Oh, let's take another question from the audience. Yeah, right here in the center. So I'm interested, when you assess trust in a human individual, we generally use a framework that's built around capability and character. So how people do things and why they do things or doing things and doing the right things. And then we take that and we put it in context because trust is really only useful when it's contextual. So no offense, but asking people if they trust organizations to do what, right? I'm really interested if we have found a framework yet to assess what trustworthiness means in AI and whether there is a danger in applying a human framework of behavior onto AI. That's a great question. I don't know that there's a definitive framework, but my mental model is the following. So I consistently trust people that are aware of their own errors and weaknesses and lack of capability. And so uncertainty estimation is actually a critical skill of these models. So let's just imagine hypothetically that we're always gonna have this hallucinations issue. They're never gonna get more factually accurate. Well, other than trying to get them to be factually accurate, the other trick is to get them to know when they don't know and to be able to communicate a confidence interval with every generation. I'm not quite sure on this. I'm a bit skeptical. I really don't know or I can't answer, right? You could think of an entire spectrum of things on that scale. And if it was consistently accurate with respect to its own prediction of its own accuracy, that's a kind of different way of solving the hallucinations problem. Likewise, if it says, well, no, I can't, you know, write you that email or generate an image because I can only do X, that's increasing your trust that it knows what it doesn't know. So I think that's one framework for, you know, that isn't exactly how we treat humans. At least we don't do that explicitly. We have a kind of subconscious clock ticking that we may be not always fully aware of. You know, did this person do what they say what they were going to do? Were they late? Did they not pay me back? Did they always say the wrong thing? Have you said five different things in the last 10 minutes? That kind of thing. Like that's our ticker, I think, for trust. Can you bait humility in? Yeah, you can bait, because obviously the model is on a spectrum, right? It's constantly producing the most likely next token. So you can do, you can adjust its confidence interval to make it more self-doubtful. If you talk to Pi today, for example, on sensitive topics like a conspiracy theory or a breaking news story, so Pi has real-time information you can talk to about what happened at Davos yesterday or what's going on in Gaza. And if it's a sensitive topic that it's not sure about or if there's a lot of conflicting information, it will often reserve judgment. And by default, we've actually made it quite cautious if it's sensitive or if it's conflicting. And that's a safer place to be. It's not as useful to the user if we were a bit more bold, but I think in the long term, it's the way that we try to build trust. Yeah, so I will just say just on last note, there were 122 definitions of trust and technology. And so it is a moving target and I think he's correct. One of the movements is not just about capability, but it's about capability and the interaction with the human. I think that's the one definition that is coming to, I would say, have more prominence in the whole trust technology area. There's an aspect, I think, of this framing, just sort of this discussion in general. And it ties into a bit of the regulation question where why would you not want regulation or why would you be wary of it? And I think the worry is sort of a classical one, which is what sort of benefits or capabilities become foreclosed that sort of never get developed. There's a question here, you know, if it's anonymous, so I can call them very cynical, they say, why should we trust technology besides profit? How does it benefit humanity? And I think that one thing that occurs to me, I'd love to hear both your sort of take on this, is is there an aspect has, have technologists done an insufficient job talking about why this is a really big deal and people ought to be excited. And certainly we should be aware of and address the issues, but why is this so important? I feel like all we do as technologists is talk up the benefits and, you know, that people accuse us of sort of hype and stuff. I mean, we couldn't be... So has there been an overcorrection in this wave relative to social media? Maybe there is, maybe there is. I mean, look, I'm not... I'm not sure that, like, that's necessarily the right framing. I mean, these models are clearly delivering provable benefit, and to the extent that it's a useful tool, the way to distribute it in the market is to generate profit. Like, profit is the engine of progress that has driven so much of our civilization. It doesn't mean that it shouldn't be, you know, restrained and regulated in certain ways. But you're not paying with their wallets. They're making that choice with their wallet. And I think it is important that we start talking about which capabilities are potentially in the future going to create more risk, right? And that's where I think we need to start thinking about regulation. So, for example, autonomy will clearly create more risk in the world. If an agent has the ability to take a whole series of actions independent of human oversight, unquestionable that that creates more risk. Or the crossover to the physical world. Yeah, if it interacts with the physical world, there's clearly more risk there, right? No question. If it's a high-stakes environment, like healthcare or self-driving, clearly creates more risk. If the model has the ability to self-improve without human oversight or a human in the loop, it's kind of a version of autonomy. But if it can adapt its weights, learn new information, change how it operates without human in the loop, clearly has more risk there. Like, no question. If it is inherently general, right? Clearly there's more risk. If it's trying to be good at absolutely everything simultaneously, it's going to be more powerful. So, whereas the flip side of all those things, if it's more narrow, if there's more of a human in the loop, if the stakes are lower. So, there's clearly a framework where, you know, regulation is going to have to intervene at some point in the next few years. Yeah, so this is the story of technology in general. I remember back when laptops and internet came in, it was like, oh my gosh, the world is going to get destroyed and there's going to be the haves and have nots. And I would say the internet has really leveled out and made the world a little more equal in some aspects. If you look at Africa, when they went from landlines, they just over, like, leaped over to now have cell phones. And actually have connections in terms of the room of communities. So technology, I believe, is always moving forward. I think the problem is, as technologists, we aren't trained to be social scientists or historians. I mean, traditionally, we're positive because it's our field. It's like, we're in this field because we love it. And then someone's like, yeah, but it's bad. No, no, no, but it's perfectly fine. It allows all of these opportunities. And I think that is one of the things that we do really bad as technologists. We don't necessarily build bridges with others that can translate what we see as the positives. And we know some of the negatives as well, but why are we gonna get rid of our own jobs? That's not gonna ever happen. And so I think this is a room for improvement, is how do we as technologists build bridges with others that understand us, understand the technology, but can also translate that in terms of the risks, as well as the opportunities in a space that's more holistic. We have time for one final question. So we'll take it from the audience if there's any takers or the table is yours. I would love to hear your take on the future of LLMs. I mean, is the size game gonna continue and how long and what does the future look like? And if time permits, would also love to get your take on this, the open versus closed debate that's going on right now. I think the short version is that the models are being evaluated against the threshold of human performance. And that's a fixed threshold, like how knowledgeable I am, how creative I am, how empathetic I am. And yet the models are pushing through this curve over time, right? So that's one trajectory, bigger happens to be better. But as with all inventions, once we achieve a certain state of capability, there's huge pressure to fix the same performance, threshold and make it much, much, much smaller. So today you can train a GPT-3 level capability model. Let me just get this right. At 60 times smaller in terms of flops than the original 175 billion parameter model, which crudely means 60 times cheaper to serve. Any time you ask a question of it, it's you're paying that much less in computation. That's a phenomenal trajectory because you're getting performance increases from scale and efficiency gains as they get smaller. So that's definitely gonna continue, which is good for the small ecosystem and open source and startups. And obviously it's good for the absolute peak premium deliverables as well. The question is, what is the next technology? Well, I'm curious from your, anyone who's worked in AI or robotics for as many years as you have, by definition the LLM is the new kid on the block. It really is. So is it sort of gonna be one piece of many pieces or is this sort of the end state? No, it'll be one piece of many pieces. As you know, symbolic AI is coming. Is it? Yes, it is, believe it or not. So it'll be one piece of any. I think right now it's very efficient, it's very effective. We have to drive down the energy costs. I think it's ruining our planet a little bit right now. But at some point we will get that, but it won't achieve what we really want. It won't achieve AGI necessarily. It won't achieve XYZ. And there'll be some other new shiny thing. It's like, oh, if we add in LLMs plus X, our generative AI in general, plus something else, it'll make us leap to the next one. And it's accelerating. I don't know, I think betting against an LLM is kind of like betting against Elon Musk. Like you don't really get how it's happening and you don't want to believe it, but it's like, geez, I wouldn't take the other side of that bet. Well, we couldn't go to Mars at one point and we will day one go. In an all full circle for you and your career. So I don't know if we have increased or decreased your trust in technology. Hopefully we have increased your trust in the members of our panel here. Thank you, Mustafa. Thank you, Dr. Howard. And thank you everyone for your questions, both in the room and online. Thank you, Ben. Thanks everyone.