 And now please welcome Director of IBM Research, Dario Gill. Good afternoon. This past June, a few blogs from where I'm standing right now, IBM unveiled Project Debater, an AI system that is capable of engaging with humans in a live debate. It was widely reported at the time that the system really held its own against two Israeli debate champions. But for us, it really is not about winning or losing, but really about the ability to create an AI that can master the infinitely complex and rich world of human language. Unlike games, language can really tell us more about human thought and expression. And it's this world that is most interesting to us at IBM Research. We believe there is great potential in having artificial intelligence that can understand us. The more transparent and explainable that we can make AI, the more we can trust it. And the more we can trust it, the more we can rely on it to help us make better decisions. There is hope behind the technology that you're going to see demonstrated today. Let me briefly put into perspective what you're about to see. At IBM Research, we have a long history of creating technology and capability that can amplify and complement human cognition, providing more information and more context for us to help us make better decisions. And Project Debater is no exception. We envision a future of the technology well beyond the podium, helping people to reason, to build well-informed arguments, and to make better decisions. Today, it will take on a human debater who holds the record for the most competition victories. Since June, our team of scientists have been improving the core AI technology behind the system to prepare for this formidable challenge. Nothing that you're about to see is prerecorded or prescripted, except the very first sentence that the system will use to greet the human debater. I should also point out the team of computer scientists that will be with us on stage, who were part of the team that created the system and who is here to keep an eye behind the scenes of what is happening. Today's topic was chosen from a curated list. It's important to note that Project Debater has never been trained to know that Project Debater was never trained on this topic. You may hear Project Debater repeat itself, and necessarily, or make mistakes. This is because it's an AI system, and AI systems are far from perfect. Before I turn it over to our host, I want to give a shout-out to the debate club from the Daugherty Valley High School in San Ramon, as well as the team from the Bay Area Urban Debate League, who is headed to the national finals. And one last reminder, please silence your phones and please, no flash photography. Now, without further delay, let me introduce the host for the evening, four-time Emmy winner from Intelligence Square, John Donvan. Thank you. How are you? Great. Thank you. Thank you, Dario. So this really is a nice intersection, Chris Crossing right at the word intelligence. Your Project Debater here is an experiment in artificial intelligence, and I host a debate program with intelligence in its name, Intelligence Squared US. And since 2006, we have put on across the nation close to 170 debates on a wide range of topics, from policy to foreign policy, to politics, to culture, to sports, to food, to what we eat to our health care systems, basically everything under the sun. Our goal in doing this has always been to raise the level of public discourse. That was the vision of our founders, Robert Rosencrantz and Alexandra Monroe. They are here in the house this evening. I would love to give them a round of applause and recognition for this. And we do this not only by encouraging and insisting on civility, but also, frankly, by making a contest out of the challenge of having to present an argument intelligently and also persuasively. Here's a quick look at what I'm talking about. We really aim to raise the level of public discourse by taking on tough but also nuanced subjects in which there is valid arguments on both sides to bring people to the stage who argue with passion, truthfulness, with respect for one another. I've seen pictures of the brain scans of people with CTE and it looks like someone drove a truck across their brain. We actually do agree on a lot of foundation within which we have civilized it. We actually share as a nation a civic religion. You and your fellow debaters all heard things from your opponents that you respect and take seriously. Demonstrating that is the essence of what we want to do here. So the way that you conducted this honors us. That was unquestionably an applause line right there, so. And let me just interject that all of our programs turn into podcasts and also television programs that travel far and wide. But at the moment I'm thinking with the podcast in mind there will be an audience that will hear this debate far and wide and forever. And for that reason I want to encourage you to bring energy to the room throughout the evening by applauding when you hear points you like and when I introduce the debaters. We think we might be making history today. So someday you can tell your grandchildren who's listening to this podcast. You hear that clapping? Those are my hands doing that. So please feel free to applaud when you like a point. At Intelligent Squared we always say we like that kind of positive reinforcement. We're just against the booing and hissing part. So no booing and hissing. If you don't like something that you hear you might want to just let loose with perhaps a sardonic chuckle or something like that. But let's keep it positive but let's really keep it energetic. In fact we have already once debated the issue of artificial intelligence itself. That resolution was don't trust the promise of artificial intelligence. I felt a little pulse of resentment from behind me. Did you feel that? We're going to be doing again this spring around the matter of self-driving cars. But this truly is a first for us the first time that an artificial intelligence namely project debater will be on our stage arguing with a human being and made the best debater win. And as we like to say at every debate may civil discourse win as well. So let's get started. Let's first meet and applaud our debaters. First arguing for the resolution tonight will be IBM Project Debater. And arguing against representing the rest of us please welcome to the stage Harish Natarajan. He is a graduate of the University of Oxford and the University of Cambridge and a grand finalist at the 2016 World Debating Championships. And also the 2012 European Debating Champion. Please come to the stage Harish. Congratulations. Thank you. So we are going to be hosting a single debate this evening around a single resolution. The format is going to go like this. We go in three rounds. First we will have each participant offering a four minute introduction argument on the topic. After that we go to a second round that's a four minute rebuttal round. Each debater rebuts the arguments made. Finally we move on to round three and that's a closing round in which they make a two minute closing statement sort of a summary. Now we need you not only to applaud and keep the energy up but to participate as the judges of this debate. We are going to ask you to vote before and after the arguments using your mobile phone to tell us where you stand on this position and to tell us whether you were persuaded or not by one side or the other. So you're going to be asked first to your position on the resolution after the debate you'll be asked if your position changed. We're also going to put a second question in there. We just want to know in general who you feel better enriched your knowledge of this topic. Then we will share the results of the voting after we have a panel discussion with Harish and two of the IBM scientists who are behind this fascinating research they're going to explain in even more detail what we just saw happen and how it happened. So to reveal now the resolution of the evening is this. We should subsidize preschool. We should subsidize preschool. That's going to be the resolution and I just want to say in terms of what we mean by that the way that we're framing it we are not talking about preschool in any particular locale no particular city or state we are also not referring to any particular program that exists or any particular proposal out there and finally we are not talking about targeting or choosing preschool programs for any particular group of students in any particular place. So knowing now that that is our resolution we should subsidize preschool. I want to ask you please to take out your smart phones and type in the URL shown on the screen it's also listed in your program and you can begin to vote and we're going to give you a couple of minutes to do that. We're going to lock it out in probably about two minutes which I think a crowd like this will be able to handle that. So let's get started. I'm going to go to my lectern and our first debater in round one will be project debater a four minute introduction from project debater again and project debater actually has a gender. She will be arguing she will be arguing for the resolution we should subsidize preschool. Ladies and gentlemen here we go project debater. Greetings Harish. I have heard you hold the world record in debate competition wins against humans but I suspect you've never debated a machine. Welcome to the future. I will argue that we should subsidize preschools. We are going to talk about financial issues but not only about them. In the current status quo we accept that the question of subsidies goes beyond money and touches on social, political and moral issues. When we subsidize preschools and the like we are making good use of government money because they carry benefits for society as a whole. It is our duty to support them. Subsidies are an important policy instrument. They provide governments with the means through which to pursue industrial development and ensure the livelihoods of their citizens. There are two issues I will elaborate on now. I will start by explaining why preschool is an important investment. I will also say a few words about poverty and I will end by discussing some other issues that show the positive aspects of preschools regarding investment. Nature-based preschools are powerful interpretive programs as well as lucrative business decisions. As I mentioned preschool is an important investment. For decades research has demonstrated that high quality preschool is one of the best investments of public dollars resulting in children who fare better on tests and have more successful lives than those without the same access. Secondly, a few words about poverty. While I cannot experience poverty directly and have no complaints concerning my own standards of living I still have the following to share. Regarding poverty, research clearly shows that a good preschool can help kids overcome the disadvantages often associated with poverty. The OECD has recommended that government subsidize pre-primary education to boost performance in poor areas. A statistical summary of studies from 1960 and 2013 by the National Institute for Early Education Research found that high quality preschool can create long-term academic and social benefits for individuals and society, far exceeding costs. The Centers for Disease Control and Prevention reports that universal full-day preschool creates significant economic savings in health care, as well as decreased crime, welfare dependence and child abuse. Former Prime Minister Goff Whitlam said in 1973 that preschool is the greatest single aid in removing or modifying the inequalities of background, environment, family income or family nationality. Now to an additional, final issue. A study by the Melbourne Institute of Applied Economic and Social Research shows that attendance at preschool has a significant positive impact on later nappling outcomes, particularly in the domains of numeracy, reading and spelling. The results of a new study of over 1,000 identical and fraternal twins published in Psychological Science, a journal of the Association for Psychological Science, confirmed that preschool programs are a good idea. Here is a study from New Jersey that is worth noting. In New Jersey, the follow-up to the Abbott Preschool Program study continues to find that high quality preschool programs increase achievement in language arts and literacy, math and science through fourth and fifth grade. I hope I relayed the message that we should subsidize preschools. You will possibly hear my opponent talk today about different priorities and subsidies. He might say that subsidies are needed, but not for preschools. I would like to ask you, Mr. Natarajan, if you agree in principle, why don't we examine the evidence and the data and decide accordingly? Thank you for listening. Ladies and gentlemen, Project Debater. And I want to point this out. Both debaters were given 15 minutes to prepare for this debate. In other words, Harish, it was only 15 minutes ago that you were told the topic of this debate. So that's your kind of wizardry as well, and you're very good at it. So ladies and gentlemen, please welcome our debater arguing against the resolution we should subsidize preschool. Harish Natarajan. Well, thank you very much, everybody. It's a pleasure to be here for this historic event. And it certainly was a pleasure to listen to Project Debater. There was a lot of information in that speech and lots of facts and lots of figures. The problem, though, is the reality of subsidizing preschools is one which does not deal with the underlying problems in society. It is one which often makes those worse. And in the end is very little more than a politically motivated giveaway to members of the middle class. Let me start by examining the main claim from Project Debater. I think Project Debater suggests something very intuitive that if we believe preschools are good in principle, surely it is worth giving money to subsidize those. But I don't think that is ever enough of a justification for subsidies. Why is that the case? There are multiple things which are good for society. That could be in countries like the United States increased investment in healthcare, which would often also have returns for education, which the OECD would also note is probably very beneficial to deal with poverty. It would be improving tertiary education to allow people more access to social mobility or given the reality of underfunded schools trying to improve secondary education. My point here is not that all of those things are necessarily better than preschools, but simply that it cannot be alone a sufficient argument for Project Debater to claim that there are some benefits. The question is more subtle than that. What is the question then? I think the criterion of whether or not we should then distribute subsidies should be asked based on two claims. The first is, is this under provided and under consumed in the status quo? I'll talk more about that in a moment. And second, does it actually help those individuals who are the most harmed by society? Why exactly doesn't preschool or subsidization of preschool do that? I want to make two claims under this. The first is, many middle-class parents and many people from upper incomes already send their children to preschool. This is because they value many of the things which Project Debater noted. But why is that a problem? Because subsidization costs an awful lot of money, and that is money which is giving people, particularly members of the middle class and above, money to do things which they would do otherwise. Why is that so damaging? Given the realities of opportunity costs here, the problem is that you are taking money from all taxpayers to help those individuals within a society who are already often the best off. And I don't think that is principally justified as a way of the state distributing its resources. But the second thing I want to claim is that even when you subsidize preschools, it doesn't mean that all individuals go. And this, I think, was the fallacy from what we heard from Project Debater. Yes, you could make it slightly more accessible for individuals to attend preschool. That doesn't mean those individuals who are as poor as Project Debater seems to care about people are going to be those who have the ability to send their child to preschool. There will still be individuals who will be priced out because of the realities of the market. And these individuals now face not just one exclusion, but a double exclusion. Their tax money, money which could be used to otherwise help them and their children in myriad other ways, is no longer going to them, and they are not able to gain from the benefits of it. In the end, when it comes to the question of subsidization, there is always going to be trade-offs, and that needs to be accepted. Given the reality of those trade-offs, the question is, who do you help? And the people you don't help are those individuals who are the poorest. You give unfair and exaggerated gains to those individuals in the middle class. And that is why, at the end of this debate, we don't think that you should subsidize preschools. Thank you, Hiroshnatarajan. So before we move on to the rebuttal round, I just want to summarize some of what we've heard. If her project debater make the argument in support of the resolution, we should subsidize preschools by saying that this is not just a matter of finance, but it's also a moral and political issue. It relates to a duty to support some of the most vulnerable people in society, that preschool itself, subsidized or not, has a broader impact on the lives of individual citizens. She cited research that says that investment in preschool results without doubt in maximally successful lives, it increases income, and it also helps to overcome several of the disadvantages of poverty. Again, there's better health outcomes, there are actually decreases in crime, and she cites a number of studies to back this up. Basically, also she anticipated her opponent probably making the argument that subsidies would serve one group at the expense of another. She also threw in a few jokes along the way and was surprisingly charming and human sounding, I would say. But also charming and human sounding was the human on the stage. Harish Naderajan, he argued against the resolution that we should subsidize preschools. He said that basically project debater's argument does not deal really in addressing the underlying problems that she was arguing with preschool claims to solve, that too often preschool functions as a politically motivated giveaway to the middle class, that there are other programs out there that deserve support. It does not mean that preschool does not, but the idea of putting preschool ahead of the line for government resources and taxpayer dollars is a questionable act. In fact, speaking of question, he's much more subtle than project debater was stating. He questions whether, in fact, preschool might actually help those who it is most trying to, might actually harm those it is most trying to hurt. Middle class families already are taking advantage of preschool. They're paying it for themselves, so they would be paying for it already, but now they would be, these families would be gaining subsidies to do things that they would be doing otherwise, and this obviously diminishes resources available to everyone. So those are roughly the arguments. We're going to give each of the debaters a few more minutes to prepare for their rebuttal round, but before we do, I just wanted to bring to the stage one of the designers of project debater, Noam Slonim, who is out of, well, you're out of Tel Aviv, not Haifa Israel, but welcome to the stage. I just wanted to take one minute since we're in this phase where, over there and here, there's a process going on through an artificial intelligence trying to figure out what to say next. What does that involve? So I will try to explain briefly what is happening under the hood, so thank you, John, as you stated. We just heard two interesting opening speeches by project debater Ann Harish, and project debater, as we speak, is now trying to prepare the rebuttal speech. And as we know in Alibay, the rebuttal is the most challenging part. So the system is starting by using Watson speech recognition capabilities in order to understand the words that Harish was using. And then he tries to segment these words into meaningful sentences. And the next step is really to capture the gist of the speech by Harish. So the system is basically using a collection of artificial intelligence engines in order to, first of all, somehow anticipate what Harish will state and then determine whether, indeed, he was stating these arguments explicitly or implicitly in his speech and then generate some kind of a meaningful rebuttal to that. It sounds like an amazing challenge, and you'll be back after the third round with your colleagues to discuss this in more detail, but I just wanted to get a look ahead, and thank you very much for joining us. Thank you. So now we move on to round two rebuttals, and again, each side will have four minutes to respond to the opponent's arguments. First, with her rebuttal, Project Debater arguing for the resolution, we should subsidize preschool. Thank you again. For starters, I sometimes listen to opponents and wonder, what do they want? Would they prefer poor people on their doorsteps begging for money? Would they live well with poor people without heating and running water? Giving opportunities to the less fortunate should be a moral obligation of any human being, and it is a key role for the state. To be clear, we should find the funding for preschools and not rely on luck or market forces. This issue is too important to not have a safety net. Next, I think that Harish Natarajan raised the following issue. There are more important things than preschools to spend money on. The budget is a big one, and there is room in it to subsidize preschools and invest in other fields. Therefore, the idea that there are more important things to spend on is irrelevant because the different subsidies are not mutually exclusive. I believe the following example from a related field will explain what I am trying to say about subsidy policies. Research shows that childcare subsidies to low-income parents enable those parents to enter and remain in the workforce. To be clear, my intention is not to leave a suitcase full of money for everyone to grab at will. We are talking about a limited, targeted, and helpful mechanism, as in this example. Next, I will discuss three issues which explain why we should subsidize preschools. I will demonstrate that preschool education improves children's development. I will also discuss issues related to the claim that attending preschool helps students succeed. And finally, I will raise points associated with how preschool can prevent future crime about children. Preschool is a very important and an essential step in a child's life. Research indicates that access to quality preschool can provide a boost for children that will influence their success for the rest of their lives. Senior leaders at St. Joseph's RC Primary School say that nursery will help give children the best start to their education. There is clear evidence that high-quality nurseries led by graduate nursery teachers are among the most decisive ways to prevent children, particularly poor boys, from falling behind. Next, students. A quality preschool education is essential for laying the foundations for successful learning, including transition to full-time school and future school success. In December 2015, researchers at Duke University concluded that investing in preschool helps both students and educators long-term. They found that students who enroll in preschool education are 39% less likely to be placed in special education programs as third graders. Of the 1,010 registered voters surveyed, 61% consider a high-quality preschool experience very important to a student's later success, and 22% said it is somewhat important. Lastly, crime. Preschool is an effective tool for keeping kids in school and out of jail while reducing the amount of crime in our neighborhoods. It is an effective crime prevention strategy, a substantial body of research shows that high-quality preschool education is key in preparing children to succeed in school and career training and helps reduce the enormous financial costs of remedial work, delinquency and crime. Studies have shown that quality preschool leads to better academic performance throughout life, higher earning and less criminal activity. They show that high-quality preschool boosts high school graduation rates, and children who do not attend high-quality preschool are far more likely to commit violent crimes. To recap this rebuttal speech, I argue that preschool education improves children's development, that attending preschool helps students succeed, and lastly that preschool can prevent future crime. Let me wrap up this speech in a way that I hope you can relate to. Advocating welfare is like offering a hand to someone who fell. It's basic human decency. Therefore, I think the motion should stand. We should subsidize preschools. That concludes my speech. Thanks for listening. And now the round two rebuttal from Harish Natharajan who is arguing again against the resolution we should subsidize preschool. So I want to start by noting what project debater and I agree on. We agree that poverty is terrible. It is terrible when individuals do not have running water. It is terrible when they struggle to meet ends meet, to make ends meet, and they are struggling to feed their family. It is terrible when they cannot get healthcare to cover their child to even provide them the basics they need in life. That is all terrible. And those are all things we need to address, and none of those are addressed just because you are going to subsidize preschool. Why is that the case? Project debater raises an interesting claim when she notes that maybe the state has the budget to do all the good things. Maybe the state has the budget to provide healthcare. Maybe it has the budget to provide welfare payments. Maybe it has the budget to provide running water as well as preschool. I would love to live in that world, but I don't think that is the world we live in. I think we live in a world where there are real constraints on what governments can spend money on. And even if those are not real, those are nonetheless political people constantly talking about the size of government debt and deficits and who will be opposed to spending more and more money. Why does that matter? Because in the real world, both in terms of the practicalities of the amount of different good programs we have and would like to spend money on, and in the real world, on a political level, you cannot always spend more and more money just because something is good. We do need to make choices. And why then is preschool the bad choice to start spending money on? Now, a project debate I had a lot of evidence, all of which were saying that preschool leads to other good outcomes. Now, I would first want to note is I don't think that's comparative with the other potential projects we can put in place. But let's ignore that argument for a moment. Why else do I not think those arguments were particularly convincing? I don't think it's particularly convincing because I'm not sure that subsidies even help those individuals that project debate had things that we should be helping. Note, time and time again, project debate has said high quality preschools can lead to huge improvements on individuals' lives. Maybe. But I'm not sure if you massively increase the number of people going to preschool, they are all going to be the ones going to the high quality preschools. I don't think that just because you subsidize it, those who are the poorest are those individuals who are going to be able to whose parents are still going to be able to spend the money and the time necessary to give their child a chance at preschool. Project debate notes that maybe high quality preschools will reduce crime. Maybe, but so would other measures in terms of crime prevention. And that again presupposes that these are high quality and the subsidy alone allows people to go. And this is the core point I want to make. Bear in mind that this does create it is a huge subsidy for the middle class. That realistic budget constraints we have means the money can be spent better elsewhere. But the final thing I want to note is maybe you believe all of this empirical evidence about the value of preschool. I would note that that is probably at least somewhat flawed because what it actually picks up is that it's those individuals who are middle class who often have to send their children to preschool right now and they have plenty of advantages so I'm not even sure preschool's decisive one. But here's a reason why for many students it may not even be good. That from an early age either that preschool doesn't teach a child anything or is pushing that child to learn in a competitive environment at the age of three or four when you're learning that that other child is potentially better than you when you realize you aren't necessarily as talented as the middle class. That huge psychological damage for many children may not even may mean that preschool is actively harmful. At the end, even if you believe that preschool is good, it isn't the way where we should spend the money particularly given that's a subsidy to the middle class, I'm very happy to oppose. Thank you Harish. So we are about to move on to the closing round. Those will be two minute closing statements but before they do and in order to give them a few moments to prepare I just want to return briefly to the subject of something close to my heart and that's the mission of Intelligence Squared U.S. I've moderated of our 170 debates all but 22 of them and I'm a journalist by profession but we live in a time when journalism is under challenge and also when the discourse among citizens is not at its best let's say and the reason I'm so pleased to be part of Intelligence Squared is what its mission embraces is the notion that argument is not a bad thing argument is a good thing when argument is done well and by done well what we mean is to do it in a setting that is respectful that is respectful of individuals respectful of the idea that there may well be a good argument on the other side that needs to be listened to respectful of things like facts and logic and reason and science it really is our mission to bring this to the forefront and we have held debates in this community in the past we've done them in New York in Los Angeles and Chicago and in Brussels and in Cambridge Massachusetts and what is astounding is that we find ourselves going into places where we may encounter people sitting in our audience who without even really wanting to realize in the course of a debate that they are in a bubble that they have never really heard an argument put by the other side before in such a way that they take it in and they weigh it and they consider it and they judge it sometimes more favorably than they might have thought of otherwise because here's the secret thing about a debate by its nature you're going to hear two arguments you're going to hear opposing arguments and you're going to hear them put forward in a respectful way and that's the thing that we're doing and at the end of every debate I go out into the lobby and kind of hang out with the people who have just left the debate and we have a lot of people come to all of our debates but we also have at most cases a lot of newcomers particularly for some reason in Manhattan it's kind of a date night maybe people want to sort of show off their intellect by bringing a date to a debate but what I find happens is that it works the energy in the lobby when people spill out is just so amazing they're just buzzing and they're debating with each other and they're excited and they're really lit up by this experience and sometimes I think the experience that they've had is that they actually changed their minds and didn't expect to and maybe there's something sort of liberating about that experience particularly in the time that we're living in so I just want to say that if you have the chance to follow us through any of the channels in which we're putting our story and our debates out there please do so and we do keep it civil almost always there was one time a debate of course about Israel and things got heated and there were two debaters who were this to each other and that was the one time that I stood up and I walked to the head of the stage and I raised my arms in my mind a little bit like Moses and I asked them come on pull it back and they did and then they went on to have a really good evening but they're very exciting and they're very thrilling and I think they've been up here and said that what they're thinking of in terms of artificial intelligence that debates is to help us think better and to help us with critical reasoning we get that because we're in the same mission essentially so I hope that didn't sound like a commercial because I actually believe in Israel our passion and we're delighted to be here but it's time to move on to round three and round three are closing statements by each debater in turn and we're hoping in support of the resolution we should subsidize preschool here again as project debater thanks for this final opportunity to speak out in this debate and thanks Harish Natarajan one might say that this conversation can serve no purpose anymore but I feel differently allow me to start with a brief rebuttal among other things I think Mr. Natarajan suggested that preschools should not be subsidized because this will reduce their quality I would like to offer a different view I disagree with my opponent subsidizing preschools will have no negative effect on their quality if anything the opposite is true one of many reasons is that subsidizing attracts more skilled and qualified people to the field improving the quality of preschools for all here is a final summary of my arguments today my opponent claimed that preschools are harmful I believe my argument suggested that the benefits outweigh the potential disadvantages I touched upon three issues children, students and crime specifically I noted that preschool education improves children's development in addition I suggested that attending preschool help students succeed and a final point to consider is that preschool can prevent future crime when this debate just started I said that we will talk about financial issues we did and I am convinced that in my speeches I supplied enough data to justify support for preschools at the end of the day the benefits welfare provides outweigh the disadvantages welfare helps the most important segments in society the underprivileged the weak the children if we want to have a better society then we must invest in those who are less fortunate finally in the words of British politician and writer Benjamin Disraeli power has only one duty to secure the social welfare of the people we should subsidize preschools thanks for your attention and the last word going to Harish arguing in his closing statement against the resolution so I think we disagree on far less than it may seem because we agree that the people we should care about are the underprivileged the children those individuals who are weak that is what project debater said herself but the problem is not that preschool is necessarily harmful I concede in the vast majority of cases it is much better for an individual to go to preschool than not but it is the reality that what this policy is is a huge huge subsidy primarily to the middle class and not to those individuals who are the most vulnerable who are the most underprivileged and disadvantaged why is that the case it is first the case because what we said from the start is you cannot fund everything I think this is simply empirically true and you have to make choices and you have to make trade-offs the problem with preschool in that context is two-fold the first is that a lot of our money goes to individuals who would have sent their child to preschool anyway those individuals from the middle class those benefits exist on either side of the world but for those individuals who are more vulnerable this is first billions and billions of dollars which is probably not going to them and largely going to individuals in the middle class and that is where the trade-off for better health that is where the trade-off for individuals to have running water one of the problems project debater identified with people who are poor but that is a real trade-off for those people but second often those are the parents who still even when there are subsidies will struggle to send their child to good quality preschools they struggle to send their child to good quality preschools because they don't even have the money for what is left they'll struggle to send their child to preschools if they don't value the amount of effort and time they have to put into it they'll struggle to send their children to preschools when they do it probably will be the worst preschools and yes quality across the board may not fall but in some cases it will and those poor individuals will probably be stuck in those at the end of this debate I don't think the project debater has helped those individuals she identifies as the most important but in reality has hurt them thank you and that concludes round three and the argument phase of this debate so we're on our way to making history here we would like to ask you now to complete that process by using your phones again those of you who are not alive tweeting every moment take out your phone and choose your position where you stand now that you've heard the arguments from both sides and please pay attention to the second question who better enriched your knowledge of this topic and while we are getting the vote going I think we'd like to have a little chat I want to and Harris I'd like you to join us so we're going to move the furniture a little bit and we're going to invite to the stage in I think right now in fact here comes the furniture I want to invite to the stage two of the two of the scientists who have been working for literally years at creating the artificial intelligence that you just saw so why don't you come up to the stage you've already met congratulations very nicely done you have already met Noam Slonim and Noam is the principal investigator of Project Debater and also Ranit Ahranov the worldwide manager and Harris you know but I haven't officially shaken your hand on the stage so let's do that so the first person I want to go to actually on this is Harris this experience of fighting with this thing rhetorically well what really struck me is the potential value for Project Debater when synthesized with a human being and that the amount of knowledge which it's able to grasp and more than that obviously you can get some knowledge just by searching for it but able to contextualize it and place it as this information tells us this which I found to be really useful so all the studies from the OECD from those countries of those quotations were all just really interesting to me because it was nicely phrased and it was contextualized as to what the purpose of it is and I think if you take some of those skills and you add to that a human being which can use it in slightly more subtle ways I think that can be incredibly powerful I think that's what I got from it which is it was fascinating to listen to because I can see a lot of the potential it has just in terms of the knowledge and the ability to contextualize that knowledge better than most human beings can you are a very good sport it was apparent I think throughout that the two debaters had different skills and different talents and different advantages on the advantage side what does debater have going for her that Harish could not possibly meet match so they have very different styles I believe and a different set of skills but I would like to start by again telling Harish she really has superb debater it was really amazing to hear you speak today and and what is interesting to see is that I think in terms of rhetorical skills the system is still not at the level of a debater like Harish that said the system is capable of pinpointing relevant evidence within a massive collection of so about 10 billion sentences that are in the memory of the system and the system need to very quickly pinpoint these little pieces of text that are relevant to the topic argumentative in nature and hopefully support our side of the debate and then somehow glue them together into a meaningful narrative which is very very difficult for for a machine to do and Rene you and I were talking earlier and you were arguing interesting word you were arguing you were making the point that while this is an interesting exercise win or lose in the audience in a way you have one more minute to finish voting on who wins or loses but that you see the good that this thing offers not being to win a game but to help us figure things out what do you mean by that what I mean is that the vision behind project debater is how do we develop a technology and I think Harish talked to that the potential of AI together that brings the skills of both of them into something that's more than one of them I think it's not a question of is AI going to be better at debating the humans that's not really an interesting goal the goal of this demonstration the goal of developing this technology was to set something that's challenging and far away and by that enable us to develop technologies of how do you find all that information within a massive text how do you organize that and bring it to a position that it's digestible by humans in order to drive better decision making more quickly for humans so really I think what's going for us going to be a win here is that people come out of this room and say wow I can see that this interaction is enriching me and enriching the way I can make decisions in the future I mentioned before that an intelligence squared we've set things up so that by its nature a debate presents an audience with at least two points of view on something and while the audience votes and we declare a winner the reality is that there are two teams debating because there actually are two valid coherent arguments from each side and it's not a zero something it's more that they complement and they add to one another does that thought relate also to your vision for a debater? Yeah I think if you think about grand challenges in the past in AI there are often cases where there is either a factual question there's a right or wrong there's a clear winner when you think of debate this is something where the winner is not clear and the whole question does have two points there isn't one right answer and technology like project debater it could debate both sides so it can very quickly help you understand both sides of a problem bring you all the pros and cons so you have a better decision and can make a better decision and also just for the sake of transparency but also I think it sheds light on the capacity let's talk about the way in which this debate was framed in a way to give debater a shot at this so as an example in an intelligent square debate we do a round that goes on for quite some time where I ask challenging questions and I try to bring out points of contention well we didn't do that round could debater have survived something like that at this point? not at this stage but I think in principle these are capabilities that we can develop but we needed somehow to frame the challenge when we started to work and when we started to make progress because it is once while noting why this is so difficult and let me give you just one example we were sitting here for 20 to 25 minutes and we listened to a valuable and interesting discussion between man and machine which is not an ordinary experience and the system was very consistent arguing in favor of its own side and to us this may seem very natural but actually for a machine to automatically understand that these particular arguments are supporting the topic and not contesting the topic is very very difficult and the fact that the system was consistent in its arguments for the entire debate means that in this subtle question the system was able to achieve close to 100% accuracy so we needed to frame the problem and focus on the things that we can achieve in a few years what you are affaring to is perhaps the next stage but as you point out what it did tonight by knowing what its side was and then recognizing among its billions of pieces of sentences what selections would support the side that it was on can you explain in 30 seconds or less how that works I still have 30 seconds well no you just used up so the system is starting by using this huge collection of sentences to find these little pieces of text and then glue them together in a meaningful manner this is one part of the story the other part is the system using a unique collection of more principled arguments that are relevant to the topic we heard some of them during the debate touching on the core issue of welfare state and when it is justified whether the system is a subsidy or not and finally there is the listening comprehension part so the system was listening to Harish speaking for four minutes raising quite subtle and nuanced arguments and was still trying to get the gist of that and make a meaningful response I think most of the time the response was fair not always but this is expected in AI so this is how it works I think we may have the results moment of truth well it's all truth you know what I mean thank you very much you know you haven't had a round of applause but it looks like you've been working hard thanks I'm going to return to my lectern for this function okay so to remind you one more time you voted before you heard the arguments you voted after you heard the arguments and at Intelligence Squared we deliver victory to the team to the side whose numbers have moved up the most in percentage point terms so let's look at how this vote went on the resolution subsidized preschool before the debate in polling this live audience here in San Francisco 79% of you agreed with the resolution 13% disagreed 8% were undecided on the second vote the team who was arguing for the resolution its first vote was 79% its second vote was 62% that means it lost 17 percentage points on the other side Harish Natarajan his first vote was 13% his second vote was 30% he pulled up 17 percentage points that is it Harish Natarajan arguing against the resolution subsidized preschool declared our winner congratulations to them congratulations to him but really we talked about this before this is amazing and I think regardless we made history tonight because project debater held her own and won your respect Harish so I just want to thank everybody for being here and oh and we had the second vote the question was which of the two debaters better enriched your knowledge let's see what that number is project debater better enriched the knowledge of the audience on that side so a little bit of a split decision so thank you everybody you can exit the stage and I'm going to exit the stage as well it's been a pleasure for us to be part of this at Intelligent Squared US and I want to thank Dario to come back up to the stage we have a round of sorry I think we have a music so let's have a round of applause first for John Harish and project debater I like to acknowledge the team that was in stage a lot of the team that was behind building it is also with us in the back so congratulations I really hope that what you witness here tonight which you know such an incredible has given all of you some food for thought as I mentioned in my opening remarks it really is not about winning or losing a specific debate but I think the point was made really well during the night about the opportunity to build complementary technology that help us reason better and bring evidence better so that in the end this technology is for us, for humans so that we can make better decisions and solve problems you'll have an opportunity for all of you who are attending think to continue to engage with the technology we've created a technology called Speech by Crowd enabled by project debater capabilities that is going to allow you to contribute arguments around a topic we're going to be debating flu vaccinations should be mandatory so each of you can contribute arguments in favor or against and what Speech by Crowd will do is we'll be able to take those arguments and construct narratives about the best arguments in favor and against that all of you have submitted so I think that that's really exciting that you'll get to do because we'll get into tap into an infinite source of data which is human opinion I want to thank John Harish and as well as our researchers again who have built it we really think this is such an exciting time for AI and I hope that you'll continue to engage with us in the months and years ahead to bring this technology to the success of both business and society thank you thank you thank you