 Okay, so this is the Integratives Seminar Research Series, organized by the Department of Information and Computational Technologies. And my name is Hector Geffner, as Albert said, so I'm one of the organizers, and today is my pleasure to introduce myself. Okay, so to make the story short about the introduction, I arrived to Barcelona, to Icrian, to the Pompefauda at the end of 2001. Okay, that makes me one of the first generation actually, Icrian, that's the first call, and probably one of the oldest guys in the department as well. Before coming to Barcelona, I was a professor in Caracas, in the Simón Bolívar University, and before that I was a researcher in the IBM TJ Watson Research Center in New York, and before that I did a PhD at UCLA under the supervision of Julia Pearl, that is very well known as the father of Bayesian networks. Okay, so I'll try to give sort of a relatively general talk, okay, that will touch some of the things that we do in the group. So I'll start first with the group. Okay, so I'm a proud Karkarin member of this group, this AI group, as you see we have many people, okay, so many good people, okay, we don't have that many women, okay, so maybe I can quickly introduce again people here that you often see and we've not done the name, so Karkarin's collaborator, okay, he's an Icrian professor, then we have Jonathan, we have Filippos, we have Damir, Osam, we have Guillem, there are students, there are some missing students as well, then Gergo Vicenz and Dimitri are UPF fellows, Victor is a professor and there's a professor, Gabor is a professor but not in our department, okay, he has chosen to endanger his soul working in the Department of Economics, okay, and nobody's perfect and Sergio is Juan de la Sierra fellow, okay, some of you may have been at the talk I gave a few months ago in the CCB, so part of the talk will be common, part will be new, I'll try to make some new jokes though, okay, you can count that one, okay, so one of the things about AI is that AI is very much in the news, okay, in the last few years, and one of the things that brings high to the news are some developments, some applications and so on and also some fears, okay, for instance, this is an article in The Guardian, okay, Elon Musk, okay, a warning that artificial intelligence is our biggest existential threat, so we have a few others, okay, like Bill Gates in the Washington Post, okay, I don't understand why some people are not concerned, okay, and even a physicist, okay, Stephen Hawking saying that artificial intelligence could wipe out humanity when it gets too clever as humans will be like ants, okay, so many fears, so the question is, is this fear justified, has AI been solved or something close to that, and in order to alleviate this fear, okay, so I'll show you a couple of videos on there and then we can relax and enjoy the talk, so this one, if it works, okay, sorry, it doesn't, so how does this work, okay, okay, so this one is from the DARPA Challenge of 2015, this means this is last year, okay, so the task is for the robot to get inside the house, okay, and you see that it's not, probably it's so simple, it's thinking about it, finally found the handle, okay, and this reflects a little bit, okay, some of the challenges, okay, current challenges in AI and robotics, okay, you may think that this is an accident, okay, and so on, but actually it was not the only accident in, okay, here a few more, okay, very quickly, so we get to see the same one, okay, but actually the task is not so easy, okay, so one of the nice things about artificial intelligence is that you start to value even the most stupid things, okay, so in a sense the computational perspective is very generous on the human condition, even the most stupid things are very challenging, okay, for artificial intelligence, as opposed to some of the things that in principle do not require, appear to require some intelligence, so basically the plan for the talk is, so I'll talk some general themes like, so what is AI, a little bit of the history, so why it's now in the media, and again, so a lot of this is because of some, the success of some machine learning techniques, so I'll talk a little bit about that, and then I'll focus on some of the things that we do in the group, I will not be talking about everything that is done in the group, there are many things that many research lines see in the AI groups, some in which I'm not involved at all, actually and there isn't a strong machine learning process as well, so I'll talk just about the planning part that is related to this idea of general intelligence, so if we have time then I'll talk some philosophy and uses and risk, okay, because there are certain, even the robots may be not taken over the world yet, okay, there are actually some dangers, okay, I have to do for instance about losing jobs and things like that, so the joke that I want to make about this video is actually two jokes, okay, because this is one about opening the door, another sort of challenge in 2015, first time is Amazon, okay, started the so-called Amazon Picking Challenge and the Amazon Picking Challenge is to have robots that can pick things from shelves. As you know, Amazon uses robots, but uses robots to move shelves, okay, why to move shelves to operators because it cannot retrieve the items, so rather than retrieving the item, it retrieves the shelf that contains the item and brings the whole shelf to the operator that is sitting somewhere, okay, so to take home lessons for this first part, if the robots actually take over the world, so for the time being you can be safe if you close the door, okay, that's first thing and noticing what's happening with the Amazon Picking Challenge if the robots actually get into the house and they ask you for the keys, all you have to do is they're on the shelf, okay, and you will be pretty safe as well, so let's get going, so artificial intelligence is a branch of computer science, okay, and studies intelligent behavior from a very particular perspective, okay, so the computational point of view, it uses a very peculiar type of telescope, okay, understanding a behavior from the point of view of artificial intelligence is to be able to generate the behavior by the computer, sometimes in the computer you need also some sort of body, like sensors and actuators, okay, like if you want a robot to do some cooking, not in the market yet, okay, so these activity acts maybe solving the puzzle, recognizing an object in the image, okay, understanding the story, a joke, okay, doing the dishes, okay, many, many things, okay, so understanding means I can get a computer to do it and this is very challenging, not only on things that in principle require intelligence, okay, okay, so the branch actually will say that the father of artificial intelligence is Alan Turing, okay, is one of the greatest scientists, okay, in the 20th century and definitely one of the heroes in computer science and artificial intelligence, okay, Turing was the first guy who developed a universal programmable computer, okay, so anything that a computer now can do, okay, was possible to be done in the machine, okay, that Turing envision, okay, this Turing machine, okay, just in paper, an infinite tape, okay, a head that can move to the left and right depending on what's on the tape and a controller state, okay, the machine that can be in several controller states, that's a Turing machine, anything that can be computed, okay, but a computer now could be computed by pencil and paper, okay, with the machine that Turing devised in 1936, actually the current machines will go faster, but still all the analysis of complexity about different type of problems are still, okay, use the Turing machine, so we say that problems can be hard and be complete, okay, we'll come to that later, it's basically analyzing Turing machines, not actually the current computers. Another thing that is worth emphasizing, okay, about this work by Turing is that often you hear about applied research, basic research and so on and nobody will question the fact that computers are useful, okay, okay, so you carry them in your pockets, okay, to talk to people, you can buy tickets to the concert or whatever, okay, but the goal of Turing was not practical at all, okay, so these problems could not be farther from the mind of Turing at that time, Turing devised this machine, okay, that is the blueprint, okay, from the computers that we have today by trying to tackle a very abstract mathematical problem that was posed at the beginning of the 20th century by a mathematician, okay, David Hilver. Hilver wanted to know at the beginning of the 20th century whether it was possible to mechanize proofs in predicate logic, okay, what we call first order logic or predicate logic and at that time there were no computers, there were no way to mechanize, okay, the process, so Turing basically came up with this idea of these universal computers, okay, just to have a theory of what it means to mechanize process to solve this problem and actually the way that they solved this sort of conjecture or this problem posed by Hilver is by showing that that's impossible, okay, Turing showed that proofing theories in predicate logic is an undecidable problem, there is no Turing machine, okay, that can solve the problem in finite time, meaning there is no computer now or in 100 years that will solve this problem in finite time either. This is an undecidable problem. We'll see that also for the frontier between problems that are polynomial and non-polynomial, we still use the idea of the Turing machine as well. Turing is also the father of artificial intelligence that published in 1950, okay, four years before he committed suicide, one of the first AI papers, that's where the idea of the Turing test came and the name of the movie, okay, the name of the movie, okay, that some of you may have seen recently, the imitation game comes actually from the Turing test and has little to do with what it goes on in the movie. It is about the cryptin, okay, it goes from the Nazis at that time, okay, this was in the early 50s, okay, by the end of the 50s is that the discipline of artificial intelligence, okay, takes some shape, okay, some of the founders, okay, are here, some are well-known like Klochanon, okay, McCarthy-Minsky, okay, that passed away not long ago and basically the beginnings of the slogan in it was something, if somebody will claim that machines cannot do X, so the task is to disprove the skeptics by writing a program that does X, okay, so that was in some sense the idea from a methodological point that was not too good, but in some sense the computer program was a new toy and it was a natural thing to do at that time. Actually, my first contact with AI came with this book, not in 63, okay, I don't know what else, okay, I've been born by then actually, okay, but in the early 80s when I was doing my bachelor's and found this book in a bookshelf, okay, with many papers about programs that play chess, checkers, performance in logic and geometry and things like that, okay, so if you look at the first decades, okay, so up to the 80s and even up to the 90s, okay, first you see that AI sort of split into slightly different areas that once in a while they converge and diverge again, okay, like planning and reasoning, learning, language processing, vision, robotics, okay, up to the 80s I would say there was a lot of progress in programs that have very specialized abilities, the so-called expert systems. They can make diagnosis of infectious diseases or things like that, so you could write the program by interviewing experts and actually these programs did pretty well in comparison with the best experts, okay, in sort of finding what is the hidden disease, okay, that express the patterns of symptoms. However, those programs are trying to do, let's say, story understanding or trying to understand a joke and things basically were not robust at all, many dissertations were written at that time, but mainly the programs were used as demos, okay, so if you run a few examples that the author had so you get some interesting results, but if you had a new example, okay, a new scenario, okay, basically this program just broke down, okay, by even the 80s and 90s and even today, so there are no programs with the normal abilities of a five-year-old child in the motor, visual, or linguistic, even in some of these areas this is beginning to change, okay, and this also is in relation with why we hear so much about AI in the news, okay, some applications, some interest of the largest corporations, okay, Apple, Google, Facebook, IBM, and so on, so we listen about cars that can drive by themselves so interacting with mobile phone, okay, by voice, okay, even there is a very nice movie about that, about a guy that gets in love with the operating system, okay, of his phone, I will say that I mount all the AI movies, I think, for me at least that's probably one of the best, okay, and actually to say something true that, okay, for one reason or the other, people are very happy about interacting with machines and less happy about interacting with people, okay, so there is a problem there, okay, but it's a symptom, okay, also face recognition, okay, a number of other things, maybe in the last couple of years, a couple of breakthroughs, okay, I came from this company, The Mind that was acquired by Google is a program that learns to play Atari video games just by looking basically at the pixel information in the screen and just the score, and actually more recent in the last few months, okay, a program that finally achieved top world level in the game of Go, okay, this is just in the last few months, okay, for those who don't remember, so a program that beat Kasparov, okay, so the world champion was in 97, so it's basically 20 years since that level more or less were achieving the game of chess and in the game of Go. Something in common about these different applications is that one way or the other, they tend to use some machine learning techniques. So let me go relatively quickly of what this machine, basic machine learning techniques are. Let's say that we want a program to recognize digits, okay, from zero to nine, okay, so people write these digits in different way and you want a program that will take an image, okay, of one of these digits and will output, okay, one of the ten symbols, okay, from zero to nine. So you are in some sense classifying images, okay, into ten classes, okay, corresponding to each one of the digits. Okay, it turns out if you want this function and you want to write a program that hands, okay, that program is going to be very brittle and not robust, okay, it's going to work from some drawings, it's not going to work for many others. So the best technique for this and for many other problems, okay, that was sort of found by people in the eye is that you don't want to write the program for this by hand, okay, the best approach, okay, currently for this is to learn this function from label examples. What it means, label examples, is I give you many of these pairs. So I give you an image and I tell you, okay, this image contains actually a six. I give you this image and I say this actually contains a two. I give you this image and see this image contains actually a seven and so on. So what you get is a set of label examples of the inputs and the intended output and the idea is that you're supposed to learn this function F, okay, from this example. So the form of this function is a parametric function. So all what you are learning is not so much the form of the function but the parameters of this function often by causing the problem as a problem of minimizing error and then applying some algorithm like gradient descent, okay, in too many biases that are squares and so on. Actually one common function that is used is the so-called neural networks, okay, where in some sense the parameters are inspired or the function are inspired by the working of the real neural nets even though this is just a metaphor. Actually for those of you that this seems to be strange so we are all familiar with this idea one way or the other so the most common way in which this idea appears is in regression. You have a plane with a bunch of points, okay, you see that in the newspaper all the time and you're supposed to pass a line, okay, just to see some trend in some data. So the question is how you draw the line. So in some sense what you're learning is a linear function that minimizes the square root with respect to all the points, okay, that you have in that plane. So in that case you are learning the parameters of a linear function. In most cases in machine learning, okay, you deal with nonlinear functions that don't have the same expressive limitation and in many other cases, okay, you either mix this idea of supervised learning where you're given inputs and outputs with ideas from unsupervised learning where you learn, okay, without an output or you learn from reinforcements and from penalties as we'll see in a second. Actually when you read things like this, okay, so that the Google brain, okay, can find cats in videos and things like that. So in some sense, okay, are these sort of techniques normally the ones that account for these results. Normally the type of neural networks or the neural network techniques is called in this case as deep learning. Normally phrase deep learning refers to relatively all algorithms, okay, the algorithms that people have developed in the 80s, okay, like back propagation that has ways to tune these weights, okay, and supervised learning, okay, but with more data, okay, with more layers of hidden weights and of course with more CPU power, okay. In some other cases, okay, for instance, this is also from a couple of years ago, okay, so this is about the deep mind that learns how to play Atari games. I will talk a little bit more about these Atari games later on because it's something we have done work on and in this case, so nobody is teaching, okay, how to move, let's say, the paddle in this case so the way that this program learns is not by supervised learning. All the information that it has is just the pixel structure and the score so it's a form of unsupervised learning where it's learning indirectly, okay, from penalties and rewards, okay. So now, so machine learning methods, okay, they are these days all over, okay, so if you really want to go to the industry and get a job, okay, it's not a bad idea, okay, to get into that, okay, and to put words like deep learning and big data and things like that, but of course all these machine learning techniques also have some limitations, okay. From the point of view of general intelligence, that is in some sense one of the originals and still sort of one of the current goals in artificial intelligence. So intelligence normally is associated with the ability to read some plans or problems, thinking and so on. So even though the machine learning techniques, okay, so have produced impressive applications so the idea that all there is, okay, for knowledge is to associate responses with inputs or stimuli, okay, it looks like insufficient in a sense, okay, this looks like a psychology up to the 50s, what it was called behaviorism, okay, so in principle actually most of the human learning and probably also in many animals as well, it's about how the physical and social worlds work and to use this knowledge for various purposes. Purpose like for instance generating behavior, okay, let's say behavior for achieving goals, so you use a model, a predictive model of the physical world in order to get the behavior without requiring, okay, so much training, so many examples, and to have at the same time much more flexibility, okay, if you change the game, if something happens, okay, you still can reason about what's going on and to have to do it many, many times in order to decide what to do. Other things that these models of the physical and social worlds can be used is also for interpreting the behavior of others, for sensing the world, for communicating with other things like that. So many of these tasks are actually not easy at all to capture, okay, by just stimulus responses, okay, and that in some sense make you sort of systems that may be flexible, so that inputs, okay, but have sort of more limited functionality. If the goal changes of the problem, okay, all the experience that you have, all these reactions that you learn in principle, okay, cannot be used. So you have to learn it from scratch again. So just to illustrate a little bit, okay, some of the things that we do in the AI group related to planning, so I'd like to illustrate what planning is about from the point of view of autonomous behavior, okay, so basically if you want to program, okay, do you want to program an agent, a robot, okay, to be autonomous in a certain sense, so you don't have to tell a direct, okay, the agent, so you can do it basically in three ways. Either you can program the behavior by hand, okay, this corresponds to hardware and the behavior, it's thought that nature does that, okay, for many elements and many circumstances and even for us in many cases as well. Second approach is, okay, you learn, okay, so it's very flexible, okay, you learn from the experience, like experience and you see how to behave in terms of the results that you got. Now there is a third approach, okay, that actually is relatively well known, hopefully this situation will change in the near future and this is what we are struggling with, is that you don't specify the behavior by hand, okay, you don't have a worry, not you have to learn it, but you derive it effectively, automatically, okay, from a model of the world. So basically I have a model that allows me to make predictions, how can I use this model that allow me to make predictions in order to derive the control, to derive what to do, okay, so let me just go very quickly over an example, okay, so of course those that take my classes, I've seen this example many times, those familiar with the AI basic text books, okay, I'm taking this example from a textbook by Storvacel and Peter Norvig, so here is the problem that you have, there is an agent, this guy here, this guy wants to get to the gold, okay, but in order to get to the gold, he has to avoid meeting this dangerous monster, okay, called the Wumpus, and he has to avoid falling in one of these pits, okay, the problem will be trivial, okay, but the issue is the guy does not know where the gold is, does not know where the Wumpus is, does not know anything, okay, but it knows how the world works, so it knows that if it's here, it moves to the right, it's gonna be in this cell, and it also knows it has some sensors, so even though it does not know what all these things are, so there are some sensors that give partial information about the true state of the system, so if you're a distance one from the Wumpus, okay, you're gonna sense some bad smell, okay, so let's say you move here and you sense a bad smell, so conclusion is that there must be a Wumpus here or here, not here, because you were here and there were no Wumpus here. Same thing with the pits, all these pits, if you are a distance one, okay, you can detect some smell, okay, that sense, okay, there is a pit at distance one, so in this case the bridge, so I move here and the conclusion is there has to be a pit either here or here, okay, so now, so you can take this problem and approach this problem in these three different ways, so one homework that you can give to students and give me a controller for this guy in this particular work, okay, just program the controller by hand. Second approach is use some sort of learning algorithm, okay, that every time that a guy dies in a pit or with a Wumpus and so on, without achieving the goal of getting to the goal, it gets penalized, okay, and the idea is how to optimize the work. Third approach is given that I know how the sensors work, okay, how the actions work in the world, okay, figure it out, okay, given the goal that you want to get to the goal, what to do after the solution to this problem, so you start here, let's say you move right, you see, okay, you sense something, so this is not safe, this is not safe, so the best thing that you can do is to go back, you go back, okay, you move here, again you see a signal, so Wumpus in principle could be here or here, okay, but you were here before, so you didn't sense any smell, so this cell contains no Wumpus, no pit, so you can move safely here, but you have to integrate the information from the sensors, okay, in order to get finally to the goal, so it's a very simple problem you can tackle in different ways, actually they're not very good machine learning algorithms, even to tackle something as simple as this one, okay, and of course it's not too interesting from the point of view of AI to have a controller written by hand, because you have a controller written by hand for this particular problem that is not going to work for another problem, what I want to have is something that is generic, okay, that can handle variables, can actions that change the values of variables, sensors and so on, and I plug this problem in the input and I get the controller in the output, no matter what the problem is about, so to see some of the computational challenges in this, okay, so I'll focus only on the simplest type of planning setting, in the simplest form of planning setting what you have is a set of variables, okay, with a set of possible values, you have a set of actions, the actions, okay, can change the values of some of the variables, okay, for simplicity we assume deterministically, so you know if you apply this action which variable is going to change and how, and you want to find a sequence of actions that is going to map some initial values of these variables that you know into some goal values, by the way, this is extremely simple model and this problem here, it's very hard to fit into this other model, okay, why, because in this problem there is incomplete information, so the initial state features a number of things that you do not know, you do not know where the womb is, where the pits are, even where the thing you are looking for is, okay, so this is a much simplified setting, that is still pretty generic, okay, the idea in planning is to come up with this box where you put problems of this form in this sort of variables, what are the initial values, what are the goal values that you want, okay, in the desired situation and what actions you have available and how they change the values of the variable, and the idea in this case is to come up with an open loop control, just a sequence of actions because there is no feedback in this model, okay, so it's a very simple type of model, so people in the AI call this classical by vanilla planning because it's the most simple type of model, okay, of course there are many richer models, models that involve incomplete information about the state where you are, okay, needed for dealing with the problem I had before, stochastic action sensors, okay, models that are as powerful as they are called, partially observable markup decision processes and I'm not going to get into that, okay, so the idea is to make this type of model much more interesting and challenging, however even this stupid, very simple model is already computational intractable, meaning that if you try to solve problems with a few variables, okay, even with the 20 variables and you are not careful, okay, things are not going to be solved, you're not going to get the behavior, things will explode, so I have this box, I want to build this generated box and I want to be able to input problems, okay, I don't care what the problems are, as long as I can model them as a set of variables, actions that change the value of variables and an initial situation that you start and a final situation I want to reach, in this case the variables will be the location of blocks and the actions basically allow you to change blocks from one place to the other, well it turns out that finding a plan, okay, the behavior that achieves the goal from the initial situation is finding a path in the graph, okay, however, what is the problem, okay, that the size of the graph is exponential in the number of variables, okay, so this is sometimes not so easy to grasp for people that care about scalability but this is a crucial problem, it's a key source of insight as we will see as well, for instance for 20 blocks, okay, this graph will contain more than 10 to the 24 nodes, okay, so bunch of 0s, so, and this looks like a stupid problem, okay, just solving a blocks problem where you have to go from one configuration to the other but just unstacking and stacking blocks, so the question is how can you find paths that connect a given node to a target node, okay, in such huge graphs, okay, so no time of the universe that will be sufficient, okay, to explore as many nodes as the ones that we have here, okay, related to this, okay, there is another idea that related to Turing inside, okay, so Turing what it shows is that there are certain perfectly well-defined problems that cannot be solved in finite time, these are non-decidable problems, okay, start them from the 80s, okay, so 50 years, okay, almost 50 years after Turing original papers, people also use, okay, Turing machines to establish another boundary between the problems that principally can be solved efficiently in polynomial time versus those that cannot be solved apparently in polynomial time, there are people in computer size, normally they call these problems NP-hard, you heard about NP-complete problems and so on, and the NP is not that it's not polynomial, but it is polynomial in a non-deterministic non-physical Turing machine that you cannot actually implement, okay, so people discover actually one of the open problems, okay, if you solve this problem, okay, you get the Turing award and probably more than that is whether this frontier between the problems that can be solved efficiently and those that apparently cannot be solved, okay, that are NP-hard, okay, so they are actually there is a line there or not, so this is the question of P, okay, whether it's equal or not to NP, something that is not unsolved, but everybody in computer science will think that the two things are different, that one class is polynomial, problems with polynomial time solutions, the other problem that require exponential time, sometimes even double exponential time as well, so these sort of exponential time problems include all the planning problems, but in order to see that the feature, so the number of possible features to look at grows exponentially with the horizon, okay, so basically choosing actions, getting a behavior from models, so you have predictions, so the problem of using a predictive model to get behavior is computationally intractable, so the point is even if you have good predictions, it doesn't mean that you can figure out what to do, the problem figuring out what to do remains computationally hard, and this is what the working planning is aimed for, okay, dealing with this computational challenge where in some sense evaluating the future by considering all the possibilities like solving a huge mental maze, so the question is how planners do this these days, okay, I'm just talking about the simplest type of planners, this vanilla where everything is deterministic, actions change variables in a way that you know, and so on, actually there is another question for those interested in how the brain works, okay, is how brains achieve anything like this, of course you can say that the brains don't have to deal with these huge mental mazes, but it's an hypothesis, okay, so there is no reason in principle that if you get a system to consider and to select efficiently course of action by considering futures, okay, this cannot give some insight into what the mind is doing, actually I will say that computational limits, not computers themselves, so many people in coccyx they use a computer to run their algorithms, okay, that's one use, okay, but the most interesting news is to understand the computational limits to see, okay, how you can tackle this problem, because if you don't exploit automatically the structure of problems, okay, you cannot deal with them and we don't know how the brain works, that is pretty robust and it doesn't break due to the number of variables, okay, so that is a basic computational challenge, okay, so if you look at this challenge from the point of view, you can get some insight into how the brain actually, one of the solutions that people have found for this problem, actually these solutions, so it came from working in the group is the following, very simple idea, okay, even though finding a path in these huge mazes, okay, is something that's still intractable in the worst case what you can get is estimates, so I am in this state and I can do three actions, I can take block A, sorry block C and put it on A, I can take block A and put it on the table, I can take what, block A and put it on top of C, okay, so the same idea applies when you have 50 blocks, doesn't matter, the point is, it's very easy to compute estimates of the distance to go, okay, so I can compute in polynomial time, low polynomial time, I can compute automatically no matter what the problem is about when it's expressed in terms of variables actions that change the value of variables and so on, I don't care what domain it is, okay, I can in the domain independent way, in the genetic way, I can get in low polynomial times numbers that give me some sense of direction to the search so if you go this way it looks you're going to be at distance 3 from where you want to go, if you go like this, okay, it looks that you're going to be at distance 2, these are just estimates, they don't have to be accurate, okay to get completely accurate estimates will be as hard as solving the problem itself okay, well here again you get 3, in this case the estimates are so good that if in each case you move, okay, you take the action that takes you closer to the goal according to these heuristic estimates okay, you just solve the problem with no search at all, actually some of the methods used to get these estimates allow you also to in a part of the node to tell you which are the nodes that are even worth evaluating, so here I can say these two guys are not even worth evaluating, I know they're not going to be good, okay, so basically you get not just by evaluating all the possible things that you can do in a state but if you want, something has a relation, let's say though the ways that people play chess, okay, people don't consider the possible moves, okay they get very very much immediate which are the moves, okay, that makes sense to evaluate and to consider in a more detailed way so by looking at the computational problem and at the computational challenge, so you get techniques like this one, okay getting these appraisals automatically in a domain independent way that are not only computational effective but at the same time have some cognitive appeal in case we might want to talk more about that later, okay, so more generally in AI and within the group we not only consider this type of mathematical models, okay, but we can see the variety of models, so some of the models that we consider for instance are considering satisfaction and SAT where the task is to find a value assignment to a set of variables that satisfies a set of constraints, in the case of SAT the variables are Boolean and the constraints are closest, the generalization of this is called constraints satisfaction, okay, so there are many many problems, okay, that can be mapped as SAT or constraints satisfaction and problems okay, many other problems for instance involve reasoning with probabilities okay, so Bayesian networks is a graphical language for expressing probabilistic knowledge on top of graphs and the task there is how to find a posterior probability of some variables given some observations a third problem is actually a variation of the one we talk about what happens if you are doing planning within complete information and you get feedback, okay, if there is uncertainty it makes sense to sense the environment so the problem I mentioned before this little game, the wombus, okay, you need to sense the environment in order to figure out what to do next in those cases the idea is how to get a strategy just not just a sequence of actions because what you do is going to depend on what you observe and of course there are models that involve not just one actors but many actors, these are called the multi-agent problems that bring a number of subtleties that way and that relate also to ideas in game, theory and so on, what is, okay in all these cases what people in the eye and what we try to develop at these boxes, okay, so we are trying to develop programs that will take any instance of these problems or the planning problem of a planning with feedback of a basic network and at the same time compute approximate solutions you cannot guarantee you are going to get optimal solutions, you are not going to guarantee that you are going to solve all the problems because all these problems are computational and intractable, okay they are sometimes the algorithms when they are complete sometimes are exponential, sometimes are double exponential but the point is to push this sort of combinatorial explosion as far as possible some of the models are actually particular expressive, okay, some planning models, these ones called POMDPs, okay, are relatively as some people call them AI complete, that means if you could have a solver that will take any compact representation of a partially observable probabilistic problem in the input and to tell you what to do in the output, okay you will be a long way to having a generic and a general intelligence, okay, but these are sort of very intractable problems where the key issue is how to do cost effective inference in order to scale up, okay, it's not just to have an algorithm that is complete and correct nobody cares, okay, okay, of course you care okay, but that's not the challenge, okay the challenge is to be able to scale up, actually the methodology in all these areas these days in AI as opposed to what I mentioned before where people did demos, okay, up to the 80s is based on benchmarks and competition because in the worst case all these models are computationally hopeless, okay, but in practice, okay, many, many problems can be actually solved, so SAT for instance is a problem that's NP-complete meaning that in the worst case any algorithm will take exponential time, okay, but on the other hand a problem to SAT involving 100,000 variables and 100,000 clauses, okay, it's likely it's going to work very well, okay define these worst case behaviors, so you want in some sense the worst case behavior to be exhibited only on the worst case instances, okay, that are normally not normally the instance that one is dealing with now many of these cases there are competitions so it's relatively easy to enter in any of these things, okay, you have a very good idea about how to exploit the structure for any of these models, so the best way, okay, to go around you enter into the competitions, you do well in the competition, you win the competition and then people pay attention, okay, to the technique that you have, they cannot avoid doing that, something otherwise it often happens inside actually among the relatively new competition is this general video game competition that started in 2015, the idea of this competition used to be one of these boxes and the people organizing the competition, they get this box they feed it with video games that you have not seen and they basically score the different solvers according to how well they do in the video games, same thing with the planning competition and so on, you submit your boxes, okay, they feed it with different sort of problems, you have not idea what the problem is about just the mathematical structure, okay and all these solutions are evaluated in this way, actually some of the benefits of this genetic approach, these genetic algorithms is suddenly you may get new problems, okay and you can take from yourself one of these genetic algorithms and use it, okay, so I'll show just results from one of these algorithms also on the Atari video games so basically this setting was so this interface and this setting, okay, not the algorithms but the simulation setting was done by people in Alberta a couple of years ago, three years ago, okay actually the setting that the people in DeepMind use when they publish a nature paper about how to learn how to do these things in deep reinforcement learning and so on, okay we took some algorithms, okay, that we developed in the past, okay, with a couple of former students of the group Nir and Miquel and we say, so this looks again, the mathematical structure of the problem looks that it's one that is very well fit to the genetic algorithms that we have, so let's try it and let's see what we get, okay, and actually it's a very simple paper, okay, to write because it's taking an algorithm of the shelf, just adapting it to the interface and at the end you get an algorithm that can play any of these games and this is the algorithm that you develop for something that is completely different, okay you see it can relatively discover that it's a good strategy, okay, once you get past the block to knock all the things on top, the same algorithm, for instance, is used in actually so the same algorithm, okay again, was not developed for any of these settings, it's completely generated, this is another game and actually in this, by the way, this algorithm of ours tends to do much better than the algorithms of DeepMind, okay, even though it's fair to say that the two things are computing slightly different things, so in this famous sort of paper and results that they have they learn a reactive controller, we don't do a reactive controller, we use the simulator to simulate a little bit into the future but again, so the number of possibilities is huge and to select the action to do actually in these domains they do pretty bad, okay, because in general Monte Carlo methods will not do well in this type of set, I'm not going to get tired with more examples of these, but some of the applications of these genetic algorithms that we are looking into, okay, right now are in video games in robotics and slowly, okay, we want to get into issues that have to do with dialogue, okay, again that is our goal directed activities, okay, in some sense it's planning with incomplete information, in the case of dialogue it's also a multi-agent problem, okay, where you have to model not only your beliefs about the world, but also your beliefs about the other guy's beliefs as well, okay, so I'm doing in terms of time, so I think I'm doing very well, what did I do with the slides? I lost them okay, now thank you okay where can you help me, where is what is this presentation, okay here, here, okay you can ask questions, okay, in the meantime if no, let's see okay quickly, quickly, quickly okay, so it's time to wrap up, so of course I didn't get into AI to play Atari okay, I wanted to get to the mysteries of the universe, okay our place in the universe okay, so where we are, where we are going and of course one of the appeals in AI is to use the computational perspective to tackle some interesting issues, actually questions like can a machine think? I always wouldn't make fun of this question, I said we have made a big progress in the 50s the question was can machines think now, okay, 60 years later so the question is can people think okay, so big progress there, okay second question, can a machine be 100% rational, okay okay, the only guys who will say probably yes okay, they even think that people are 100% rational, are who economists, okay okay, in computer science we know that's impossible, you cannot be 100% rational even in a logic as simple as propositional logic, because all these models are computationally intractable, okay so if you understand, okay, the limits of computation, all these questions have a very trivial answer by the way, one of the founders of the field of artificial intelligence is also one of the founders of what's called the school of boundary rationality in economics and the guy actually got a Nobel prize in economics before founding with other people's eyes, Edward Simon okay, he was very well aware of that problem another one that I like okay, so can a machine have free will okay, so the question I like to pose actually is, does a chess playing program, okay have free will of course, the answer from everyone will be of course it does not, okay it's just limited to choose one of the moves from a fixed repertoire okay, by applying some evaluation function, okay good that would change the chess playing program and now allow you, I allow this guy to play many other things even to play life and still works the same way, many options okay, looks ahead into the future and decides what to do, okay does now this program have free will or it doesn't, in principle again the tendency will be, okay it is no free will, because from the outside, if I know how the program works, in principle I could predict, okay which action is, was going to take, okay, now the question, I'm going to answer here just to pose some question is whether there is another sort of free will worth having or that you may have in a relatively materialistic world, okay, but okay, I'll keep the others reaching to some of the mystery okay, I mentioned at the beginning about the fears of artificial intelligence that people talk about the singularity that soon machines will be more intelligent than people and then there's going to be an exponential explosion because they're going to be even better machines and so on okay, so my point of view that's nonsense, okay that's nonsense, okay, whether you take this story to be utopic or dystopic, okay it's like some people said it's overpopulation in Mars, it's not in the foreseeable future however, there are some other problems that are in the near future and one of this is automation okay, so automation is a big problem okay, and it's a big question it's about how society, okay is going to deal with automation where in principle some jobs okay may end up going and may not come back as normally has happened in the past okay, now of course you can take an optimistic view and say, okay so how come in a society, so if you want to work less there is less work to do that's going to be a negative effect it looks that there will be a positive effect, okay, less work to do but as things are now, okay it's not good to be without a job okay, so in principle it's down there, actually so I would say that many of the problems raised by automation and artificial intelligence in some sense has to do with this as well okay, I have to do with with society, okay, so we are scientists but before being scientists, okay, we are citizens, okay, and of course we want that all these technological developments benefit society at large and and it's clear that the interests of society and the interests of corporations are not always perfectly aligned, okay, to say at the least, okay, so the question is what we do as citizens when they are not clicking likes and things in the Facebook, so actually I think I have this say I couldn't avoid bringing one thing from El Roto, so El Roto in Spanish says, I don't know if this is readable at all, so we are developing smart machines as men, no, we are developing smart machines as men for stupid men as machines, or here my bad translation in English, we are building machines that are intelligent like people, for people that are stupid like machines, okay, so I think it's very acute and we can talk about these things too, okay, just to wrap up I think so what we have seen in the press two things, one is about fears of artificial intelligence, about applications normally having to do with the use of data and machine learning, okay, so the idea that this is the arrival of artificial intelligence that's still a long way to go to get something resembling, okay, the generality and flexibility of human intelligence is like if you just, if you climb on top of a tree, okay, doesn't mean that necessarily you are closer to the moon, okay, so the moon is still a long way to go there are many things that machine learning techniques are not intended to do and normally what they don't do is about taking action, okay, by evaluating the possible futures, that's a critical notion for us humans and of course it's also critical for any notion of of general intelligence, about this by the way, okay, it's good to mention that this company has shown that they can get very good reactive chess and go players, okay, meaning without looking into the future so what they have done is traditional chess playing programs they look into the future, okay so I see this is what I do, what the other will respond, what I respond to to the guy then they go several levels and then they back chain, okay the value of the different outcomes and then they do what appears to be best given the possible response of the other guy, okay, normally programs being machine learning, so in the case of Goa and in the case of chess, some people have shown that you can get pretty good chess playing programs and even Goa playing programs, okay that's right, harder okay, but just learning such a value function that you don't have to look in the future, just one step you do one step and your evaluation function is so good, okay that in principle, okay, you don't need to look further, okay so in some sense this casts out about so what is the importance of looking into the future that gives these programs and this company always showing that you can do pretty well by learning techniques okay, in some sense learning reactions that just look one step into the future, now one thing about this is that they learn these value functions, okay, for very specific games using many, many examples so these programs that I do this okay, they don't have any flexibility okay, these programs that just plays chess, cannot play any other type of situation, the same thing about the learning that went into the Goa program only plays Goa okay, so the idea that you need to look into the future, okay, it's related to the idea of generality if you have something general, it's very hard to learn the value function that will span all the possible situations that may arise, okay something else, still AI is, I would say, far from the capacities of a far year old child, I heard in a Shioni, it's now heading AI center, somewhere he said, about deep learning and all this news, that this is not the end, he said this is not the beginning of the end of artificial intelligence he said, this is not even the beginning of the beginning, maybe I will not go that far, but still there are many things to be done, of course in spite of that you can do pretty well in many different types of needs, just one last thing, okay, so many immediate, okay, risk and possibilities but this requires informed and concerned citizens, okay, so if you know one of these, so let me know okay, thank you okay, I'm the organizer I introduce myself, but I also take the questions okay so easy ones first, Horacio Thank you for the very entertaining talk and I have a question regarding the general solutions I wonder once you have a problem, usually you have to model it and usually you have to develop features to represent your model and features that are humanly developed so how how you use a generic solution in that case when a problem may change and has not the same structure or the same domain that the previous problem so for example, if you play a game maybe there are some characteristics that make them uniform to reuse the same features but how you apply general solutions to totally different problems okay, good question so respect the word features, okay so in this type of model normally one talks about variables that keep track of the state of a system, normally you are trying to control some system okay, some environment there are variables and in some sense they go the state of the system so what we need are these variables that encode a true state okay, as some people call the Markovian state, so that in some sense when you reach a state, it doesn't matter how you get there so for the future it all matters so the state where you are in so in a sense these variables are different from what people call features in machine learning algorithms that if you run the same algorithm, okay, by changing the features and you have to do a lot of engineering so that let's say the classifier works correctly, so here in principle so the issues are slightly different, now machine learning and this type of model based approach are connected in the way that in many cases these models that we use to take decisions they can be learned they should be learned from data, so ideally okay, I want to be able to build these models okay, just buy an stream of inputs that only contain actions and observations, so I have a robot and I want the robot who wants to learn how to use his arms to pick up a ball let's say starting from scratch, so the optimal thing that you could do the ultimate challenge is just to let the robot build a model okay, but just looking at streams of actions and observations, so it does different things, it observes different things so it can infer the state space of the problem himself that here we take from given and once that you build the model then you can use this model not only to solve that particular problem, it can be reused so also many other types of problems that even change in the Gauss initial conditions and so on so what I'm saying is there's not so much in the split actually a whole branch of what is called reinforcement learning that is called model based reinforcement learning where the idea is not to get reactions but to learn the models themselves, so in some sense in model based reinforcement learning the reinforcement learning problem is mapped actually into a planning problem so the two things are, so even though I split them here they are more connected that when they appear, in many other cases if you had an industry and have a problem you have logistics and you have the HL and you want to do it with packages so it's easy to model directly by hand, but you don't have to encode a solution, you just encode the dynamics of the problem, you encode the sensors, you encode all these other things good Luca, I love them it's really still the general problem solving kind of paradigm and this paradigm at a certain point died at least as a point of reference and it only recently came back to life essentially one of the reasons why it died has to do at least in the ideology I cannot talk about the technicalities from some very weird ideas that were presented particularly by Jerry Fodor at a certain point which was the following Jerry Fodor was this philosopher he's this philosopher on the one hand he put clearly what I do, cognitive science in the field of artificial intelligence, the idea that he had was that essentially cognition is a computational discipline not a biological discipline for a reason that he could argue but at the same time that what really general problem solving wants to do is impossible in principle, the reason why it is impossible in principle is essentially because we solve lots of problems because we somewhat know how to evaluate the relevance of a piece of information on the fly but there is no theory about relevance period, okay, so on the one hand he said cognitive science is very you know very much a computational science on the other hand he said the kind of thing we want to solve with the general problem solving is just really impossible to go beyond what we can do so the very first part of the question is how much do you think this theory makes sense in your opinion because you are very much into the same kind of dilemma but then there is a second facet which is and that's where the question got a little more disorganized but it's part I'm more interested in and the best way to phrase this is the following I can think, so we are these kind of beings who are really good at solving problems, period so I would imagine that that there are many hints that artificial intelligence can take from human behavior not because we are you know the greatest or not because we are the optimal solved problem solver who certainly are not but because we just are the best around essentially and so I can imagine very many cases in which basic ideas of artificial intelligence computer science in general have influenced a lot cognitive science starting from the difference between memory and programs I mean many many of these kind of things do you think of any idea that we can provide that could influence the way you work or in a better can you give me an example in which any of the kind of problem solving that humans actually do has been factored in the way you work to improve the way an algorithm works or a kind of solution problem works in other words you know yeah questions first about Jerry Fodos criticism and so on so I will say that this type of criticism of Jerry Fodos problem of course didn't have any influence at all in AI or anything this happens often with armchair philosophers so you are in the trenches dealing with the algorithms and with these things and this is a very genetic sort of philosophical argument so the question is there are too many relevant scenes so it's related to what I'm saying so given that you have hundreds of variables and only few of them are relevant so how do you do how do you don't explode with the number of variables that's precisely the computational issue and the point that the novelty respect to what happened let's say in the early 60's actually as Luca said one of the first AI programs at all was called the general problem solver actually one of the others was a assignment that I mentioned before and the idea was very similar you fit problems in the input and you get solutions in the output what was the problem with the general problem solver is that computationally it doesn't work and I would say and a lot of the work done in the 60's and 70's and up to 80's in AI was also very introspective so people were trying to use insights or observations from psychology and so on Simon in particular was sort of participant but this type of introspective based AI in the 90's got a very bad reputation so these days we had to publish in an AI conference if your justification is this is the way that people do it so what you have to show is that your method actually works and the way that it actually works works much better than other methods around is to see it with many variables and see whether your solver will break or not and the challenge in some sense the journey for those is not an impediment is part of the challenge and there are many ideas that have been developed for addressing this challenge that is part of the computational complexity how not to have solvers that will break merely because the problem involves many variables there are problems that inherently hard I don't care about those in that sense from a computational point of view some problems you will be able to solve this is intracted and be hard of course things that eventually will break but the question is when do they break do they break when you have 20 variables or they break when you have 1 million or 100 million so it makes some difference and this in some sense is what is being an issue this is about the first part second part is about exchanges between let's say psychology, neuroscience and AI coming let's say from psychology and so on that we can use computationally so let me just briefly mention about the other way as you mentioned things like basic and networks reinforcement learning some of these ideas are having some impact in some areas of neuroscience and cognitive science the other way it's not a good bridge to computation so our problems are computationally for instance take the theory of mind so the theory of mind is something in psychology is very much sort of in fashion for the last 10 years or more and it's about the ability of people to deal with other people as different than objects so there is a world of objects and there is a world of people and there are many things and many abilities they have to do with people so psychologists give a lot of importance to the theory of mind and actually even kids very small kids are very good about dealing with the beliefs of other kids about their reactions and things like that about stories that involve multiple agents and so on but from a computational point of view when you look at this theory of mind ideas it looks like magic and looks to me to take another idea that to me I'm sort of impressed how lightly it is taken in let's say neuroscience mirror neurons so mirror neurons some people ascribe them sort of things that are impossible to compute and they are going to be computed that's crazy that empathy is based on mirror neurons that's a very complex thing so in a sense that is a big distance have to do with computation so we focus computation is already a very restricted game so even those of us whose real interest is the human mind we are sufficiently entertained but just paying attention to these restrictions while sort of not being able to assimilate or to use insights coming let's say from the behavioral science so that's for instance the Bayesian networks they are all over when Julia Perp published the book in 88 if you look at the cover of the book in Bayesian networks this actually is the picture of the cards basically showing that the guy has something like a Bayesian network in the head so I made fun of Julia at that time was my advisor and say okay so you are a true believer in Bayesian networks that you think that we have in the head but so years later you have many people from Berkeley Josh and some other people who actually use it the whole theory of Bayesian networks to understand how babies so learn about causal relations and so on so I would say that the computational perspective is very crucial and very interesting source of insight and I'm not sure that people were not working with these sort of understand the importance of that why by the way why Bayesian networks were developed probability theory existed for centuries at least two centuries but Julia developed Bayesian networks in some sense to make the representation of probability destructible so the first Bayesian networks were trees and rather than having exponential time inferences polynomial it's actually linear in the number of variables so many of these insights came by just trying to comply with certain computational restrictions so I think even for understanding the mind but that's pure speculation it's good to pay attention to that and that's also what makes it hard the other exchange that's a long answer ok and people are getting hungry and angry ok any other question I promise to be short and quick ok so I'll take this moment of hesitation to thank you for being here and until next time