 So, my name is Shailesh. I am going to talk about a topic called reasoning and what we will do in this talk is sort of contrasted with prediction which is what a lot of talks and a lot of machine learning that you have seen today and and you know in your work is really a different kind of machine learning which is mostly prediction kind of machine learning. So, we will talk about another kind of machine learning reasoning which is which is what we do as well as we do prediction and you know really what I want to do is show when your problem is really a prediction problem, when is it a reasoning problem and if it is a reasoning problem what are the things you can do with it right. So, that is the broad theme. So, let me start with something I talked about last year which is if you think about where is AI going and you know there are lot of speculations these days whether it is going to kill humanity or it is going to redeem humanity we do not know which way it is going, but I am going to talk about it from a very simple perspective which is there are lot of things that we call AI today right. So, lot of companies are using AI and it has become a very big buzzword for a lot of things. So, let me just separate out a bunch of other things within AI. So, obviously, the first thing we do is we do a lot of learning we learn the you know n grams we learn the prediction model. So, there is a lot of that that goes on which is AI then we do understanding. So, if you look at word of meddings or image understanding and things like that we do a lot of semantics right word sense disaffeguation. So, that is another kind of AI if you will then the third kind is what we call thinking or reasoning and we are going to spend a lot more time on this one today what is thinking and I am going to talk about thinking 2.0 because last time I promised we did thinking 1.0. So, today we will do thinking 2.0 and what is creativity? Obviously, there is a lot of philosophy around it and finally, what is consciousness and if you get time at the end we will watch a video about this, but the idea is those top two are still in the realm of philosophy and metaphysics, but these three are something that we are actually doing. So, you know we have all seen a lot of things in this realm I want to focus a little bit on what is reasoning and thinking. So, you know if you if you want to keep thinking about data science there is another way to think about it that we are going through a transition right now from pure data science the way we understand it and AI which is kind of the all consuming and I talked about this the last time which is unless we keep sticking with language and the idea that there is something called language we are not going to make the leap to the next thing we need to start getting away from language towards knowledge because really the purpose of language is to communicate and store and transfer knowledge and knowledge is the essence language is just one form of knowledge representation right. So, this is one thing we talked about second a lot of things going from tokens to embeddings now. So, if you look at the old world we used to do TFIDF in the new world we all do word embeddings and vector embeddings. So, we are going from tokens to embeddings and the third thing we are doing is we are also going to go from prediction to reasoning and I am going to make that distinction today which is what is prediction versus reasoning and where do we apply reasoning all right. So, let us look at a set of tasks that we all do in our in our jobs right. So, let us say I have a task of predicting whether this transaction is a fraud or whether you know these coupons should be sent to those customers or whether this job you know is relevant to this resume or vice versa whether you know what are the objects in this image is there a hacking going on into my cloud system or my my other systems this has become very popular these days and whether this customer is about to chart and all these problems have a very very specific very very same type of sort of structure to it and and you know these are all what we call prediction problems. The idea is you are given some input and you have to predict something and there is a whole variety of these problems that you can think of it does not depend on what kind of data you have it does not depend on what you are trying to predict I am not making distinction between classification and and regression or those things I am saying all these are prediction problems right and obviously, if you look at how prediction has evolved it has evolved from you know since since the early childhood if you will we all started with the basics and then we had neural networks and kernel machines and support vector machines and random forest and things like that and today we are doing something even more interesting we are doing deep learning and graphical models and xcboots right. So, this is a long history of where prediction is and it has evolved to in into a very very complex thing because we have put so much effort into building prediction models all the night right and and we use this to do very very complex tasks now, but it is still the same paradigm it does not matter whether you are driving an old car or a new car you are still driving and I want to make that point about the paradigm shift we are not flying yet in that sense you are still driving our cars have become faster and better, but we are still driving. So, it is not a paradigm shift from these guys to these guys we have not done a paradigm shift in terms of what do we do in terms of input output mapping right. Now, let us look at another class of tasks these are the tasks that we do very naturally machines also do, but I want to make the distinction. So, it is obvious. So, if you look at the task of for example, you know finding your shortest path or multiple paths from A to B. So, this is Google office in Manhattan this is the central park and if I say give me a bike route it is going to give me three paths and and so on right. So, we use this all the time what about the recent games that we have been winning right go chess and all that what about solving math problems right. When we solve a multi step math problem you start with this and you say ok what can I do what can I do and you keep going and you come to the answer this is another class of problems what about conversation. So, I am going to model conversation not as a prediction problem, but is it really a prediction problem right and what about diagnosis or what doctors do to diagnose right or what Sherlock Holmes does to to find the clues and solve a mystery. And you know the task that is closest to most of us is how about crossing a road on a busy street ok. Now, what is common between all these tasks there is something very common between all these tasks and that is what we are going to talk about that these are all what we call reasoning problems. They are not prediction problems they are reasoning problems and we will harp on what that distinction is and how do we get there all right. So, reasoning also has been evolving as prediction has been evolving, but now it is a paradigm shift. So, now we have started flying and you know in the early days you must have heard of the things in A star search and shortest path and theorem proving and rule based expert system. They were all designed to do these reasoning tasks which we will see what they are. Then came these Markov decision processes or or reinforcement learning Q learning ideas. I do not see a lot of those things used in our in our day to day efforts today and hopefully this talk will sort of give us some more ideas around how we can use these ideas into into our work. And now we are we are also talking about deep reasoning or hierarchical reasoning or multi agent systems right. So, the idea is reasoning is a very different ball game it is very different from prediction it has been evolving the same way prediction has been evolving and we just need to know when to apply prediction when to apply reason ok. So, let us look at the differences between prediction and reasoning. So, if you look at the prediction paradigm it is a very simple paradigm you start with some features that you have created either through feature engineering or statistical things and then you build a model and it gives you a prediction right with a very simple input output mapping. And the characteristic of this is that the next data point is independent of the previous data point right IID assumption. So, you do not have to worry about state in this case right a prediction is done move on next prediction is done you can do a billion predictions per second today and they have nothing to do with each other. It is a very simple paradigm and that is what has led us to this far and we have been able to do very very complex things with just this paradigm. What is the reasoning paradigm? How is it very different is that it is not a one time prediction a reasoning paradigm is really about multiple predictions and trying to figure out after this prediction what is the next prediction I will do and then the next prediction I will do right. So, imagine if you start with let us call it a state and you say I have three choices and then you take this choice and then again you have three choices then you take this choice and so on and so forth. And that is sort of the idea that here you are not doing a one you know a mapping between some input to some output, but you are making a series of things right. So, if you look at how Google search finds your path or how we play a game really it is a sequence of decisions that we make and it is a very different paradigm and we will see how that works. Another interesting thing about reasoning is that in case of prediction you get an immediate feedback right. So, you make a model while training as soon as you are done with the output you get a expected output and you get an error that you back properly right. So, every prediction algorithm does that in some form or the other, but in reasoning what happens is you do not get a feedback right away right you make the first move of chess you do not know if it is right or wrong you may move the 10th move in chess you do not know if it is right or wrong. At the end of the game you realize why one or loss right similarly in other such problems we make a sequence of decisions and sometimes only at the end of those sequence we get a feedback and then we realize whether we did the right thing or not and one of the problems is then you start blaming and say is this the wrong decision is this the wrong decision is this the wrong decision why did I lose the game right. So, that is our difference between prediction and reasoning. Now, there are different flavors of reasoning like I said reasoning has been evolving since since a long time. So, the very first kind of reasoning was very algorithmic. So, this is where all the computer scientists got together and said ok we know how to do breadth first search, we know how to do depth first search, we know how to do branch and bound and all that and now how do we take this forward and then we came up with A star search right. So, A star search was a very important algorithm in AI and it is an example of what I call algorithmic reasoning. It is a computer science way of doing it it is not a data science way of doing it, but it gets you the same result. Then you know for a very very long time in the history of humanity we have been doing logical reasoning right. So, we have predicate calculus and all the calculus and we do a lot of theorem proving and there is a lot of theory around around how do we do logical reasoning right. If if all birds can fly and this thing is a bird then can it fly right that kind of thing and we have been doing this for a long time and this is the foundation. These two algorithms are sort of the foundation of the modern AI and and they are all reasoning algorithms. Another kind of reasoning you can think of is a Bayesian reasoning which is you are given a bunch of you know cause and effect relationships and you want to understand what caused what and there is a causal structure around it and a lot of work has happened in graphical models Bayesian networks. So, there you can talk about you know if if you have fever what would be the cause and what should I do with it right. So, you can go backward and you can go forward and there is a whole bunch of theory on how to take logical reasoning which is binary true and false to probabilistic which is Bayesian which is what is the probability of fever versus do you have fever right. Today I am going to talk about two other kinds of reasoning I call them this one I call a Markovian reasoning which is really about the states and actions and you know the sequence of actions that we take another you know broad area is Markov decision processes which is sort of about this kind of reasoning and then I will also talk about reasoning that we do in our brain you know related to associations right and knowledge graph and things like that. So, we will focus on these two and see what these are and how do we apply that all right. So, let me introduce this notion the idea of a Markovian reasoning right. So, there are these notions of states actions and goals. So, I am going to use several examples in different domains. So, we really understand these ideas. So, that we know how to apply them when we have to build our own reasoning systems right. So, if you think about the reasoning problem that shortest path is solving we have two states you have a start state and a goal state. So, the idea of state is very important then you have other states right at any given time you could be here you could be here you could be here. So, the location on the map where you are at this point is a state. Now, in addition to that you can say is it raining is it you know flooded is it blocked right that is also part of the state if you will. So, state is anything that you can observe is state not anything that you can control, but anything you can observe is a state ok. So, let us look at actions now. So, at any given state you could do multiple things you could go here you could go here you could go here. So, at every state you can define a set of actions you can take right. So, the idea of states and actions in the map world and the overall goal we are not just talking about this goal to reach you talk about the overall goal of this whole problem is to reach the destination as safely and quickly as possible right. So, there is this broad idea of the goal there is this idea of the goal state start state actions and states let us look at tic-tac-toe right. So, if you look at a board position on a game that is also a state. So, this is also a state and then you can your goal here is to win the game and and obviously, you have a bunch of moves that are actions right. So, you could take this action or that action and again depending on the right moves or wrong rules you are going to win or lose. What about mathematics again we have states. So, now I am going to introduce this idea that let us say I want to solve this equation or expression simplify it again I am going to say this is a state and what are the actions the actions are the things you can do in the current state right that is the action. So, here you cannot do sin square plus cos square equal to 1 identity because that is not possible what you could do these things right and depending on what you do you are going to get into new states right. So, that is a very simple paradigm and therefore, it it can be expressed in this is in this form states and actions all right and here the goal is to simplify the problem in the minimum number of steps for example, all right. So, let me talk about another completely different kind of thing and see why that is also a reasoning problem. So, let us talk about chatbots. So, every second or third startup is doing a chatbot today. So, let us talk about chatbots now if you look at two kinds of chatbots. So, I distinguish between two kinds of chatbots one is what we call a form filling chatbot. So, these guys help you fill a form in the background and then they submit the form and things get done right. So, if you want to go to fly to US you talk to your chatbot and say I need to fly to US and it is going to ask you a bunch of questions. Ultimately what it is doing is filling a form which you would have filled at the other dot com or one of these websites and it is just another interface to fill the form because we love to type and not click anymore right. So, that is the idea behind what I call form filling chatbot. So, what is the goal in this case? If I want to model this as a reasoning problem what would be the goal in a form filling chatbot right. So, the goal would be a fill the form accurately do not assume anything fill the form completely do not miss out anything and take the minimum number of steps do not ask stupid questions and things like that right. So, that is the idea the goal is defined what about state? What is the state? The idea of a state in a form filling situation. So, imagine you are filling the form and you are at a certain state you have not filled certain things you have filled certain things think of that as a state. So, here also if you think about what is the request. So, this chatbot can do all kinds of things right it can book a cab it can buy some stuff it can order food. So, what is the request? What is the context? Where are you? Which city you are in right? Things like that what is your context? What are your past preferences? Do you like veg food non veg food? Do you like oil versus this seed right? So, these are your past preferences and the chatbot knows about it and what is the metadata? What is your credit card number and all that right. So, think of that as a state and the state is being filled as we go along. And then what are the actions the chatbot can do? The actions are the things that the chatbot can say to you right. Those are the things he can say to you and he can say ok when do you want to travel? Where do you want to travel? It can also tell you the today's weather, but in this conversation is that a good action right. You do not know if this is a good action or you can ask you about the movie that you watched yesterday. Say how is the movie right? It is trying to have a conversation or you are trying to book a flight. So, just because you can have an action does not mean it is a best action to have. And the real problem with the thing the chatbots that we have to solve is how do you ask the right questions? How do you pick the right actions? So, that you meet your goal in the current state ok. Now, let us talk about another kind of chatbot which is very different. This is what I call a small talk chatbot right. It has no goal. It does not care about your booking or doing something. It is just a general thing that you talk to and what is the goal there? You know you just having a casual conversation. Exactly. Exactly. Keep the conversation going. Thank you. I will do that right. So, basically the this guy is basically say keep the conversation going and you know keep it interesting right. Keep it coherent. Do not keep talking about random things. So, that is a goal right. It is a very abstract and a very different kind of goal than the first one. What about the state? How would you define the state of such a conversation? Last thing that you said ok. Mood right. So, you can think of the state and say what is the state of the conversation and you can think of you know what is the stage of the conversation? Have we just met? So, I should say hello hi or has the conversation been going on for a while. So, I should say something else right. What is the stage of the conversation? Am I about to end the conversation? So, should I say something else? What about the interest of the other person right? I am saying something and this is what we do in human conversation a lot. We try to gauge the interest of the other person and see if we need to change the subject and things like that. Where has the conversation been so far? What all have we talked about? So, we do not talk about the same things again and again right. And like you said what is the last utterance and based on that I want to make the next utterance ok. And again the actions could be many. So, you can say hey I could exchange pleasantries hello hi bye. I can make a statement the weather is great today or I could ask a follow up questions right where do you live for example. And then I can change the topic depending on whether the interest is there or not right. So, that was one of the goals of even if you look at the Turing test and those kind of Turing test kind of systems. They are really trying to solve these problems which is how do you have a general purpose conversation which has no specific reason and yet be able to keep the conversation going and feel like the other side of the human right. So, this is the real chatbot that we need to build eventually this is a good stepping stone in that direction ok. All right. So, let me now ask you. So, we understand the idea of what is the state, what is the goal and what is the action. So, now let us talk about the atomic unit of reasoning. I like these questions atomic unit of something. So, what is the atomic unit of reasoning? What is the simplest atomic thing we do when we do reasoning? Pardon? Trinking ok? Decision making ok? Backtracking ok? If you keep backtracking then there has to be something forward looking too right yeah next action right. So, let me let me put this problem in a different way. So, let us say your friend calls you and he says hey I didn't how do I come there right. He says how do I come there? What is the first question you ask him right and then what do you do? Then you decide how do we get here and he takes a left that is all this is the simplest thing that we do when we reasoning. So, the idea is you have a goal right I have to come there and you have a state where are you now and then you have to decide what the action. So, this is the building block if you think about back propagation what is the building block? A neuron think of this unit is the building block of reasoning. So, you can build any kind of complex reasoning system by putting a whole bunch of these units together right. So, the real goal is how do we think about goals and actions how do we represent them and then how do we learn this box over time yeah alright. And the best part about reasoning is that it is not a static thing unlike prediction which is done once you are done with the prediction you are done what happens here is once you take an action it changes your state and now you have to do it all over again the goal does not change, but your state has changed. So, now you put in the new state and then again you do an action. So, now you see how we learn to play chess and go and bicycle and everything it is really about this loop and really these are the units that we use to build our reasoning inputs. Now, let me present a very ethical dilemma. So, we talked about state actions and goals. Now you know we are living in an era of autonomous cars. So, one of the recent news around that was that why the self-driving cars must be programmed to kill right. That means, it is ok to kill should not be you know there is a threshold to it you cannot just say definitely not kill and in that article they show this picture. So, the idea is the car is coming a pedestrian is crossing and so, there is a state the idea there is a state the car can observe the state and then the car has to take an action right. Again a car is coming bunch of people are crossing and again the car has to take an action, but the third figure is where things become blurry right and this is where it says hey if you take this action you are going to kill this guy and you cannot turn this side and you cannot stop immediately what are you going to do. So, eventually our reasoning systems that actually depend on or you know on which such things depend on will have to not only think about states and action in a very simple way, but also incorporate some of these moral dilemmas into them ok. So, we understand the idea of states actions and goals and now let me talk about Markovian reasoning what this whole processes and here what I am going to do is I am going to distinguish between what is knowledge learning and reasoning in a in a in this in this context ok. So, let us go back to tic-tac-toe you know the AI researchers favorite game tic-tac-toe we always go back to it and now tell me who tells me what to do next. If I throw a board position in front of you who tells me what to do next not what to do next, but what are my options how do I know what to play next. So, this is the state right I am in this state I need to win the game and I need to make a move. So, there are two parts to making a move the first part is I need to know what are my possible moves right in optimization they call it the feasible solutions and the optimal solutions right. So, what are my feasible moves in this state and that is what we want to know. So, who tells me what are my feasible moves rules of the game right. So, the rules of the game is the knowledge in this case rules of the game is the knowledge and that only tells me these are the possible moves you can make it does not tell you anything else knowledge does not answer the next question which is which is the best move it only answers this question if you present me a state I will tell you what are the possible moves how about math what can I do. So, if a student is struggling with math he is looking at this equation how does the teachers ask him what to do next right. So, the student knows certain things the student does not know certain things based on that knowledge the student will say ok either I could do this or I could do this or I could do this and it depends on the knowledge of the student right if the student does not know this formula he cannot make this as a feasible move. So, we understand the idea the role of knowledge plays knowledge is really about enumerating the options that is all it does and says what all can I do at this stage. Now, what about the complexity I want to define this notion of complexity of knowledge. So, we all understand at some intuitive level the idea of complexity of data right data is complex because there is a very difficult decision boundary to learn like VC dimensions all kinds of things you can think of. So, the idea we understand at some level what is complexity in data we also understand what is complexity in a model right the larger the model the more complex it is. Now, let us answer this question what is complexity in knowledge what makes knowledge more complex or less complex the more connections more alternatives right ok very good. So, I will give you a very quick anecdotal example. So, one day I was watching this movie which had a thing called the extinction level event. So, I passed the TV and said that to my 8 year old hey do you know how the dinosaurs died and she says yeah I know how the dinosaurs died and she was about to tell when my 3 year old came and said I also know how to how the dinosaurs died right and they always know all the answers right 3 year old they will know all the answers you talk about knowledge right. So, we all gathered together and we said what is her interpretation and we said how did the dinosaurs died and she said they died like this and this is simple right that is simple. So, the idea is depending on the knowledge that you have you are going to make those kind of answers right. So, a 3 year old's knowledge versus an 8 year old will give you a different answer. So, that is the idea of complexity. So, as we grow we learn more and more complex things right we learn math from so many years to so many years we learn so many things in a way our complexity of the knowledge is growing right. So, what does that really mean in our terms in data science terms if you. So, let me put that question in a very simple way. So, let us say if I ask you which of these 3 games is most complex go it has more more complex right. So, if you look at these 3 I can think of knowledge complexity in multiple ways right. So, I can say hey the complexity could be the number of rules right how many rules do I have to follow. So, if you look at that as an example then chess has a lot more rules than go right and obviously, this guy has very few rules. So, the order will be different go will be less complex than chess if the number of rules is what thing was right. So, knowledge the size of the knowledge if you will the number of rules does that make something complex right that is one notion of complex I am not saying yes or no another notion of complex could be the size of the state space how big or how many possible states the system can have and obviously, this guy can have 2 to the 9 possible states and, but chess and these can have a very large right. So, that is another notion of complexity which is how big your state spaces and the the third reason that a lot of people talked about is really the branching factor which is how many options do I have at any stage. So, if you look at tic-tac-toe and you look at the first move there are only 9 options, but if you look at chess there are 20 options and you know those are few who know chess you can calculate and in go there are 361 options right in the first move. So, this is what we call the branching factor and the branching factor the higher the branching factor the more difficult it is to pick from them right the more complex the model you will have to build to pick the right action among large number of action. So, there is this notion of complexity of knowledge and we can look at all these three to think about. So, coming back to knowledge. So, the idea of knowledge is obviously, if a third grader is solving this problem maybe he has nothing to do because he does not know what to do next, but maybe a fifth grader can apply more knowledge and and eighth grader can do it even better right. So, the idea is depending on your knowledge it can enumerate all the options, but how do we know which one is better and that is not the job of knowledge if you think about it that is the job of what we call learning. So, when we are told the rules of the game do we become a good player of the game? No right we still have to learn the next thing which is how do I make a move among the feasible moves in the current state unless I learn that the knower of the rules is not a grandmaster of the game right. So, how do we learn? So, what do you do in tic-tac-toe we play a lot of games sometimes we lose sometimes we win and then over a period of time we figure out that certain actions are better than other actions right. So, that is the idea of learning and this is what learning does learning only quantifies the goodness of a state or the goodness of an action even the current state on the go right. So, that is what learning does and the whole you know the theory around reinforcement learning and other kinds of learning is really centered around this idea how do we learn once we know the knowledge. So, knowledge only constraints my forward path and say these are the only things we can do learning really decides where I go among them and what is reasoning? So, I am going to give you a sense of what is reasoning? Reasoning really explores and uses knowledge and learning in a very interesting way. So, if I start with this my knowledge tells me that these are my three options it is the knowledge that tells me right, but learning if I solve many such problems before and I know which one is going to give the best solution quickly I will say this is better than these two and then I say ok now I am I am going to take this action and now what I am again back to a state again I have to do the same thing my knowledge tells me you can do these two my learning tells me you should do this one and my knowledge again tells me from here what are the options and my learning tells me this is a better option and again my knowledge tells me these are the three options and my learning tells me that this is only right. So, we understand the role of knowledge learning in the process of reasoning. So, reasoning is recursively using knowledge and learning in this process ok. So, now let me give you an architecture of this whole thing we talked about different things. So, the idea is you start with the current state anytime you start you are in a current state your knowledge is going to tell you what are the feasible states then what do we do we take each of the possible actions in our mind. So, think about how you learn to play chess you first take the action in your mind and then you decide am I better off or worse off now and then it basically scores all the actions if you will and now what can you do should I just pick the first action should I just pick the best action. So, it has given me a recommendation of action right and n number is decided by the knowledge which says these are the only actions you can take and this guy has ranked out at these actions should I just pick the first action some learning right. So, we have already learnt a bunch of things. So, I have played many games. So, I know how to play chess now, but should I always pick the first action no you have to look through the future. So, let us say all that knowledge after looking into the future has already been encoded in this right. So, it depends on who I play with who I want to win or lose it is not who I play with it is really about am I confident about this right. So, if I am a grand master and if this is my model then definitely I should do this right. So, the idea is really the question is have you explored enough now there are three kinds of things you can do three kinds of questions you can ask you can say given subject and predicate what is the object a question like this given subject and object what is the predicate right and the third question you can ask is the third one which is given predicate an object what is the subject right. So, we understand that factored question answering is now as simple as converting it into this language and creating answers from right. Again a very simple task now I want to go into two things which is how can we do similarity and reasoning on top of these. Now, we all understand word embedding right. So, we all know about word embedding then the basis of this is this notion that we shall know a word by the company it keeps. So, the idea is skip grants and all these models all they do is they look at a word and look at the context and say two words the same context they must be similar right. So, that is what has led to a lot of research in word embedding, but my hypothesis is that there is no such thing as language. So, how can you learn word embedding what does it even mean. So, let me question this a little bit and see what do these similarities actually mean. So, what happens when you learn a word embedding all the words in your dictionary are part of the same embedding space right. All the words are part of the same embedding space my question is is that the right thing to do right. So, let us look at some examples does it make sense to ask this question what is the similarity between Apple and each or Bangalore and likes or Google and search linguistically you can say Google is search and all that, but linguistically these are two different things. What about this Apple and red Bangalore and progressive Google is profitable. Does the notion of similarity even exist I am not even saying whether they are similar or not all I am asking is does it even mean something, but you could compute these if you did your word embedding right. I can come up with a number and say yeah this is the similarity between Apple and each because here is one vector is one vector and you can do a dot dot, but it does not mean anything right. So, there is something weird going on about embeddings in that way what about this Bangalore and target. Now both are nouns right dog and aspirin tree and Google right. What about Apple and orange what about Bangalore has above. So, which of these four is actually meaningful if you think about where we can compute similarity last one right. Now that is the idea behind how do you compute similarity using knowledge and not using text right. I do not want to get into that text and co-occurrence in this thing, but because it does not mean anything. So, I cannot measure similarity between verbs and nouns, nouns and adjectives that does not mean anything they have to belong to different spaces. Similarly, I cannot do similarity between entities of two different types even if both are nouns Bangalore is a noun target is a noun, but I still cannot do similarity because it does not mean anything they are entities of different types. So, it is not just about part of speech it is about entity type right and here we can do because all these are entities of the same type right. So, that is where the embedding models breaks down and therefore, you want to talk about knowledge embeddings which is how do we create embeddings from the knowledge graph directly because there is no such thing as lagging. So, the hypothesis I am going to go where this you do not have to have a single embedding for everything you have to have different embeddings for different things. So, if you are thinking in an e-commerce environment all my customers and all my products and all my reviews have to be in the same embedding no because these are three different types of entities think about different embeddings for each and now think about how would you translate your word to vector ideas into this kind of thing. So, you have different embeddings for people. So, it does not even allow you to do this similarity between this one ok. So, how do we compute this similarity how do we compute these embeddings right. So, why do we know that aspirin and ibuprofen is even comparable because they are entities of the same kind. But now I am only given knowledge graph I am not given text data. So, I cannot think about neighboring words and skipgram how do I create similarity between these two. So, what can I say is any ideas we do the same thing that we do with text except now the idea of context is different right. So, we say what does aspirin do aspirin is a node in a knowledge graph it has a relationship with some other entities this has a relationship with some other entities and because of this this is similar to this. So, the idea of context in a knowledge graph can be thought of as can you think about the similar context and then say these two are similar ok. And now I am going to propose an alternate way of looking at this whole thing which is you shall know an entity there is no such thing as a word right. So, that does not mean anything you shall know an entity by the company it keeps in the knowledge graph ok. So, that is the new definition of of what we can do in similarity. Let us make it a little harder how do we know that Isaac Asimov and Arthur C Clarke are similar order right. So, if I traverse the knowledge graph and say this guy wrote these books this guy wrote this book and then this guy's book has a genre of science fiction this guy has a genre of science fiction therefore, they are similar right. So, the idea of similarity now we understand and this process that we went through right going from entities to states to entities to states if you will is what we are calling reasoning right. So, in a way we are using reasoning to actually compute strategies ok. So, my favorite part how do we do analogies how do we think about you know this became very popular when when word word vectors came about and we did king queen versus men we men. So, let us see how we do it how we do analogies right. How do we do it in our mind what is thinking in this context how do we know what is the answer here right Steve Jobs is to Apple who is to Microsoft and if I say Barack Obama is to Varin Modi is to Varna now I do not know how many people know, but this is his birthplace right and if I know that then only if that edge and relationship is in my knowledge graph then I can solve this right, but the process of solving all these how do we solve this given a knowledge graph right. So, I do not have embeddings yet I am not doing differences between word embeddings because there is no system as a word. So, what do we do these are all entities they are related to each other and let us see how do we solve it in our word right. So, how do we do this what is going on in our mind right. So, we say what is the relationship between these two things right things and then we say ok we run a query on a knowledge graph and says what is the relationship or a predicate between this as subject this as object and it says founded right and then we say ok if I apply the same thing to this other predicate which says who founded because I got founded from here and then I put this guy here understand how it is very simple we do not have to apply large vector products and all that we just do a very simple two stage reasoning process to think about this. So, again knowledge movie predicate you say born in and then you get that all right. So, now let me talk about how do we create a new architecture of reasoning right. So, we have talked about deep learning and other things these are great architectures I will show you a very simple idea or how do we build a reasoning engine in that that is very conducive to all these things to this topic. So, imagine you have three networks one network is mapping subject object to predicate another network is mapping subject predicate to object and all we are doing is a look up on a knowledge graph right there is nothing special about these networks these are not even networks these are just look up tables on hash map and now let us see how do we solve that problem. So, if I put Steve Jobs as a subject and Apple as an object this network is going to tell me founded and then what happens I take this guy back into these predicates and then I put anything here I put Mark Zuckerberg here or he tells me Facebook and if I put Microsoft here it would be good right. So, the idea is this is how our associative reasoning kind of works which is it goes through these three networks and if I want to put a general structure imagine I have a set of active entities in my mind and a set of active predicates in my mind. If I take these active entities and put them in these networks nothing will happen well this guy will do something this guy will not do anything this guy will not do anything right. But when this guy does something it will give me 10 minutes right. So, then what I can do I can put it back here. So, now this guy are going to generate the red lines and entities and this guy is going to do this and I can do this because of that right. So, let me just summarize. So, we have talked about two kinds of reasoning one is the Markovian reasoning and we think it is different, but it is not really states and actions can be thought of as entities and predicates and the real question we have to ask is it is a problem of prediction problem or a reasoning problem ok. So, I will stop there and take any questions or. I have been watching artificial intelligence for about 40 years so probably most of you since most of you were alive and it has always felt almost mature according to its practitioners. When I was one I was just begun. When I was two I was nearly new. When I was three I was hardly me. When I was four I was not much more. When I was five I was just alive. But now I am six I am as clever as clever and I think I will be six forever and ever. I make one prediction that in the fifth elephant in 20 years time let us call it the 25th elephant all of this material will fit in the childhood stage and just as an example of kinds of reasoning you had the mathematical one of simple replacement books. As a mathematician that is not how I think. I look at x cubed minus y cubed I look at x minus y which I would like to divide by. I see that if x minus y is zero then x cubed minus y cubed is zero therefore that is a way of dividing it and I don't remember the formula like you said. I know there must be an answer so I do the division. The big thing about mathematics it saves your memory because there are systematic principles. I gave up physics because I had to learn too much let alone history. I could apply ideas and I don't think you can really claim that this scheme is yet capturing idea thinking. It's doing an amazing amount. I'm not dissing it but it's not adult yet. Yeah, that's it. Thank you. Hi, so there's a lot more. Excuse me. Yeah, just a quick question. How do you model induction or inference in your framework? So the Bayesian reasoning does that for you, right? So if you look at how the graphical models work they can go backward if you have to apply the Bayes theorem and keep going backward or you can go forward by applying it in a different direction. So different types of reasoning require different types of framework like he's saying mathematical reasoning needs even more than what we have talked about. Hi, so I'm just trying to draw parallels between prediction and reasoning, right? So in some sense prediction is basically a search through the hypothesis space. Reasoning is a search through the state space. No, no. So the reasoning is think of reasoning as so as a search in the hypothesis space except the hypothesis is a sequence of actions now. Earlier in prediction the hypothesis is one prediction. Now if you just transform it and say what is the path? And I say now obviously the problem has become more complex, right? Many paths. But really if you think in those hypothesis terms then the hypothesis is different. I'm not looking for one action but hopefully through a collection of actions the path will be correct and that is the hypothesis, not the action. So I mean is there a kind of way to kind of identify a certain problem as being a prediction problem versus a reasoning problem? Or is it just a modeling choice? No, no, it is not. For example, I mean taking the example of the knowledge graph, right? So people have tried to do kind of link prediction and all that, right? So versus you could do reasoning as you said. So is it a difference that reasoning is more kind of deterministic versus a link prediction being kind of a probabilistic approach? I mean there could be statistical reasons for saying is this link should be there or not? Is it consistent with the rest of the knowledge or not? So you have all those, those are prediction problems. Whether the link should be there is a prediction problem. But how to use a collection of links across the graph is not a prediction problem because you are traversing a path if you like on a map. So I'll give you an example. If you want to think about predicting churn of one customer that's a prediction problem. But if you want to optimize the lifetime of a customer for yourself through a series of actions, it's a reasoning problem. So it's not a modeling choice. We only think well when we think of single actions. But we just need to now think that nothing is a single action. It all depends on the sequence. So even other notions of action. So I'll take more, we can just... We have time for one more question. Hi, sir. Excellent talk as always. So when you talk about associative reasoning, right? So you're actually reasoning within a closed state space. So where do you put the bounds on how much of reasoning do you need to go into? For example, depth. So a classic example is when we are surfing Wikipedia, we get lost in a sea of information. So what's the depth to which we go to when we do? Yeah, yeah. So again, there's a very big trade-off in AI, which is how long do you need to think about predicting the next action? And if you can think long enough, like if I sit for 200 days and think about my first move, I can make the optimal move. But then, you know, I won't be able to play the game. So that trade-off about the quality of solution you get and the amount of exploration you have to do is always there. And we need to find the balance between these two. So my just follow-up question is how do you do this exploration in an unknown space? So the big assumption here is the knowledge is complete. It's a closed world of that. So that is somebody else's problem. Thanks. Alright, thank you guys.