 So, I made a last minute change about how I am going to address the topic for today. I am going to start by telling you a story, hopefully that the story will have a lesson and you might find it amusing and perhaps wake up. And I am going to end by telling you even a better story and between the beginning and the end it will be all full of stories except for a small piece where I would like you to pay attention because I do want to take you behind the scenes to show you each aspect of solving real world decision problems. As you know every business big or small has to make decisions. Some of these decisions have to be made in real time, some impact today and tomorrow and other decisions might impact a much much longer period of time. So, let us start as I said by telling a story and the story is about a priest and a truck driver or sorry a bus driver in Bangalore. They happen to die at the same time and both of them reached the gates of heaven. So, the God looks at the priest and says well I like what you have done and assigns him a run of the house room in heaven. The driver he was apparently drunk had been could hardly walk and was about to fall and God assigned him to the palace the presidential suite in heaven. So, obviously the priest was very disturbed he said to God I have been preaching your gospel all my life and this man that you are giving the palace or the presidential suite room to has been a drunk abuse drugs has been mean to people has never ever done anything they are to be proud of. So, God looks at the situation and says unfortunately it is all about results when you were preaching the gospel most people were yawning as some of you might be and they were falling asleep nobody was remembering me but when this bus driver half drunk was driving on the roads of Bangalore everybody on that bus was folding their hand and remembering me God save my life. So, indeed it is all about results and when it comes to solving very difficult business problems business decision problems it is about the results. Business people do not care whether you use machine learning or deep learning whether you use operations research techniques or optimization techniques or what you have to produce results that is important. So, I will go back to my favorite scientist what happened there you go once he was asked once he was asked if you were given an hour to solve a problem how will you go about it and he said I would spend 55 minutes defining the problem and only 5 minutes in solving it and that is so true when it comes to business problems how we need to solve them we really need to spend a lot of time in obtaining the right model before we attempt to solve that model. Now, that part of Einstein we do agree with however Einstein also said that God does not play dice with the universe unfortunately that part is not true every business that operates operates in an uncertain environment and you are almost sure that whatever you predict will not happen and dealing with that uncertainty and making sure that you are making decisions that are not only going to optimize what you want to optimize but will also cope with the uncertain nature of the demand in the marketplace becomes one of the key factors to be folded into solving these business problems. The challenge really is to create the right model and if you do not create the right model and you pretend that your predictions will come true or you pretend that you are operating in a certain environment it can be very dangerous it can not only leave a lot of money on the table in fact it can lead you to a situation which is highly suboptimal remember that in the beginning we said it is all about results and in order to get the right results in the business environment we have to optimize whatever we are optimizing but we have to make sure that we account for the uncertain nature of the demand and the marketplace so in the beginning I told you a story time for the next story and this story may be true Prashant you remember when I asked you to light a bulb a thousand watt bulb in one corner of the room and I gave him the task of predicting the temperature here and Prashant said to me that's easy he didn't follow Einstein's style he didn't think much he didn't think about modeling he says very easy what I just need to know the position and velocity of the molecules in the room I write those equations I learned before and those equations will decide how these molecules collide and collisions I know produce heat and I will do all that and very soon I would have solved the problem but how great so he bought all the compute power but he needed to call this humongous amount of data and while the machine was churning somebody opened the door and walked in and Prashant says hey you didn't tell me that somebody was going to open the door all my calculations you have to recompute and please before I recompute I wanted to look at what might happen between now and the time I solve the problem you know it's a difficult problem I need a lot of compute power so that I can account for it otherwise I would have wasted my time oh good so I called Prashant I said Prashant it's not up to you let me see if I can get help from someone else and another person comes and says oh I don't need any computers I just need an envelope I'll model the problem that the temperature varies as inversely as the square of the distance from the source and I cannot forecast the impact of lighting that bulb here very very easily the real world problem remains the same but there was one way of modeling the real world which resulted in a solution that was very fragile it couldn't cope with any disturbance used a lot of data and somehow people believe that the more the merrier but is that really true I love big data but I love relevant data even more I love complexity but I love making things as simple as possible but not simpler more and guess who said that you should try to make everything as simple as possible but not simpler Einstein I have a lot of respect for that so moving on all of you I'm pretty certain of familiar with the tower of Hanoi so before I go there I want to ask everybody here I'm told it's an Indian story a king who invented the chess board is that place one grain of salt or one grain of rice on the first one two grains of the other one and fill my chess board because he was very happy that this man had invented the game of chess and the king enjoyed playing it so the king says you can ask for my daughter's hand in marriage I'll give you half the kingdom what the hell do you want this nonsense for just some amount of rice actually it turns out that the amount of rice to fill that chess board exceeds will exceed the production of rice if the entire earth surface was the rice cup tower of Hanoi you have to it is said the table says that God had 64 this place these are golden discs on a rod of diamond I guess and the job of the priest is to transfer those from one of those towers to another tower and the constraint is that you cannot place a larger disc on a smaller one so if you figure out the strategy you will soon realize that it would take one move to move the first two moves for the next one for the next one and so on so how long let's say the priest is very fast and he can do a move every second how long would it take him to complete this task any guess it well how many billion years it will take if my calculated correctly I think 58 thousand billion years and 58 thousand billion years and according to other sources people who have tried to you know measure or expectation of how long the universe will last it's approximately 20 thousand billion years it came very close I don't know about the story but there are certain problems which have this problem that in order to solve them as the size of the problem increases it start to take exponential amount of time your traveling salesman problem as you all know is NP complete and the complexity theory paradigm tells us that the general problem solving paradigm is not going to be effective in solving problems that are NP hard or and so on so how can one deal with this interact ability in a practical word you know if a traveling salesman has to start from a city visit each city once and come back to where he started from and you want to do that by minimizing the total distance travel that really is not a real word problem because what you might be interested in is minimizing the overall time it takes for him to service to go through that route in any case if you have to solve this problem you know that if the number of cities is too large your problem your the general method of solving this problem is not going to work problem salesman problem is NP hard or NP complete I forget but in any case you might want to figure out that you want to divide the cities into clusters small enough that you can efficiently solve the traveling salesman problem within those clusters once you have gotten that then you consider those clusters as nodes and see how you can minimize the distance and in the real world whenever you have to do it you are really going to find out the trunk routes of how to go from here to there and separately solve the problem of how to go to that trunk point from various places so the real word usually has a structure and the general problem solving paradigm throws away that structure and does not take advantage of it so if you can find a way where you can try to simplify the problem by exploiting the structure of the real world and simplify to a point where you can actually be able to solve it in real time and be able to hope with the uncertain nature of the demand in the marketplace then that is what may lead you to a solution to a practical problem in a meaningful manner and remember that a slightly suboptimal solution found in time may be much better than an optimal solution too late we are dealing with the business world and therefore we have to be very practical in how we approach these problems so everything has to do well I have never given a talk in India so this I chose my favorite example because I am aware of an individual that went from India he was from a village his dad used to have a farm where he kept cows and sold milk and this guy went to Cornell got his I think he went to Carnegieville and I was so he went to Cornell and got his education in operations research non-linear linear programming integer programming we even learned machine learning deep learning and all of that and then he came back to his village and he looked at what his father was feeding the cow and said this is very expensive have you ever tried to optimize the father was reluctant to allow him to but said well perhaps all this education is not a waste let's see what he will do so this man this young kid went and said here are all the foods that are available for the cow and for each item what are the minerals and vitamins you know per kilogram and then he set up an optimization problem of say you know x1 of food 1 x2 of food 2 you know the rest minimize the total cost subject to the constraint and the constraints were that all the mineral and vitamin needs of the cow are satisfied he found that the solution would cost one third of what his father was spending and feeding the cow he was very thrilled and went to the dad his dad and said dad I am not only going to save you a lot of money I am going to increase the milk production it might even double why because all the mineral and vitamin needs of the cow will now be met I have done this study and I have this optimization done so father reluctantly agreed because he believed that the best way to teach someone a lesson is to let them learn from their own experience my father did that to me and I am sure that I am doing it to my children as well yeah experiences there is no substitute to experience well that reminds me of another story there is sometimes remember Einstein said imagination is more powerful than experience because your experience is limited to what you know and what you understand whereas imagination has no such limits imagination includes what you do not know today but mainly know tomorrow it also includes things that you have that have not been a part of your experience and can become a part of your experience later but then I am going on a tangent the real thing was that instead of increasing the milk production the cost started to look very sad and gradually the milk production actually dropped to a point where the cost of giving milk how could that have happened the cow did not like the taste of the food and my friends mathematical optimality is very different from business optimality now we do work with the hotels and airlines and we provide them with pricing solutions so we can solve a very complex NP complete problem by using tricks of the trade by making sure that we do not assume we know all we know that it is uncertain we also know that we can model things differently things that are important will be modeled in a lot more detail and things that are not so important may be modeled very in a very cursory or ignored totally and doing all that if we produce a price well what happens if the customer does not perceive that to be fair or if he feels that the airline or the hotel is gouging them all your mathematical optimality may go down the drain and you may find yourself in deep deep trouble so yes mathematical optimality is important but then you are really solving a real world problem there are things that you have to take into account which people may be ignoring and that should not or could be ignored so essentially then so far what we have said is that there is a real world out there and we really want to control that real world or make choices and decisions regarding that real world we have to capture that real world through some kind of an abstraction now once we have a model of the real world we can apply any technique to solve the problem and we use the output of our optimization to control the real world so it becomes a legitimate question to ask if I find that it takes me forever to solve the model or if I find that it is too complex could it be just could it be that this complexity belongs not in the real world but in the way I have chosen to model the real world let me give you an example remember we talked about the travelling salesman problem and we said what that a general travelling salesman problem cannot be solved if I give you a billion cities how many of you think you will be able to produce an answer that is optimal for me by tomorrow morning you can use all computers you have I don't care no worry but if this the cities were around the lake and idiot like me can solve the problem in no time just go around the lake now you take a problem where the cities are around the lake and you feed it to the general purpose technique of solving the problem the problem will still be intractable you will not be able to solve it whereas somebody who notices that structure and can explore that structure would have produced the answer in no time so my contention is that when you are dealing with a real world if you look hard enough you will find structures that you can use to simplify the problem and you should be able to make things tractable and solve them to their optimality the difficulties associated with the real world are if you look at how let's take the example of airlines and airline has to planes of different sizes they have to publish a schedule and they have to assign capacity to the routes I am going to this flight which goes from this city to that city connects so on and so forth the hub that these are the planes size of the plane that I will put on that route there is a separate department other than the capacity planning department that has to price each of these flights now imagine that these people are going to solve the problem that they have to they are solving in their own silo so the capacity planning guy looks out the window perhaps he has a telescope I don't know but he sees this flight always goes full he sees this flight always goes full without a system what do you think is thought next time if I get a chance then I get a chance to re-plan capacity I am going to do that put a larger plane on that route unfortunately the pricing guy is also looking at the same flight and sees this flight is always going full so he says once I get a chance to optimize what I am going to do re-optimize I am going to increase the fare both of them have done solve the problem in their silo to optimality and in fact their decisions are right unfortunately for the airline though they are now landing up in a situation where they are increasing capacity on a flight where prices are being raised now does that make sense to anyone what went wrong and that is what goes wrong when we try to solve the problem by dividing it into solvable pieces and we decompose the problem now decomposing a problem into solvable pieces and optimizing in silo can be very dangerous to the health of the organization yet the overall problem solving it is not possible because of the computational complexity and especially when the problem that we are solving sometimes may need solutions in real time so what choices do we have what can be done well every time you take a problem and you break it into sub problems you increase the total amount of work because the sum of those pieces will not have to the complexity of the original problem there is some decomposition penalty that you pay so clearly when you do this you have to do it in a way that you minimize that decomposition penalty also now once you are able to solve these problems in their own silo you know that you are not going to be able to reach global optimality for the overall solution so what you have to do is find a technique that allows you to partition the problem in a manner and solve it in a manner that global optimality is preserved which means that perhaps what I can do is once I solve those problems in their own silos I use the current solution to do something different and re-optimize and re-optimize and I have to do it in a manner that when I converge I would have converged to if I could solve the total problem whatever the optimal answer was I am able to achieve it there lies the science the art of solving these very difficult problems so yes we need global solutions and yes if we partition those problems and only optimize in silos we are not going to be able to reach the global optimality therefore we have to learn from solving these problems to partition the problem again in a manner that allows me to preserve global optimality now for all business decision problems we need ways to solve them quote unquote optimally and these are of great great value and we have one part of it discussed so far that is dealing with the complexity let us not forget that there is another part that says we also cannot and should not assume that we can predict because we know whatever you predict is not going to happen think of forecasting for a minute and my claim is that people including academicians as well as vendors who work in this area are not really getting to the crux of the matter and the crux of the matter is this how do we measure forecast accuracy every time I have looked into it people talk about after the fact what was the forecast you may use any method you want including AI techniques neural networks for forecasting machine learning I do not care ultimately to judge the accuracy of your forecast what people are using is what happened after the fact compared to what they forecasted to happen and that difference some square of that or some function of that is a measure now I can give you a forecasting problem where your job my I am going to throw away throw 100 coins and I want you to forecast the number of ends the correct answer is 50 there is no error however when you do toss those 100 coins sometimes 40 sometimes 60 but according to my measure of forecast error I would see that no different from before but the real word consists of both it has patterns it has intrinsic volatility so how will I tell well if I if the error is due to a pattern that I am missing looking at the problem more carefully and working harder will create a better answer on the other hand if this is intrinsic volatility I will be wasting my time I can spend instant time trying to improve my forecasting accuracy but it has got nothing to do with it it is the natural volatility and unless if the forecasting measure itself cannot distinguish between the two how well there is an easy answer the answer is it is never enough to forecast a number we also need to forecast the uncertainty around it and for those of you who are not done a lot of forecasting can think of real data consisting of patterns and some noise added to it so try to think about it as excluding you know removing these patterns from it by some means so these patterns apparently you are going to say rely on them repeating and therefore you will use the pattern to produce a point forecast hey after you removed all the patterns this garbage that was left which is pure noise and there are mathematical ways that you can tell this is pure noise by looking at it Fourier transform and seeing the energy spread all over so what do you do to this point forecast you need to say this is prediction and scientists don't believe in predicting to convert that prediction into a forecast I need to take that noise and add it here and say all I can tell you is it will be within this region and when I am making decisions I cannot pretend that I am only going to look at the point forecast and forget about what yeah because if you do that or if you do linear programming or if you do ignore uncertainty and work with the averages as sometimes people do I would suggest that you keep your head in an oven and your feet in ice cold water perhaps in Minneapolis when it is snowing outside and on the average you should be comfortable oh but that is how we are solving many many problems today there we are using deterministic techniques and we start to believe in predictions so for me if somebody is going to throw a die I am not going to predict what will come out what the outcome will be gamblers do the only claim I would make is there are three equally or six equally likely outcomes that's all I can say and so let's move on and talk about what we actually do we work in the area of review management which again is decision making and here is where I would like to take you I will try to explain the real word issues in a manner that you will be able to should be able to relate to it remember here that we are defining it to be a rational and disciplined decision making process for maximizing company wide profits not profit not maximizing things in silos but while managing risk under current and anticipated market conditions now let's start with a hotel and I will show you how people that love data can go wrong people that believe data tells the whole story and the whole truth and nothing but the truth can sometimes make blunders so imagine that you have a hotel which has 100 rooms and the demand for this hotel if this hotel is always full as you see if this hotel was always full you will find in the data are sold out hotel but the decisions you need to make when the demand is 150 are very different from the decisions you need to make if the demand is 1000 we all agree that if the demand for this hotel was 1000 we will make very different pricing decisions than if the demand was barely 150 but in both these situations my books are going to show me that I have a sold out hotel so how will I ever be able to forecast a true demand then that true demand data is not in your history and if you treat the data that is in your history I can tell you the kind of problems that you can run into let's take another example in this hotel if people are managing it I don't care they are managing it using systems through machine learning and deep learning through operations research techniques to whatever or from their gut no matter how they are doing it when the demand is very very very strong what will typically hotels do raise their prices despite the fact that their prices are large they will still have a high utilization of their capacity a high occupancy what will hotels do when the demand is very very weak they will reduce the prices and despite the fact that they have dropped their prices the utilization may still be very poor so if I want to understand the price sensitivity in order to in order to max optimize the price I need to understand the relationship between demand and price and computer scientist and data scientist tell me go look at the data and the data will tell you the whole story so I go to the hotel and I look prices and I plot the number of souls across those prices and I find the larger the price the more people buy I wish that was the last city you know we will all be done and for those who still don't get it let me put it this way get hold of a good forecaster forecast is a number until I talked about it we were not wondering about do we really have to understand so let's play the stock market very simple buy low sell high so why are we sitting here this afternoon and why aren't we all billionaires very simple you have data scientist you have you know a lot about just do a good forecast and buy low sell high what makes that problem more difficult or impossible is the fact that you do not know when it will be low or when it will be high there is an ambiguity there is a uncertainty associated with it the same is true for almost all business problems we deal with including pricing and review management that is we don't know with certainty what is going to happen and therefore then we are deciding suddenly if you took the volatility and then decided what is my expectation of playing the stock market you will come to the right conclusion and say well unless you know inside trading or unless you want some illegal means maybe it is not the best thing to put money in that in the hope to make a large amount of money we cannot in the real world ignore and this point is so important that I will never get tired of saying it and it should never be forgotten so I am going to illustrate this by means of am I running out of time almost so since I want to leave time for questions I will just talk about two more things one I want to really show you how decision change with uncertainty suppose you have these bottles of water that you are going to sell and you know that if you sold them at hundred rupees or hundred dollars the demand will be exactly fifty not one less not one more it won't be forty nine it won't be forty eight it won't be fifty one it won't be fifty two it will be exactly fifty at the price of hundred and you have a hundred and fifty of these bottles and I come to you and say you know what I can sell as many bottles as you want I will not cannibalize your market I will go to no one where you are selling I will go and sell them in the US I have some connections and I will sell them for seventy five dollars how many of those bottles will you give me hundred fifty you will keep because you make more money from them so the forecast of demand remains the same fifty but at the price of fifty the forecast is fifty it is fifty plus minus ten it could be forty it could be sixty it could be forty five it could be fifty five now how many bottles of water will you give to sell at seventy five well you know that if you now kept the fifty at bottle how you will only be able to sell it when the demand exceeds capacity when the demand that means you will be able to sell the fiftieth bottle if the demand was fifty one or more not if it was forty nine forty eight or less assuming symmetry it says that you will only have half a chance of selling your fiftieth bottle out of hundred days on fifty days you will be able to sell it not on the other fifty so that is the point you see in the center and then without going into the math of it what happens is that if you look at the blue curve that is a curve for fifty plus minus ten and now you see where it intersects seventy five is a number lower than fifty so you are now giving me more bottles to sell because you are maximizing your prop revenue while coping with the risk associated with that uncertainty how do you cope with it by allowing more to be sold at a cheaper price as your increases if you now increase the uncertainty from fifty plus minus ten to fifty plus minus two look at the curve in black and now the point of indifference intersection if you look at it is even fewer that you keep very interesting and if your prices were on the other hand the next price I was going to sell it at was twenty five instead of seventy five oh then you can do the same arithmetic and now you will be giving me you will be keeping more so it depends upon whether the price next price is below fifty or above fifty so clearly uncertainty changes the decisions we made okay I won't have time to go into more detail so let me go back to let me just go to the last piece where what are things that we need to remember think think before you solve think globally optimize locally data only tells a part of the story I am being kind sometimes data lies and please never ever ignore uncertainty and common sense so I said I will end with the story so guys listen very carefully the story is about my favorite scientist Einstein who happened to be traveling with an Indian businessman on an international fight of seventy in ours and Einstein brain was always working he couldn't sit idle so he proposes how many of you heard the story a few of you have okay so indulge me and maybe if I you could course correct me if I happen to be telling not the right one so he says well let's play a game the game is do you ask me a question and if I can't answer it I will give you a hundred dollars I will ask you a question if you can't answer it you give me a hundred dollars that's the well this is not fair you are a scientist you know a lot you are very intelligent this is not the right odd so Einstein said okay what all right if I can't answer your question I will give you a thousand dollars and if you can't answer my question you only give me ten dollars the guy says done so Einstein asked him a question about the universe or something the guy had no clue about he simply took out ten dollars and gave it to Einstein then he proposes turn and he says Einstein tell me what is that animal that goes up the mountain on four legs but comes back on three Einstein scratched his head scratched his head couldn't figure it out finally he gave up and said okay before we proceed this is driving me nuts what is it that that goes up the mountain four legs and come down in three this is Indian business guy takes another ten dollars and says to Einstein tell so never mess with revenue management because what we call revenue management is the airlines pioneered and which hotels are now doing in fact every more and more people are getting into this field this field even today is using mathematical techniques that belong to operations research or we are not using game theory and we are not using machine learning and this has got to change because game theory and machine learning are natural fits which have not been exploited to solve this problem and remember that whatever people may say in India people have been doing revenue management for a long time in fact my grandfather told me a story which was and this is the last story that you are going to go and buy something for me oh the story okay whatever price he quotes immediately half it I went okay hundred dollars no fifty fifty no no no twenty five finally the man got so irritated that he said okay come on hell with you just take it free no I want two of them India is far ahead when it comes to revenue management thank you for your attention