 So, I will get started, I think two of us are tag-taming, I do not know where Siddharth went, can you just say, so I think two of us are tag-taming and we are from Flipkart, both of us and the purpose of this talk is to sort of give you, so the first I will spend not more than five minutes giving you sort of an overview of what we do in analytics and decision sciences at Flipkart and Siddharth is one of the scientists at Flipkart and he is going to take one use case. So, I think we give a very brief generic overview and one use case related to price elasticity and we have done some work around price elasticity and price intelligence more importantly that we will talk about. So, that is the context. So, just to give you why this problem is important and see why it is a different problem, right? Elasticity is a fairly straightforward problem in economics literature solved a lot, means in very sophisticated ways by multiple people in econometrics and statistics and so on so forth. But what makes it different for a business like Flipkart is fundamentally the scale. So, the next four or five minutes I want to spend around that, right? So, we have around 80 million products, around 150 million I think around registered users. This is a little bit dated, it is probably six months dated and the rate at which we grow this number keeps, means I cannot update this slide fast enough. So, I think that is the scale at which. So, the point is you think about the scale of the problem, right? We have to look at pricing of every one of these products on a daily basis and account for multiple things like self-price elasticity, cannibalization because you have 80 million products, many of these are cannibalizing other products so on so forth. You look at whether there is a sale event happening, whether a sale event is not happening, the browse behavior of about 100 million plus users and then being able to understand a competitive position, all of these things, right? And so, and doing all this and being able to update price on a near real time basis and being able to, then there are multiple guard rails around making sure there is a price trust that is established with the consumer. Every day you want to, don't want to be saying different prices for different products. You bought something today, tomorrow it gets cheaper, you bought it on a discount, you don't feel too good. So, there are multiple components to solve this problem at scale, to build price intelligence at scale. And that was the thing that I wanted to communicate with the slide more than anything else is that it's not a very, very sort of well isolated simple problem of price elasticity, price intelligence when we talk about large e-commerce sites. And obviously, we are a large platform, one of the largest platforms in the world. We have a lot of, so these are all the kind of signals that are used for driving sort of price intelligence, right? Understanding consumer behavior as a time series, understanding the impact of seasonality, impact of things like events like a big billion day and so on and so forth and other events, looking at browse paths, browse history, cannibalization, competitive position, all of these things in a scalable manner, right? And now just going through what we do as an organization, right? This is one use case that we are going to talk about, which is price intelligence. Broadly, our goal as an organization, right, is to sort of move decisions. See, all our decisions are anyway driven by data for any e-commerce, internet company. However, there is a mix of what is done by machine, what is done by humans. I think the goal is to keep moving that from human decisions to machine driven decisions. That's the larger sort of this thing, right? So this kind of gives you a view on what are some of the different kinds of problems we are solving at Flipkart today. Basically, we have a data platform where literally whatever product you build, you make sure that the data from that gets ingested by our data platform so that you have one repository of data. And then you have multiple sort of solutions. So we talk about insights, but insights are essentially signals, right? These are models which build a signal that can be consumed by various products in the organization, some of, or for human decisions, right? There are sort of product insights. Elasticity can be a product insight at a SKU level and that gets, so that's the model that Sid is going to talk about. And then that can be sort of consumed by our data platform. And that signal can be used for many different things. It can be used by other products. It can be used by humans, like business category leaders to make decisions and so on and so forth, right? Similarly, there are user insights which can again be consumed by multiple different traffic insights, pricing, supply chain insights, and that's what you see at bottom gives a very small sample of some of the use cases that sort of have been built and that are getting consumed by various systems in Flipkart. So now I'm going to sort of stop here and invite Sid to the stage so that, and he's going to talk about one use case which is the price intelligence. Thank you, Ravi. Hey guys, my name is Sid. I work as senior decision staff decision scientist with the analytics team in Flipkart. And so just deal with one of the particular problems that we work on, right, which is price intelligence. So essentially what is price intelligence? So the idea of price intelligence is very simple, right? The idea is that you can use elasticity, which is basically the percentage change in demand divided by the percentage change in price. The effects, rather the trend in elasticity can be used to our advantage. So to speak that there are certain regions of price where your elasticity curve or demand curve actually changes drastically and certain regions where it does not change drastically. And so for example, let me point out to you. So you see this curve right here. This is a typical elasticity curve where, forgive the lack of access here, but this is basically increasing price and this is increasing demand on the y-axis. So you see that as you keep decreasing, rather increasing price, you see that there is a sharp decrease here and then suddenly it flattens out. So it's like a real sigmoid curve that you can see in a lot of places in real life, right? So let's fast forward to basically this curve because this is basically what I'm going to be kind of using to explain, right? So essentially here you see that you have an increasing price on the x-axis. As you keep increasing price, your demand drops. So at some point, essentially you have your total revenue kind of increasing and then start decreasing again after that. So essentially that means that there is some sort of optimal price point where you can place a particular product such that you can essentially get the maximum possible margin out of it, maximum possible demand out of it and therefore maximize your revenue, right? But there is one further complication, right? Which is this. Cannibalization, which is a big problem, right? So you say you have a particular category, right? And you have a whole set of, you have a whole gamut of products, right? So what happens is that when you decrease the price on one particular product, then the demand for that particular product increases, but the demand for the others which actually are cannibalized by this product also decreases. So what you need to do is essentially come up with a pricing vector for all of the products in your entire, so let's say, category, right? Which such that you are maximizing some sort of measure sale, you are maximizing margin, maximizing revenue, maximizing units overall, but essentially it boils down to the same problem which is that how do you account for the effect of cross elasticity? So to speak that the price change of one particular product affecting the demand of another product itself, right? So once you do that, so essentially the whole goal of this is to come up with a set of optimal prices such that I maximize an objective function, right? So with that, let's basically go on to the next slide. So the way that we go forward with solving this problem, right? Is we look at the product sales, essentially we try to model the demand as a function of time. Now the demand has a lot of pieces. Essentially we have a whole bunch of signals basically affecting the demand which I'll come to in the following slide. Essentially having all these signals basically large number of signals becomes a feature reduction problem, feature selection problem and essentially a forecasting problem, so it's like app for machine learning, right? So we use a random forest with lag variables or a support vector machines or a unobserved component model or something like that. Any model which basically can suitably sample the posterior distribution of the demand itself depending on what it is. Once we do that, what we do is we filter products based on model accuracy and we say that because we won't find like infinite model accuracy everywhere you go. Essentially it's not going to be always 100% it's going to be like 50% 60% and so on. We don't want to be modeling products which have very poor model accuracy and explicability, so we don't use them. And also the other thing is that price has to be a significant variable. So either the price of the product itself or the price of other products affecting the particular demand has to be a significant variable. Otherwise basically there's no point of actually taking it as part of the elasticity optimization itself. The next step is basically we say we have the demand curves for all of the products across the entire category. Now what we want to do is essentially say what are the set of prices such that all of the demands are such that we optimize a particular objective function. So essentially the objective function can be something like we maximize the total monetary value that comes out of the entire category. Or it could be that maximize the margin which comes out of the entire category subject to some constraints. So this is a problem which is definitely not convex. So essentially you're looking for something which is like a stochastic optimization of starts. So I will come to one more complexity in this, but we generally end up using a genetic algorithm to solve a problem like this. The other complexity is that there is a demand stochasticity, meaning which when you predict a certain demand it has a natural noise which comes along with it. So essentially you have to account for the noise when you're accounting for your overall fitness function that you're trying to optimize as well. And the second thing is that you also need something like your model accuracy is not 100%. So you've got to account for these two. There is demands stochasticity. There is model error which you'll have to account for as part of your genetic algorithm itself. And this process is called stochastic optimization. So that is kind of the next step that we do along with the genetic algorithm. So we factor that as part of the fitness function as well. And finally we spit out a whole bunch of product recommendations. So the demand response function you take this as with a pinch of salt because this is actually a kind of a, let's say a simple representation of what the sales would look like. So essentially let's say that sales actually represents a particular product sales in a particular week, right, in a particular category. And essentially what happens is that you're looking at something like this is the lagged sales of week one. So it's kind of an autoregressive model here based on the demands for the previous two weeks. This is a display effect essentially how frequently does something turn up in a search when you put it on a e-commerce website. So what is the impression share or something like that? There's an availability effect. What happens when something goes out of stock? So essentially something else takes it up. There is cannibalization by something else. So therefore that's definitely going to result in a lower demand in the future weeks as well. And then it's going to go back up later because this is autoregressive anyway, right? And then there's a self-price effect. And there are few other terms here at the bottom which is basically similarity based cannibalization. So essentially there are similar set of products which you can identify either based on browse behavior. So essentially somebody actually looks at two different products in the same session, for example, that's kind of co-brows. There is price-based cannibalization, essentially products which belong to the same price category, so to speak. And finally there is competitive price cannibalization. So you have competition from other companies, other online platforms and so on. So you account for that as well. So factors affecting demand. So this is a very comprehensive slide of all the possible factors that, you know, could potentially affect the demand. I would say this is necessary but not sufficient. So take this with a, again, this is probably very comprehensive but not complete. So then you have product quality related attributes. You have display data related attributes. Sales data related attributes. So essentially, meaning up here, you have other factors like inventory styles, age of listing, how old a listing, particular listing is. You have offer data. You're basically saying that something is going on offer. Essentially, is it something that we are putting out there with more frequencies at a greater discount? Is it getting more traffic? And then there's a cannibalization module. Essentially, it goes into the model. So this is kind of all of the factors which kind of subsume anything which could potentially affect your demand for any particular product. So the demand prediction model itself, this is more of a summary of how we combine it, right? So essentially what we do is we take a particular category. We have a particular product which is called the focus product, right? We have the focus product here which is the product we are trying to predict the particular demand for. We have all these products which basically can potentially cannibalize from this particular product. So essentially what we do is then take the sales data at t minus one. So essentially it's kind of an autoregressive model at this point, right? And then we take the ratings, reviews, competitive views. So essentially all of those things which can potentially affect how the demand works. Then the cannibalization at time t minus one. So when I say time t minus one, I also mean time t minus n. So essentially take that to be an nth order autoregressive function, right? Cannibalization based on co-brows, based on correlations between demand and product price. One thing you should just remember is that cannibalization always is demand of something with respect to the price of something else. So essentially demand versus demand is demand versus price, right? So then we actually model the overall demand over a week and then essentially spirit out so that they can do the price optimization. So essentially this is kind of the overall process of how we do this, right? It's a very technical process. I can just go by it very quickly. So essentially we do the data cleaning transformation, very standard stuff. We do a rolling test and train test just like we do for any kind of long time series where you have multiple change points. Essentially it's very important to not take the entire time series to do the train and test, but have a rolling train test. We do a feature selection using rank correlations or mutual information. So sometimes we have categorical variables. So we need to use mutual information in those cases and we actually do something called a adaptive feature selection using a Markov chain Monte Carlo where we adaptively select how correlated features can be with a particular objective function as opposed to within themselves, right? So because we don't want too much correlation between or rather we don't want multicollinearity. We evaluate all possible hyperparameter combinations. So essentially we choose the model itself adaptively and the hyperparameters adaptively and the features adaptively. So all of them kind of are baked into the MCMC. We do a forward-looking cross-validation. So essentially we obviously can't use future information because this is a forward-looking time series. And then finally we spit out the final settings and train the final model and then we use that, put that on an ML platform and then we score. We give out the prices and say these are the prices you need to go forward with this particular week. So essentially once basically we have each model, right? So just expanding on the last point, we have once we have each model, essentially what we do is we feed it to a GA. So essentially that each price that you give out for or rather each price vector that you give out for a set of products is fed into the GA which essentially does one thing which figures out what the demands for each of the products is going to be for that particular price point and says okay here is your objective function value. So you can imagine that if you have a particular objective function you're trying to optimize all you need to do is just keep iterating over this vector, figure out what the prices are going to be, right? So this is just a standard GA algorithm that you have here. It's nothing new. This is quite vanilla. So let me go on to this slide basically where we show you what the optimization strategy is, right? So essentially what we're trying to do here is we have different types of products. We have low elasticity, high margin products, high elasticity, high margin products and high elasticity, low margin, low elasticity and low margin, right? So the general optimization strategy is that when you have a high elasticity and high margin you can actually change your price by a little bit and get a lot of demand goodness out of it. So whenever you have high elasticity that's what you want to do. And whenever you have very low elasticity but high margin so essentially you want to basically get more margin goodness because you know that changing the price by a bit is not going to really do anything to the demand so you might as well get some margin goodness out of it. So this is very simple. So this table right here is just showing you evidence for the fact that the algorithm actually works in one of the categories that we've implemented it in. So this is something that we are going to do full time and actually expand across all our entire gamut products very soon. And that's about it. Thank you very much for this opportunity.