 The first is getting customer address data. So address data can be available in different forms. As you know, the data, the address itself in India is not structured. All across India, you have different ways of writing addresses. And there is no set format that you can put even if you're collecting address in the most efficient manner that you think. So the challenge becomes more, I would say complex for third party logistics company because then you're dealing with data, address data collected from different clients who have different ways of collecting the addresses. And then, so the first, this challenge can be met by, you know, putting a logic where you have all the customer address data from the different clients in one format and that should be just text. And with whatever the, there are occurrences of a particular client, you can remove it and then put all the addresses in one place and that becomes your corpus of the customer address data. Second step is address structuring, which is I would say the heart of it. If you have it done right, you have the automated in the run sheet right. And why is there, and none of the algorithm, let me tell you will work with the efficiency that you expect it to work for if you don't have structured data, structured customer address data. Why do we need this structuring of the address data? As you know, there is no constant format of the address as I mentioned before also. There is no spelling check that is put when you're inputting an address. It's a free flow text in most of the cases. And there are combined words because customer is inputting, he or she can write anything. They can combine the words, they can write things which are not meaningful. We had instances of occurrences of addresses which said in the address field, you say, we got to see that you cannot find me, just call me on this number. So you can have addresses like that, which is not any address information. So you can just leave it. Then there could be abbreviations, right? Few abbreviations which are needed, few abbreviations which are just customers, you know, way of writing it. So how do you take care of all these things? This is very much required if you are trying a machine, learn something from an unstructured data that to address data. Now, how do we do the address structuring? Now that we know all the problems that exist, the first thing that we will talk about is basically creating a vocabulary. Before you even go and think of doing a spell check or doing, separating the combined words, what you need is the ground truth. You need some source of data which will help you do that. Do a spell check or do a separation of the combined words. And how we can attain it is by using a basically reverse geocoding approach where what we did was for a particular dispatch center, you create a grid of lat-longs, decide on the distance that you want between them and then reverse geocode them. Now, Google has a particular way of storing the address which is mostly formatted. If it has the address, it will be formatted. If it does not have the address, it will give you no address. Now, how it is formatted is not necessarily in this form where I have shown that this is the formatted way, but it is generally separated by comma. In your original address, there is no separator. Trust me, there is no separator which you can use universally across your address to separate the words. When you go for Google Maps reverse geocoded address, you see there is a comma separated address and it becomes very clear that something at the end is actually a pin code. Now then the second last is country. Then you have city and similar things. So you have an idea of what these tags possibly can be, what each segment of the address is. And then from here, you can create a base of each of these tags, be it locality or be it sublocality, roads for that matter. In Bangalore, we have mains crosses and then build over it. So that becomes my source of truth. I know that these are the sublocalities which exist for Kormangala. So suppose I am dealing with Kormangala DC, I know that Kormangala first block, fourth block, fifth, sixth and some layouts. These are the part of localities which will show up in my addresses. So that becomes my ground truth. I built a spell corrector over this data. So whenever, so, and here, when you're doing a spell corrector, any similarity algorithm, not a usual similarity algorithm can help you. Now you cannot go ahead and use an NLTK's spell corrector because NLTK is not built for this purpose. It has not used address data for generalizing its library. So you cannot use that library. In most of the cases, you cannot use a predefined library. So you have to build things from scratch and they're not difficult. So you can develop a similarity score for each of the words that occur in your address with this base where you, I'll show how you can do that, where you just compare and replace. So basically what you're doing with this step is cleaning the address, cleaning in the format of spell check and removal of the combined words. The third step when you will get into named entity recognition because it encompasses the natural language processing, you'll have to go through the pre-processing part of the data before the first step was cleaning, getting rid of spelling miscorrections and combined words. The second step is pre-processing the data as a textual data where you basically tokenize the words which are available in the data. So when I'm saying data, it is each address. So you're tokenizing the address into words that is existing in the address. The address is cleaned now. Then you're removing this top words. In general, addresses do not have so much of stop words because it's not, and it should not be an English flow of sentence, but you can have stop words of addresses itself. Remove, people might write anything. So just to be extra cautious, remove all the English stop words and also remove the address stop words that you see, which might not add any value to your algorithm. The third thing, important thing that you have to do when you're doing named entity recognition is a supervised training data set. How do we attain that is, according to the steps that I have shown so far, what you have now is each address, all the words in that address, clean the address, separated and treated for the pre-processing aspect. Now, there is training that you have to do. Now, why am I training it? What is the goal of the training? The goal of the training is basically to identify the entities, right? Identify entities and the entities here are tags. So the first step is to create these tags, which you want in your address data. What we did, we went ahead and we found out around 43 tags that will define addresses. When I say tags, these are something like, you can see in the example here, there is house number 36, wire strangle apartment, SD-bed layout and then the address continues. Now, in order to show how tags can be formed is, house number 36, I have given a tag FNS, FNC, FNC. Now, FN is a tag, which is flat number. FNS is starting of the tag. FNC is continued, continued. So the moment I extract FN from this particular text, I get flat number. Similarly, BNS is building name, start continued. And the moment I extract BN from this text, I get building name. So these are the tags. The tags are FN, BN, sublocality and similarly. So based on the address that you see, you can tag them, this is actually a manual process where you'll have to go through the addresses to understand for each dispatch center, what are the ways in which the addresses are written. Do this dispatch center generally deal with, are there many roads around it? So it will have multiple road level tags instead of you say anything like sublocality. Might be it is identified by roads than by sublocality. So that depends again on the kind of data that you're dealing with or a particular dispatch center that is around a particular locality. So once you have these tags finalized, you go ahead and train the data for around, what we did was around 8,500 addresses for a particular dispatch. And the training is not supposed to be heavily manual. Once you have decided on the tags, what you can do is train them through a macro basically or write a function to train it. Whenever you see an apartment, you tag it as BNC. And there are multiple things that you can automate and then go through the automatically generated train set to manually verify it. The next step is building the model. Now that you have a trained data set which is giving you tags for each of the word, what you want to build is a model that given an address will give you the tags for each of the word in the address. And the model that we tried was maximum entropy. And what maximum entropy does is if you have information, multiple information about a particular class, it helps you weigh them based on the different data points that it will see and gives you the class accordingly. So maximum entropy model also allows you, so it basically works on features. It helps you put your own features into its processing. So you can build feature data points which are, say n grams. Now what are n grams? Basically combination of words or tokens. So suppose I build features on my data set, address data set and say that whenever the third word of a trigram or the last word of a trigram is an apartment, APT or APART or whatever, all the ways that you want to put it, the tag which follows before it is BNC or BNS. Now this becomes one feature. You can also put features like landmark indicators. We also had one tag which is called LMI which is landmark indicator. Now they become very important. They're really important because you can put conditions like if you see a word in a biogram which is opposite, near, above, below, then just say the tag is LMI. So these are the features which will go into Maxent and it learns with the data and also the features that you have passed to it and gives you the tag for each of the word given a particular address. And the accuracy of this that we saw was 98%. So the Maxent does a real good job in giving you the tags of the words of the address but that's not it. That's not something that we are building. That's just the first step towards what we are building. So the second step is geocoding. Now that you have the tags for the addresses, how do I geocode it to get the exact point where this delivery boy has to go and deliver the shipment? Now in the geocoding itself, what you'll do is basically get the tags for all the addresses, form a priority of the sequence of tags that you see in each address because not all the tags, an address can be lengthy as 45 words, 50 words. Not all the tags are important. Not all the tags can get you the exact pin point of where they should go. So basically you have to form a priority sequence of tags and then create a geocode layer over it. When I say priority sequence of the tags, what do I mean? Basically give priority to all the tags which help you identify a pin point of the customer address and what are those tags? Generally the tags like building name or set of building name, set of building being tech parks or any smaller organization kind of a thing. Then you can have apartment names. All these things have more, much more priority than a sub locality or a locality because when you geocode a locality or a sub locality, what you get is the centroid of that locality which is not helping FE in reaching the customer. So what you want is exact points. So all the apartments and buildings, they have higher importance. Definitive landmarks have higher importance than non-definitive landmarks. What are definitive landmarks? Generally landmark indicators which with words like above, below, opposite, they are definitive. It says that it is there. There can be landmark indicators which are near, around this much kilometer away which is not a definitive landmark indicator and they have very low priority. So that's how you define your priority of their tags and then you geocode it. When you geocode it, you have to put a logic in place when you pass a lat long to the Google API which is geocoding. You have to pass a combination of the different tags that you have got. So you can pass apartment name along with a locality or you can pass apartment name with a sub locality, apartment name with a road or something and then check on which one of it is giving you the highest accuracy because accuracy of the lat long that you're getting is very important. It should not be one kilometer inaccurate where the FE is just roaming around. Now the next step and the final one is actually to cluster. Now that I have addresses, I have the lat long for each of the address. I've identified the tags, used the tags to geocode and I have the lat longs with 200 meter accuracy. What I would want to do is still my DRS is not automated. I would want to automate the creation of DRS and how can I do it is for every dispatch center early in the morning, you can run a job which does all these things in the background and once you have the lat long, you cluster them for that particular dispatch center. So when you're clustering, you basically get a few clusters which is defined by, I would say, majorly the sub localities that this particular DC deals with. So this might be, if it is just Kormangala fourth block, this can be one layout, SD-bed layout, Achandra ready layout or the other layouts. So because each FE is assigned to each of these sub localities. What we've used here is a modified K-means because there has to be a business logic put in when you are clustering also. You cannot, so what if you're clustering and one of the cluster has 15 shipments, the other cluster has 85 shipments. So you have to put a logic where you are putting a threshold on each cluster containing at least 35 shipments or whatever is your norm of the company that each FE should deliver at least this much per day. And then you can also customize it. So if at a particular day, you see that the attendance of the FE is supposed to be 10, then the usual is suppose 15, then you put your clustering as 10, cluster K, with K-means with K as 10 and you get 10 clusters. Thinking and in most of the cases, FE's know the locality well. So they can go to each of these cluster and then deliver it. Now, how does it function? This clustering is what it has done is has created a label of cluster name with each of the addresses for that particular dispatch center on that particular day. Now, when this center manager walks in this morning in the center, what he does is just scan each of the shipment and what he sees is the route number or say the shipment pile number. So basically he'll see that this belongs to route zero, this belongs to route one, this belongs to route two and he keeps piling each of the shipment based on whatever route number it displays. And when the FE's walk in, it just takes 15 minutes. We reduce the time from three and a half hours to 15 minutes. And then when the FE's walk in, they don't have to wait because the center manager is shuffling and sorting, but they just pick their pile and they move for there. And this also, we also automated the printing of the DRS where every route will get, you know, all the shipments that belong to it and you just print the DRS and hand it over to the FE's. So that's it that I have. This is how you can automate. If you guys have any questions, I can take them. No questions? Oh. A couple of questions. First is that what is the advantage of doing all the natural language processing before using a service like Google? You can just throw the address itself to Google geolocation and get the lat long. Okay. We did that. So that is the first thing that comes to your mind when you are doing anything around geocoding or reverse geocoding. But trust me, it does not work because of multiple things. When you are throwing an address to Google for geocoding, it needs the address in a structure format. It needs you to give it the, give it the apartment name or whatever is the POI along with sub locality or something which it can map. If you don't give it, if you give it pass it a random address, it will give you the lat long of the centroid of whatever it can identify in the address. Sometimes it identifies the state. It gives you the lat long of the state center. Sometimes it identifies just the locality in the address and it gives you the centroid of the locality. But that's not something that we wish for. What we were wishing for is the exact point where the FE needs to go. And we could not achieve it by doing that. Hi. So yeah. Thanks for the talk, it was very interesting. So how do we ensure that the process that you described for determining the lat long, that it's always within 200 meters, like, is that? Yeah, that's why I had it in the recommendation aspect. It's not always possible. It depends on a lot of things, I will tell you. It depends on the previous step that you have followed. It depends on how good is your tag at the first place where you fitted a Maxent model. How good is your Maxent? If the tags that have been identified by Maxent, how good is your priority of the tags that I described? And how good is your combination of the priority of the tags that you're putting to the Google API? As I said, Google API works in a certain manner. And you have to exploit it in the best manner possible. That's why we were doing a combination of the tags, important tags, and passing it to the Google API. So for one particular address, I would tell you, we were hitting Google API like 15, 20 times and getting the lat long for that particular address only. And then comparing within that address which one has the maximum accuracy by looking at the distance from the DC or from any fixed point. So that's how you, that is a process that you'll have to kind of fine tune and curate on your own. There's no set way which I can say, but one of the ways that I have done is something that I've explained. Hi Divya, thanks for such insights. I was wondering, have you guys taken care of language input as well before NLP? No, because most of the addresses coming from the clients, it was in English. So we did not take care of the language aspect. We did not run into it, I would say. Had it been there, then yeah, one step ahead would have been the conversion or translation of the languages. Hi, so if you put in, say, 100 addresses, what's the time it takes to do the geocoding? How many of those 100 would you have to manually correct the whatever input? The geocode stuff or the training? From address to the geocode. Okay. How in 100, how long will it take? How many of, what percentage do you manually intervene? And do you create, do you have to keep hitting Google over a period of time or do you can shortcut Google, say, over a period of time? So, okay, to answer your first question, how much time does it take? Generally, we were running it for each DC, which has a load of around 20K, at least at a particular time of season. So for that particular time, we ran it for, it took seven, eight minutes for the entire algorithm to run. So it is not much of a time consuming process. And second is how many, the second question of yours was, how many out of them do you have to manually intervene? So once you do the entire process and the flow is created, you don't have to go and manually check them. But yes, we were collecting feedback. Initially, when you roll them out, you're collecting feedback from the FEs on how well you have been able to identify the lag logs. So that process continued for 15, two weeks or so, where we took the feedback and fed it into the system. So that is one thing, but manual process comes only in the training side, where I showed you the supervised training happening. So that is the process where the manual it will come, which also you can automate to an extent. To your answer of how many times do I have to Google, hit Google API? Well, Google does not allow you store addresses or the details. So ideally you should not be storing them and hitting them every time that you are using the API. Correct, correct. So what, but then you have used the entire Google's resource in order to build it. So it is actually, I would say, companies call whether they want to store it. We were not storing it and we were hitting the API because it did not take much time. We were hitting the API every time and we were paying for the API. So that was, but yeah, you're building a database of your own with support of Google. Hi, so once you have the geo code, would not a routing, a vehicle routing problem be more useful to your use case than a clustering formulation? Correct, yeah. So I did not explain here, but within each cluster we did vehicle routing as well. But you don't necessarily need it because of, I would say a phenomena that is very popular here because the FE's know the entire area. They have it on their tips of the mouth. So basically you don't have to give them that go here, then go there and then go there. But yes, in case you want to get rid of that manual human intelligence also, you can just go ahead and route it within each cluster. So that's what we did. So I think routing it within each cluster makes more sense than routing it across the clusters. So how do you define these clusters? Are these already defined that these areas fall into cluster A or B, or is it real time? So for example, if I have a pickup from Coal Mangala, will it be picked up with HSR pickups or with the Domino pickups? Okay, so when you're doing this entire process, it is happening for a particular dispatch center. And dispatch center in logistics is basically the end point from where the delivery happens. So this dispatch center will be one dispatch center only, which will be Coal Mangala. It will deal with shipments dealing to around Coal Mangala or around that center only. So you will never hit up a case where you have shipment from HSR coming into this. And how do you define the clusters? It is real time. It will happen based on the shipments that have been received that day. You might have a few shipments, you know, larger shipments for a particular sub locality that day. So it will be richer cluster as compared to the other one. So that's how it happened. And how do we include the time slot part in this? Yeah. So time slotting, we had to modify the clustering logic. If you're supporting time slotting, like there are a few customers who want delivery to be done in this time slot. So you basically within each cluster, you'll have to create sub clusters. And then see if that makes sense. If there are just one shipment that is happening from three to five, then particularly you'll have to include in your routing within the cluster. And put it at the end. And this is what it was a combination of both clustering and routing. So in the routing logic, you'll have to put that time bucket in consideration because not all of the shipments in our case had the time constraint to it. Any more question? I think we're out of time. Thank you, Divya. So the next speaker is Aditya Madan, who is Senior Product Manager at Flipkart. And he's going to talk on perfect grocery deliveries, solving location problems unique to India. Over to you, Madan. Good morning, everyone. Thanks for being present here on a Saturday morning. I know most of them you've been partying on a Friday night. So my name is Aditya and I am part of fulfillment service group at Flipkart. I am part of the product team. And today I'll be talking about the location problem. Some of the aspects that Divya also mentioned in her talk that how this problem is very unique to India and how at FSG or Fulfillment Service Group at Flipkart we are solving that problem. Okay, so the agenda for today is I'll be introducing FSG or Fulfillment Services Group what at FSG we are doing currently. Then I'll be talking about the location problem of India and what is the impact of location problem to the supply chain and what is the havoc that it's causing on the supply chain then how FSG is solving that problem and the way forward. What is our vision to ensure that we deliver the best in class customer experience that we promised to the customer. So introducing FSG, what is Fulfillment Services Group and what it is responsible for at Flipkart. So Fulfillment Services Group is responsible for ensuring best in class customer experience after the customer has placed an order. By this what I mean is that once the customer has placed order on a Flipkart website from that point till the order is delivered to the customer and even beyond delivery the entire experience is controlled by the services offered by FSG. FSG offers different kind of services and to get into detail of those services I'll quickly walk you through what is the entire flow of a journey or order journey after an order has been placed. So quickly talking about it whenever an order is placed it's created into multiple shipments. Each of the shipment then flows through different facilities or the supply chain network that exists within Flipkart. So a shipment will flow through a fulfillment center. So we have hundreds of fulfillment center and it will then flow through a sortation center where it will be sorted further to move it to a delivery hub and from delivery hub it moves to a customer. Now think about a situation where you have hundreds of fulfillment centers you have hundreds of sortation centers thousands of delivery hubs and millions of packages are moving through this complex network. So there are millions of packages that flow through the complex supply chain network and you'll be hearing about this more when my colleague Senthil talk about this in his presentation on orchestration and network intelligence. So just giving you a brief to tell you how a shipment flows through the supply chain network within FSG. Now coming to the address problem in India before moving further I have a question like how many of you understand your address? Like all of you, right? I mean, you live there you have to understand your address, right? Okay, so why I ask this question is that I want to share a few examples of how India understands the addresses. So this is an example of a very good address where the user has been kind enough to provide the details about house number, the apartment name, city, locality and all the things, right? So user very well understands the address. Another example of a address that user has provided. So as we move forward, right? An example in front of you. So here the user has only provided locality. So user believes that this is what the understanding of my address is and this is how I've been giving my address and this is how everyone is able to connect through me to my address. Moving forward, the user says that a locality or a landmark. So this is my address, right? Anyone who has to connect me has to come to near this bus stop and I'll be there. Another example looks like a perfectly good address. If you see one of the important address attributes here is the pin code. The pin code is something that the Indian Postal Service has provided us. That's how the entire geography of India has been divided, right? But I don't think that all the users have understanding of the pin code also, right? So in this case, in this example, the address is of some other pin code. It actually belongs to 560017 but the user has mentioned pin code 5600103 and this is because of numerous reasons like one of them being that this is how traditionally they have been writing address and this is how they write it. And this one is my favorite, right? So this is example of an address where user has provided, I think a lot of information here, right? And user has shown great intent to actually find himself also, right? So the user has mentioned that you can ask anyone. So just translating whatever you can ask anyone and they will take you. So I think thanks to the user for giving a good intent of the location. What exactly they want to do, right? And helping RFEs to reach to the location, right? So this is how a snapshot of how India understands the address, right? So each and every user, each and every customer, everyone has its own understanding of the address, right? And there is no right or wrong, right? And why there's no right or wrong because address itself is a combination of numerous attributes. And all those attributes are not standardized, right? There are a bunch of attributes such as state city or even pin code which can be standardized across. But then that does not solve the problem, right? So consolidating all this, what are the problems that are associated with addresses in India, right? First of all, the scale itself is a big problem. There are zillions of addresses, multiple combinations of how user understand, multiple attributes. As I mentioned, none of the attributes are standardized, right? For example, city like Bangalore have crosses, right? City like Delhi has sectors. Even within a city, Delhi has sectors, Delhi has nuggers, they have gullies, right? So you don't have, you don't get the luxury of standardizing the attributes. Another problem is that the addresses are organized very poorly physically, right? So logically it looks like sector 23 has to be next to sector 24, right? Or house number 43 has to be next to house number 42. But that's not the case always. So the logic division does not work always within the Indian context. And as I have shown you numerous examples, users don't know or understand their addresses, right? And all these problems are further accentuated in tier two, tier three and tier three plus communities which are the next growth engine for the economy as well as the e-commerce industry. So what does this all mean for the supply chain, right? And I think Divya also mentioned in her talk, right? It results in a lot of inefficient manual planning. And why manual planning? Because you have to rely on the local knowledge of your hub managers or your field executives to actually do the entire out planning. To actually do the entire shipment flow planning. Another problem that it creates is unoptimized shipment flow. So in my first slide, when I talked about the flow of how order moves through the various facilities within the supply chain network. If let's say customer has given one wrong pin code, right? Then it can enter into a different facility from which it is supposed to be delivered. Which would mean that either an order is delayed or the customer is contacting multiple times, the customer is supposed to know where my order is. And the third most important thing is customer discovery in the last mile. I'm sure most of you would be an online shopper and may have faced this situation that delivery boys are calling you that, okay, I'm here, how to come to your location or are you available, right? So the discovery of the customer, and mind you, the customer has given the right address. So customer has said that this is how I know my address and this is how I've been giving my address. So now it's your responsibility to come to me, deliver it without disturbing me, right? So what does this all mean, right? All this pans down to one point which is degraded customer experience. So a delivery boy calling multiple times to the customer, right? Is very relating for the customer. Shipment not getting delivered on time, or whatever time has been promised is again degraded customer experience. So keeping all this view or all the problems in mind, how at FSG, we are solving this problem. So before jumping into the solution, what we feel is that every address has to be standardized in some form. One way of doing that is obviously you get into different standardized attributes. You create labels of address, right? But ultimately the micro point for doing or the micro unit level for doing any planning is the geo codes. So what I'll talk to you about is and how we are solving the problem of geo prediction, right? Different companies are doing it in different way. So today I'll talk about how FSG is solving that problem. So what at FSG we have been doing is we have built a flip location intelligence platform which we called as flip. Flip is a crowdsource platform with the aim of geo predicting each and every lat-long and we use machine learning algorithms and some of the techniques that we use I'll be talking about the flow of how do we geo code and address. Within flip when I say crowdsource platform, right? We are not using or not relying on data source from one particular source. We are using multiple data sources. One of the most important being one of our own database. I think there was a question from another gentleman in the audience that over the time you get your own database then why are you not leveraging it? So this is what we do exactly. We are leveraging our own database because we do millions of deliveries and we have got millions of exact location points that has its own set of challenges which I'll talk later but that's one of the very important database that we have built. The other database that we use is map my India data. So we also rely on their own, their network how they have created the idea of localities, sub localities and all that is called as the location type. So I'll give you a brief about how do we geo code using a geo location service that we have built. So you see an address mentioned here 691 Shahid Bhagat Singh apartment pocket three Dhuarka. This address is fed into the geo location service as an input. So address attribute is a standard input and we tokenize an address. So again if we are talked about some of the tokenizing techniques we follow some of those techniques and I've also built on top of it. So a token, what would a token look like? If you see at the bottom of your screen Dhuarka is one token sector 14 Shahid Bhagat Singh apartment pocket three. These are examples of a typical token that will come out of an address and these are very few examples. There are numerous tokens that are created within that address. For each and every token we retrieve a polygon. What do I mean by this? So for Dhuarka as a token, we'll obtain a polygon. Now how do we get a polygon for Dhuarka as an address? We have done a lot of deliveries on the locations where Dhuarka is one of the address. So we'll have many geo coordinates available for that. So we'll take each and every geo coordinate and generate a polygon out of it. Once the polygon is generated, right? Each and every polygon for each and every polygon for each and every token. Area intersection score is calculated. So what do I mean by area intersection score is? So for example, in this case, let's say sector 14 as a polygon. So sector 14 intersecting with Dhuarka. We calculate what is the intersecting area of a sector 14 polygon with a Dhuarka polygon and divide this by the total area of sector 14 polygon and come up with a score of a particular polygon. This process is repeated for each and every token and the area which has the highest score is returned or the polygon which has the highest score is returned. So the idea of taking the intersection is that we get as close or pinpoint the exact location of the address because that forms the basis of the precision or accuracy of your geo prediction. Now, how do we get to the geo prediction accuracy? Once a polygon is returned or once the final polygon is selected with the highest score, we create a bounding box across that polygon and we take the diagonal of that polygon as the precision accuracy in meters. So what are the critical outputs of a geo location service that we have built? So latitude, longitude and precision. So precision is very important to know exactly that okay, what is the level that you are operating or you are geo predicting? And that's a very critical output because different precision levels are required for different use cases and applications. So this service has a lot of use cases and application and some of them are like navigation. Another one is like identifying what is the exact customer location for a field agent. And another one is automated beat planning or route planning that we do. And I'll be talking more in detail about the automated beat planner that we have at the FSG. So before going into details of what is automated beat planner, what it all is doing, how it's consuming the geo location service, I'll quickly introduce the concept of beat. So beat is nothing but a sequence or order or route that a field executive or a delivery executive follows on the ground when he's handed the shipment. So every day delivery executives are handed say 40 to 50 shipments and they deliver those shipments in certain order or and following a certain route. So that is typically called as beat in supply chain language. Moving forward to automated beat planner. So with automated beat planner, we have, we take number of inputs, we process this inputs and the idea is to generate a automated beat plan which maximizes or which minimizes the overall cost of the delivery, but ensures that the guardrails of customer experiences are maintained. So let me detail out some of the inputs that go into automated beat planner. One of the inputs is order size, right? What do I mean by order size? So number of shipments that are there in order, typical characteristics of a shipment, which is like what is the size of the shipment, what is the type of the shipment, whether it's a hazardous material or not. So all those inputs which are associated with every shipment, what is the category of shipment goes as one of the inputs. Then geo codes, which is coming out of flip service, geolocation service, slot time or delivery time. What are the delivery times or slot times that customer has chosen or what we have promised to the customer for the delivery capacity. So we have different type of vehicles that are available in our supply chain to attempt deliveries, beat vans, beat bikes. So each vehicle has certain capacity or there are different ways of defining capacity. So we define what is the maximum carrying capacity or what is the range of capacity that you want to operate into. And the other important one is the traffic. So what are the traffic conditions that exists on a particular day at a particular time of the day? So all these inputs and there are other inputs as well. These are major inputs that we are currently consuming in automated beat planner. All these inputs are fed and ABP processes. And once ABP processes, the output is a beat plan. So the beat plan is generated to again minimize the cost. And when I say minimize the cost, it's function of numerous factors here. The productivity, the number of shipments that a field executive can deliver, the capacity or how much utilization you have done of a vehicle. So all those factors contribute to the overall cost equation. And we ensure that the guardrails of the customer experience are maintained. You don't want that you have made a promise to your set of customers that we'll deliver by this time. But to minimize on the cost, you miss that promise. So the idea here is that we continue to deliver best-in-class customer experience. But by deploying technology, we keep on reducing the cost of operations. So that's the overall journey of an automated beat planner that how it does. Now, let me talk more about what happens when a beat plan is generated. If you see on your screen, this is a UI that is shown to our planners. It tells completely what is the area. What are the different, what is the different routes that different delivery executives have to follow? What is the order? So each and every geo location here is an address or a customer location where delivery is supposed to be done. There are certain features that we give to our planners. One of them being manual override. So what I mean by manual override is that whenever a planner feels that one of the orders is not planned in an efficient manner, right? There is a chance to or there is an opportunity to move it from one beat to another, right? Why would that happen, right? If the system is predicting it, why would a planner want to do this? Or why we have given this functionality, right? So what happens is that many times at ground, right? The reality on that particular day may not be exactly what has been planned, right? For example, you have planned for 10 beats assuming that 10 field executives would come every day, right? But there has been an attrition or not all of them have reported or let's say all of them have reported but because of some unforeseen circumstances, one of the field executive had to cut short his delivery of today, right? So that's one of the reasons why we have given this plus every model or every system with have certain accuracy, right? And this is a continuous evolving process. We take feedback of every delivery that we do and we feedback into the model. So we keep on improving the accuracy but whenever there is an error rate, we throw it out of a delivery area and then we have to rely on the ground knowledge. The idea is that we want to do away with it but if we don't give that address, we have to deliver it, right? So all the exceptions are handled in different manners, right? Some of those we even call the customers to try to get a better feedback of their addresses, right? But those cases are very small and the idea is that we reduce or minimize those cases or those use cases with time. So the services that powers the key features automated beat plan are geo location, which I've already talked about, the geo distance service. So geo distance service basically gives you the distance between various lat longs. So you basically feed the lat longs and it will give you a road network distance between different lat longs and we use to create a distance matrix to come up with what is the route plan that has to be created. Maps interface, so the entire map interface that you are seeing on the screen, this is a Flipkart map interface. So this is created on the data. So we create our own tiles on the base data that we have in our database as well as we use the map idea data to create tiles and this map interface is powered by Flipkart itself and traffic prediction. So these are the services that are currently used by automated beat planner to do the entire beat planning or the run sheet planning. So what is the way forward? Looks like, okay, we are solving the problem for address. We are moving away from manual planning. We are doing the automated planning but is that good enough for customers? Is this pushing the boundaries of customer experience? So we at FSG believe that there is a lot of room to push the boundaries of customer experience itself. So what we want to get to is a perfect matching of supply and demand. Now, before going forward, let me talk to you what I mean by perfect match, right? A perfect match means that each and every shipment or each and every order should be assigned to a particular delivery channel in a perfect manner. When I say a particular delivery channel, what it means is that a particular field executive, a Kirana store or a third party logistics partner and who will define this, right? So there are two key stakeholders that are involved in this entire operation. One of them is of course our customers and the other one is our field executives or our delivery executives, right? So customers will themselves keep on pushing the experience boundaries, right? They will find that, okay, for me, perfect experience is that you are able to deliver it to me whenever I want, wherever I want and that is my choice, right? You are able to deliver me without contacting me even for once, right? And for our delivery executive partners, right? They would want that, okay, I want to do deliveries from this time to this time only. I want to do deliveries in this particular area only, right? So the matching or the perfect match of supply and demand is dependent from the supply partners which is our field executives as well as our customers and this is our vision. This is what way forward that we want to go to. So this brings to the concept of the allocation engine, right? Again, the idea of an allocation engine is to ensure perfect match of supply and demand, match each and every shipment, each and every order with the most convenient or the perfect delivery channel and there will be a lot of input services to it. I've already talked about FLIP and ABP, the location platform and the beat planner. The other key critical service that we are building and that will form the genesis of allocation engine, right? The most important one being customer availability service. So what do I mean by customer availability, right? Can I predict during the day or during the week, the point when the customer will be available to make a delivery attempt? Now, what do I mean by that? Many of us place orders on our office address, right? Because Monday to Friday mostly in our peak times we are in our office, right? Imagine a situation where I as a customer have placed an order, right? I have been given my office address and I've been given delivery date or I've chosen by mistake delivery date as Sunday, right? Now we know that a customer will not be available there on that particular day, right? So if we can predict for each and every address or each and every customer input when the customer will be available, right? This will become one of the important components which will again push the boundaries of customer experience. The another service that is very critical to push the boundaries of customer experience is FE scoring engine. So FE is basically the field executive or the delivery executive. What do we mean by FE scoring engine, right? Now, different FEs or different delivery executives will give different performances, right? None of the two people are similar, right? So we want to ensure that the feedback or the performance of each and every FE is given as input to the allocation engine. So as to ensure that the customer experience is not degraded. Example is that let's say one of the FEs has, one of the FE says that the customer is not available at the location. But actually the customer was available. He made what we call a fake attempt, right? So we want to get that and power that into the allocation engine. Another thing on the supply side for the perfect match is FE skill and preference. So as I mentioned that every FE would have preference of delivering certain time in delivering certain area. So we captured that. And FE skill, so what do I mean by this is that certain categories may require certain skill to do a delivery. A category such as TVs or washing machine, which would at the time of delivery require some kind of demo versus a category such as mobile accessories will require different skill levels of the FE. So we take this input and push this and other one being service and capacity engine. So for each and every delivery channel, we have different capacity and different service ability. So what we want to ensure is that every shipment is given to a certain delivery channel, which ensures a perfect matching of supply and demand. So this is the vision that we want to move towards and are working towards this. So that's pretty much. Thank you. Happy to take any questions that audience might have. Hi Aditya, my name is Santil Kumar. Hi. I'm here. Sorry. Okay. See what kind of accuracy where you guys able to attach in terms of identify the exact location in terms of distance. For example, 10 meters, 20 meters or anything of that kind. Okay. So before answering you that question, right, I'll give you one example of a live experience that I experienced on the field, right? And what I did was I went with one of my field executives, right, on the field. We have pointed him to an accuracy up to 20 meters, right? And believe me, even after making few calls and he be going into one direction, I go into another direction, we were not able to find the customer location, right? And why this was right, because customer had never mentioned his house number outside his house. And the gullies that we went into, right? I have never been to those gullies, right? So why I'm sharing this example is because this has different accuracy levels are required for different applications, right? If let's say I am doing a shipment planning, right? That which delivery hub it should land into, right? Then maybe I don't want to get into an accuracy level of 100 meters, right? I want to get into an accuracy level of one kilometer is good enough for me, right? But if I am doing a planning for the last mile delivery, right? Then we want to get it as close to as 200 meters, right? Now, how we have arrived at this number of 200 meters is we have a internal metric of, let's say how much reattempts are happening, right? So we see that, what is the sweet point at which the reattempts start flattening out, right? So we figured out that sweet point currently is 200 meters, right? Having said that, that is not the, I'll say the best number. The reason being that it will change across the geographies, right? We move into a tier three cities, that number can change again, right? So this number, I'm talking more about the metro cities as of now. So this is what we achieve towards or plan to do towards. Hello, Mike. Hi. I'm sorry, I think we have to stop right here because we're already out of time. So thanks, Aditya. Thank you. We can catch up outside, right? For any questions. I'm sorry for the time. Right after this talk, there's a, tea break. So we're running late about 10 minutes and so the talks are all shifted by 10 minutes. So the next speaker is Jairam Kasi, who is a co-founder of picole.com and he's going to talk about using visualizations for addressing fleet management problems. Over to you, Jairam. Good morning, everyone. Unlike me, this is going to be very short. So what I wanted to talk about was our approach towards solving data-centric problems first through a visualization, right? I mean, this is the typical flow that we as data scientists take. We visualize the problem, we help business users visualize the same problem that we are seeing and we understand what kind of decisions are they making? Are they choosing door one or door two, right? And then we make a study of those decisions and then we build it into some kind of either logical algorithm or some kind of machine learning optimum algorithms, right? So ideally, so this is a very dynamic data set and I will tell you how we've used this for one particular use case, which is freight matching, right? So any data set, right? We convert that into a data set of decisions which business users come up with, right? And then that gets converted into an automated algorithm. Now, what am I talking about when I'm talking about the approach, right? What we typically do is a ground up approach which relies heavily on what kind of variables are there in the data set rather than looking at it as a charting problem, right? So if you look at visualization as a charting problem you are limited by what is available to you in either the BI tool that you're using or any charting library that you're using. When you're building things from ground up using, see, there are a lot of very powerful JavaScript or Python libraries for it, right? There is D3 which is available in JavaScript. There is Py... Processing which is available in Python. What that helps is transferring the kind of insights that you want directly to a business user that it is visible to him as some sort of a attribute, some sort of a visual attribute, right? So first you look at the data set and there is a very interesting paper by Colin Ware which talks about pre-attentive attributes which basically are in four different categories which is color, form, some sort of spatial features like position, right? And movement, right? And all everything that you talk about like shape or length or width or collinearity form some sort of a subcategory of each of these, right? Or if you look at color, there are further subcategories of color like hue, saturation, lighting. So once you understand this sort of a split of pre-attentive attributes, whenever you see any visualization, you try and decode what the person who's built the visualization wanted you to read in the first place, right? Then you look at the detailing which is required. When you have to encode information for a business user to decode, you also have to know up to what grain that he wants to go. And finally, and this is one of the most key areas actually is the design choices that you're making. Are you going to use multiple hues in the color scale that you're using? Are you going to take a linear scale or is it going to be a log scale? What is the kind of values which are there, right? So here it is not just a data science problem. It's also a design problem. It's also a UX problem. It is also a psychology problem. It's also a consumer behavior problem, right? So I'll talk about how we've done this for one particular use case that we had which is freight matching. Talking a little bit about pickle.com, we are a 3PL service provider for sensitive cargo. So we ship white goods, furniture and certain sensitive cargo like ATMs, servers, things like that, right? So we get a lot of mid, I mean, long haul LTL loads. When I say LTL, it means less than one truck load, right? So it is important for us to be able to visualize what kind of load, what are the kind of loads which are coming in a similar date range and in a similar territory so that we will be able to couple them and send. So this also is a purchases problem for us because purchasing team is not able to visualize what kind of loads are there which is there between two cities. So once there is a data set which is coming up which shows how costs come up, come down drastically over the larger load size in a long haul scenario. So it also gives the purchases team the ability to forecast what kind of truck loads we'll have to place, right? So roughly this is how it is going to be, like there are multiple ports where shipments keep coming in a very random Poisson process, right? So it's a chicken and egg problem for us. We cannot cut costs without volume going up. We cannot get volume without cutting the price, right? So if you look at the typical total cost which is to be for and compare it against shipment size, it comes down exponentially over time, right? And the marginal cost for a volume of shipment is almost asymptotic, right? And the curious case is that the smaller trucks can be sent only for a smaller distance. So over smaller distances, the aggregation opportunities are smaller, but over a larger distance, it becomes increasingly more, right? If you look at an international use case, there are loads that we can couple actually from multiple cities coming to a same port, then aggregate and send to the destination country and then split the shipment from there. Another problem that we have is that the demand patterns are highly skewed, right? Even though the net average demand for the company is more or less the same, but channel-wise demand is heavily dependent on projects. Let's say if we get a project suddenly to deliver 200 shipments in Delhi, it just comes in a very seasonal manner. So this is the dataset that we are talking about. We have, we map the city locations. We have the places and we have the location attributes and we have the load data between city on, we have a day. So the problem is basically find loads given that some other loads are already in, you already have loads for certain territories. So if I break down the dataset, it looks something like this, right? Now I look at the pre-attentive attributes. We look at seven of them which we very commonly use, which is color, position, movement, sizes, shapes, orientation and pattern are not, so we, these are some of the typical ones. There are others like collinearity, which do not translate very quickly on a UI. So after that, we looked at what kind of detailing we need for each of these, right? And what is it that the business users want to see? Business users wanted to see what are the loads which are going to come up in a particular channel. What are the loads which are originating from a particular port at a particular date, right? And, you know, some sort of a visualization on how the, what the type of loads which are going to come in a particular date. So this was the first step we took. What are the very obvious encodings which are possible, right? Positions can be encoded, I mean the lat-long can be encoded as a position on a 2D scale. The shape is basically a line, a curved line from point one to point two. And then ports are encoded as circles, right? So we started off with this. So this is a very iterative step-by-step process. So I'm skipping a few steps here because we're not short on time, right? So when it was a smaller data set, it was fairly easy for them to visualize. There were like four or five lines from Bangalore. But once we started operations in other cities, this became slightly cumbersome for them also, right? And so then we decided to, so they wanted to see what is the aggregation rate, right? So we encoded that as a single hue and with multiple lighting based on a scale of zero to 100%, right? What they said was that does not work for us. You either give us yes or no decision. These are go-no-go decisions for us, right? So we encoded them into a two-color scale, whereas we felt the lightness comparatively going from lighter to darker shade would give them a better view. But this was something that business came back to us and told, right? So in this context, they were also losing the context of where certain nodes are going. Is that Kolkata or is that Bhuvaneswar, right? So they wanted us to encode this whole thing inside a map. So that's when we started with this. And then what we decided was that they couldn't, see, we're not a big fan of using movement inside visualizations, but in this case, it works because it directly shows the movement of goods from point A to point B, right? So, and to add to that, we gave them more drill-down features and we also gave them a voice interface so that they can talk to it, right? Tell me all loads between Mangalore and Pune and it'll just suddenly pop up with all the loads which can be coupled, right? So once we did this, right? Once we looked at the decisions that they were taking, we got some very simple heuristics that they came and talked to us about. If they see a green line, they click it. If they see a very thick orange line, they click it. So this forms an input for us to build optimization algorithms and figure out, you know, where all can we build these? How do we go about building these things? So this was built with D3 and Dialogflow and the code is available in a GitHub repository, you can go and check that. So once we did this, we figured out, we saw that the volume was significantly going up. The cost per shipment had come down, but we were able to cut price even more than that and still got a revenue uptake. So these are the two papers that I mentioned. One was by Colin Webb and the other by Stephen Fu. Thanks, I'm off for questions. Can you also talk about how the interface and the interactions was, like the voice interaction that you said, do you have it? Can we see it? Yeah, let me just check. Just give me a minute. Excuse me? I have a question. Yeah. Yeah, here. Yeah. So you were saying that it's a chicken-eating problem about the transportation and how much you are covering. So in the starting, do we have any criteria that in the starting, if my volume is, say, 10 in this area, my cost can go up to 50% or 100% but with time it will come to 50% or 20%. Do you have any numbers around it? My target is this much and my volume will increase this much. It will be decreased by X amount, right? Yeah, yeah. So that was one of the slides that I had shown, which is basically, yeah. See, in time, actually the cost does not go up drastically or come down drastically because these are long haul shipments that we're talking about. So beyond upon, there are not a lot of visualizations, there are not a lot of optimizations that you can do on long haul logistics costs because it is just not possible to send a 120 feet container in roads in India, right? You have to split it and do three 40 foot containers if you have that bigger load, right? So over a longer period of time, it becomes very constant. So there is a very, in this particular case, right? There is a big, so between a 40 feet and the 120 feet, there is not a lot of difference in marginal cost. The cost of sending 40 feet and one, I mean the extra 80 feet of load is more or less steady but the cost between an ACE, let's say 200 cubic feet size and the 1000 cubic feet size comes down drastically, that is this area here, right? So basically the incremental cost is basically the area under that curve. So the area under that curve beyond that point is more or less constant but in the beginning portion, it comes down drastically. So when you decide the pricing for these, how do you do that? Like, because in the starting, you don't know what's the volume, right? So you would obviously take the maximum, the minimum possibility of it but then how do you, what's your margin and all? Like, what's the upscale which you take? So what we typically do is, see we know that the arrival of loads is going to be in a poison distribution, right? So we kind of, but beyond that, where is that load going to come for is basically fairly random. So we run a few simulations and try and figure out how many loads we are likely to get but we know that the load size are going to be in the top six cities in India for us, right? So once we know that distribution, we try and forecast that piece, right? So I mean, on top of this, what we are building is that kind of a prediction model which says that, you know, between 31st and 2nd of December, I'm likely to get 4,000 feet of requirements from Bangalore to Delhi channel, right? Based on that, we place the truck requirements accordingly. Yes, drastically, because I mean, that is one part of it which we cannot predict, which is there are some external factors which affects the costing, which is petrol prices for one, right? Do we have time or? Completely, thanks. Thanks, Sarah. It was an insightful talk. It's completely out of time. I think now we'll have a tea break and then we'll reconvene for a couple of talks from Zoomcar and Flipkart. And there will be a Bob's session that is sandwiched between the two talks. So we are running about 10 minutes late. So since we're starting at around 10.50, we'll reconvene at around 11.20 back here. Welcome back in the post-tea break, a morning tea break. So we're going to kick off this part of the session with two talks and this Bob's session in between. So starting off with Arpit Agarwal, who is head of data science at Zoomcar. And he's going to talk about revenue maximization from pedal trips, which is a cycles, using network analytics and geospatial mapping. So over to you, Arpit. And start off. Thank you, Naik. Hello, everyone. Hope everyone had a good break, right? So today we'll be talking about a very different kind of business. Till now you have been hearing about e-commerce, about food deliveries, right? About different share mobility. But today we will be talking about bike rental. So how many of you have seen these green bubbly bikes around you, right? Okay, so a lot of people already know about this. Great. So just to reiterate, we are pedal. And pedal is a bike share, it's a shared mobility where you can rent a bike, go from point A to point B, right? And pay per use. Now how it works. So let me just give you a brief background about it. So you can go to a station, you can see a bike. It's QR enabled. So you scan a QR code, link your Paytm wallet first, scan the QR code and that's all, right? So you can go from one place to another. When you want to end a trip, you just lock it and on your app push the end-trip button and it ends. So as simple as that. So the purpose of starting pedal was to make shorter commutes convenient. So we have cabs from going from one place to another that are far off. But imagine that you're going to a place where you end up in a metro and now you have to go to your home. Now how do you go to your home, right? Do you want to take a cab again? That'll cost you around 100 bucks or 50 bucks or you can take a cycle from there and go to your home. So that is the whole point of starting this business. Now, just to explain a little further, if you have seen these bikes, there are two main components to it. One is this basket. And trust me, this basket is a solar panel, right? What it does it, it gets charged by the sun and it powers this IoT lock. So this lock at the back is where all the magic happens, right? So there's a small box which have all the GPS tracking. It captures the battery status. It gives signals to lock or unlock a cycle, right? It tells you the strength of the signals and everything else. So these are the two features that are put on the bike because of this is whole ecosystem works, right? And this is also the way we capture data. So let me take you through how we expanded because it's a very new business. So I thought that it'll be good to share with all of you. So it started somewhere in January 2017 where we built our minimum viable product and we set it out in HSI layout. So we had only 15 stations at that point of time and it used to be there was no IoT behind it and it was completely manual. They were fleet at all these 15 stations and people used to find these good looking bikes and they would come and ask, what are these bikes for? And they will say, I want to go, you can take this bike and go to BDA complex. You can take this bike and go to 27 Main Road. And we had stations there and people will tell, okay, I want to go from this place to another. So we would communicate to the fleet at that station that, okay, this bike will be coming there and that is how it used to work, right? But what are the challenges in it? The challenge was, obviously all the data was manually collected. There was no IoT, there was no tracking, right? So those are the major challenges but what we learned from this was that people really used to like it. There was a void in the market that there was no way to go shorter commutes. It was either through walk or either through caps. And even the buses had very fixed stop and the routes were not convenient for residential areas. So we did this expansion for around six months and then in November 2017, we expanded to 1,000 cycles. Now this is when we actually introduced the lock. This is then when we actually introduced the solar panel and we started collecting all the data. Now what happened was the reason behind it was that we want to make it manless. There was no one present at the station. Now people used to discover themselves. They used to unlock themselves. They used to go from one place to another and they used to park it back themselves. Now initially we started in Bangalore, especially HSA layout, but then we expanded to Pune and Kolkata. The reason being these are smart cities. We had a lot of government support. They wanted to go eco-friendly, right? And they wanted such initiatives to come to the city. Then we expanded in Agra, Jaipur, Gurugram and all these cities. And but we had a little setback there because there were high cases of vandalism. Now the obvious question would come, you know, why would I not see the cycle, right? How do you protect all those things? So though we had a lot of things by which we mitigated theft, so inside a lock, we have a buzzer. So if you try to move a cycle when it is locked, it will buzz off, right? If you try to shake a lock, it'll buzz off. So and it'll send an alert to the fleet or the city operations team that someone is trying to fill with the cycle. So this is how we tried to protect vandalism. Then we thought that, okay, it's maybe it's a good idea to launch in close circuit areas like college campuses. So we launched in ID Bombay, ID Chennai and IISC. And then when we had success in that, in around April, 2018, now we have a fleet of 12,000 plus bicycles. We are in 15 cities across India, New York cities like Ranchi, Raipur, Varanasi and the maximum density we have in Bangalore, Kolkata and Pune. Okay. Now what is the problem here? So we had problems of vandalism. We have problem of the GPS location not coming in the correct way. We had problem that the battery is drying off. But what is the major problem? The major problem is when you expand to so many cities, right, how many cycles do you put at a station? How do you maximize your revenue? It's a very low ticket item, right? For if you are taking a ride for 15 minutes, you hardly pay like three rupees. So how do you sustain this business? So for sustaining this business, this is the problem that we are trying to solve and this is my main focus for this talk for today. How to increase the number of trips per cycle and hence maximizing utility and revenue. Now, why do you want to do that? So these are the challenges. Right now allocation of cycles at the station for heuristic base. So when you're launching in say HSI layout, so I know the places, I would know that, okay, these BDA complex, 20 cent main road, these are areas where you have a lot of crowd, where you have a lot of people shopping around. So I should put more cycles here. But when you go to newer areas, like for example, Kudluget, Victoria layout, right? How do you recite the number of cycles you put there? It's very difficult. It cannot be heuristic base and you would know good about a city. But what about you go to Kolkata and they want to expand there? How do you recite the places? Then the trips for cycle right now was very low before we launched the experiment. The third was rebalancing of the cycles was totally fleet based and it was done only once a week. So on Saturdays or I think Sunday morning, they used to go and they used to just rebalance all the cycles there. If we have hoarding of cycles at one place because everyone took from point A to point B, no one took from point B to point A. Now, how do you solve those problems? And last was identification of new sites. So I launched an SSR layout. I launched in Kormangala but I launched in Indira Nagar. But what next? How do you expand? So to solve all this problem, we thought that it's not as easy as taking out a report or generating some statistics. We have to really visualize it. So to do that, we followed this approach. So there are two legs to it. One is, how do I optimize the current network of stations? So if I have SSR layout and I have around 15 cycles in SSR layout, how do I make sure that I have the right number of cycles at each of these stations to maximize the number of trips from each station and overall network? For example, if I have, as I said, if a lot of cycles go from point A to point B and not coming back, then how many cycles should I put at point A at the start of the day? Because I know they will get depleted. So cycle rebalancing to optimize for number of trips per cycle. Second, identifying the dead zones. So you decided to start a station somewhere, put cycles there, but then you realize, oh, no one is taking cycles from there, right? So can I relocate those cycles to somewhere else? Third, identifying frequented routes. Now, for example, there's station A. Now I take this cycle to a station B, but in between, I stop at a hypermarket, right? Because I want to buy groceries. So can I open a station at the hypermarket? Because these areas are demand aggregators, right? They will build a network. So how do I find these things? Identifying areas where people abandon cycles, these are very important point. Reason being, you can only end a trip at a station. You cannot take a cycle and leave it anywhere, right? Your money will keep deducting. So for example, we don't have station every 200 meters. That is what we want to achieve in future. But right now, what do we do? So we find out which are the areas where people took cycle from and then abandon it here. So maybe if a lot of people are abandoning there, if you open a station there, that will help people. The fifth point, I'll just touch upon a little later in the presentation. The second leg to it is expanding into new areas. Now, for example, when Uber Eats started, right? So it started with Kormangala, but how did it identify? So if I was sitting in Whitefield and I wanted to avail that service, so they would tell you which area you're coming from and then we will shortly expand to that area. So these are called empty searches. This is a concept of empty searches. That you get a search, but you don't have a product there. So we used to capture that when we started this and identifying connecting neighborhoods. So for example, you have HSA layout, you have Kormangala, but these are pretty apart. They are at least five kilometers distance from each other, right? So can we open station that is in between these two? The demand does not originate from that, but people, you know, it's like a connecting station. So it helps bridge demand between these two big stations so that the overall network density increases. So what we'll do is I'll first take you through what our federal networks look like. So this is a short video. Okay. So what we have here is these yellow dots are the stations all around Baglow. So for example, this is JNagar. Then we have around residency road. We have around IISC. We have Indranagar, Kormangala and HSA layout. Now you see blue dots coming, right? Now what are these blue dots? So blue dots are the places where the people are starting trips from and the red dots are places where people are ending the trips. So now there are some obvious observations here. So you'll see a lot of people are starting and ending here, but these are kind of empty, right? JNagar, residency road, Kaban Park. IISC is pretty good, right? So there are some obvious insights that you get if you plot it well, right? What next? Let's go dig down a little deeper. So let's go to IISC, right? So this is IISC. Now you will see all these stations are doing well. A lot of trips are starting and ending at these stations, but these three stations, no one is either taking cycles, no one is either ending at these stations. So basically all the supply at these stations is kind of dead, right? So this frees up my supply from here. Similarly, let's go to the area near Kaban Park. Okay, you see here. So this is a whole set of stations. Stations on the right-hand side, these have a lot of trips starting and ending, but station on the left-hand side, they're almost dead. Now there's so much of inventory lying idle. There are so many areas where I have a good network and areas where I have a bad network. So if I were to show you how the trips are panning out, so it'll just come in a second. So this is how the trips are happening, right? See here. All the people from this area, from residency road are not really going within. They're transversing to either Indranagar or Kormangala. ISC is a very close circuit, right? And there's a huge network density in these areas. So this is the power of visualization. Now how do you utilize this? Now we know this helped us in defining the problem that we really need to solve it now. Now, how do you do it? So next is what the major talk is, decoding a network. How do you decode it? So our objective is to identify the right number of cycles that we should deploy at the start of the day at each station so that the number of trips per cycle gets maximized and does the revenue. So to understand this, there'll be two concepts that I'll be introducing. First is the rate of outgoing trips from a station. Second is a transitional probability of going from station A to station B. Now what is the rate of going, rate of this from a station? It means that it is the expected number of trips per day from a station given X number of cycles at the start of the day. So if I have, say 10 cycles at the start of the day, how many outgoing trips can I expect? If I increase the cycles to 15, can I expect more trips, right? So what we did is we had data, so because we had launched it for around more than one year, for every station, we had data when they were different availability of cycles at the start of the day. So we figured out when the availability differed over time, what was the increase or decrease in the odd trips per day. Now, initially we thought that we'll build a linear algorithm to it and if you keep increasing the supply, the demand will keep increasing because understand this is an impulsive business. You see a cycle and you write it. It's not that you know and then you come, right? But what happened was it's like a projectile, right? It has diminishing utility over time. It demand is capped after a certain point of time. So that is why what we did is we fitted in a polynomial equation. So this becomes our rate of trips. So what it tells us is that if I deploy C cycles at the start of the day, at what rate my trips outgoing trips will increase. So we defined this function using past data for each and every station we had across Bangalore. And this is the objective function of number of trips. So the trips at the start of the day into the rate of trips expected at the station. So if you do it for every station and sum it up, it'll give you the total number of trips for the whole network. And this is what you want to maximize. Let's go to the next thing. So this is building a network chain. So say there's a station A and this station B at the start of the day, they are 10 cycles. And the rate of trips of the station is 0.5. It means that if you have 10 cycles, probably you will have five trips from that station. Now those five trips can either be a round trip. I can start from the station and that station or it can be a trip to a station B or it can be a trip to a station C. So the probability of round trip is 60% for the station A to B is 20% and A to C is 20%. So we calculated something called as cycles we are expecting at the end of the day. And I'll tell you why it is important because it helps us defining constraints. So cycle at the end of the day, cycle at the start of the day plus incoming minus outgoing, simple. What is incoming? Incoming is the cycles I expect to come from different station to station A. So incoming at station A is cycles at the start of station B into the rate of outgoing trips I'm expecting at station B into the transition probability from B to A that is 20%, similarly from station C. So this will be the total incoming cycles I'll expect at station A at the end of the day. For outgoing, again cycles at the start 10 into rate of trips into one minus probability of a round trip because if I take a round trip, it'll end up here. It'd not be outgoing. So cycles at the end of the day is like this equation and it's 14. So see you started at 10, you ended up at 14, right? And this is the most important thing to analyze the network. How much should I keep at the start of the day? So now to optimize, we define three constraints. Constraints are important, obviously. So the constraints are cycles at the start of the day at any station should be greater than equal to zero, right, logical. Why greater than equal to zero? Because we want the algorithm to tell us, right, that you don't keep any cycles at all also at a station. That is why we said greater than equal to zero. We want to remove those death stations. Second is the cycles at the end of the day at any station should be greater than equal to zero. It cannot be negative, right? It has to be positive. And the sum of cycles at all stations is equal to the total cycles. So this is a macro constraint. So this is what it looked like before. This is HSL layout. This is how the trips look like. And after we run the optimizer, this is how it looked like. You can clearly see the density of trips have increased. Now, how much they have increased? Let's see. So we had around 150 total cycles. The uplift in trips per cycle was 20%. The uplift in revenue was 15%. Why the revenue uplift is less than the overall uplift in trips per cycle? Because there's a rebalancing cost also. So you have to pay the fleet to move a cycle from one place to another. But even after factoring in the rebalancing cost, the uplift in revenue was significantly higher. So that is the use of doing such kind of things. Now, this solves for the first case that how do you optimize within a network? But how do you identify new network? That is the second problem. Again, three legs to it. Identifying the frequented routes as I talked about the hypermarket. Identifying areas with no station and high abandonment and identifying areas with high empty searches. This is a route map. So what we did is we collected all the GPS traces when you're going on a cycle and we plotted it as a heat map. So this is HSA layout. You can see these places marked in red that this is HSA flyover. Where a lot of people are going and stopping but there's no station. Just Sandra, no station. Silverboard, no station. So these give us probable areas where we can open new station because they already demand aggregators. We call it at the founding graph. So this is Jacsandra. A lot of strips are starting from nearby areas in HSR and Kaur Mangala, ending up at Jacsandra but there's no station. So these people are manning here. So this makes a use case for creating a station here. And then we plotted the empty searches. You will see in Marthali Bridge, Outer Ring Road, Cabin Park metro station where we don't have any network at all. People are searching but they do not find the product. So by this, we expand to newer areas. So these are three use cases where you, how you can expand. These are tools and techniques you can use. We use Kepler and Deck.gl. We use Folium Mapbox. Techniques are heat mapping, network analysis and operational research, right? Then other initiatives that we took after we did this is that we launched subscription. So I don't think you have heard about it or not but we lost subscription at 49 and 199 where you can take unlimited trips in a month, right? And why we did this is so that we increase the frequency of trips from a particular user. So how we utilize it in mapping is the areas from where we get more subscribers, we make sure we have more number of cycles in those areas. So we put a layer on top of a maps and we also plan to reduce our rebalancing costs. We also plan to incentivize users. We'll give them the credits to take a cycle from one area to another, right? So that ways we don't take the burden and the user also benefits and makes money out of it. So implementation was not a cakewalk, right? It was really difficult. It took us several months. We started this initiative somewhere around six months back where we went and talked to the operations team and they said, you know, we don't wanna do it. We don't see value into it. And then we attended various conferences, we talked to various people and got to know about these tools and techniques. So these are learnings. Start with small experiments. We started with HSL layout, we proved it to them and then pushed it to them that, you know, why aren't you using it when you are seeing the value? Keep measuring it and keep flashing the results, right? Send them a report every day that if you do this, this is the result. If you do this, this is the result. There's no way they can deny that. Build map to highlight actions and not just describe data. So tell them that these are the number of cycles, you know, required at the station at the start of the day and don't just tell them how the trips are moving around and don't underestimate the power of making it look good. So you might see that it looks a little fancy, but that is one of the reasons people are using it, right? Initially we used Folium, but then we switched to Kepler.gn and it really looks fancy to us too. So we also like working on it. So how can you guys benefit from this? So I've listed down some of the use cases that you can use, identifying areas to expand your operations using app search data in terms of food deliveries, groceries, medicines, e-commerce, area where they don't have right now. Second is recruitment or allocation of free persons by areas to optimize delivery time. So if you already know you have delivery orders from these areas, how many persons should be present at the start of the day? Second is decentralizing warehouses and distribution centers across the city to minimize the time of delivery. If you know that you are getting a lot of orders from this area, why don't not have a warehouse or distribution center near it? And tracking of fraud during delivery. I think Aditya talked about FE scores, right? Fleet executive scores where they're making fake attempts. So we can know that what is the distance of the GPS location the fleet executive went to and what was the geocode of the actual house and was it close enough? If it is not close enough, it's a fake attempt. So all these other things that we can use it. So that summarizes my talk and we are open to Q and A. Yeah, but okay, very interesting talk. That curve that you showed, right? The rate of outgoing as a function of the number of cycles. Yeah. Now to the extent that it fits well, even if assuming it fits well, it's dependent on the properties of the other nodes. It's not something that's just independent of that station, right? The thing is that, so you're sort of doing something local, which is applicable only if you hold other things fixed. Yes. But then you're doing everything local there as well, in each of the nodes. So there's no reason to assume that the same coefficients to the extent that, as I said, they fit well. They don't fit particularly well, but let's ignore that. Yeah. That no longer is going to stay the same. Yes, absolutely true. Yeah. So I'll tell you what we do is that that equation keep changes every day. So what we do is we base that equation basis on the last month of date, last month data, and we filter out to make that equation that, you know, we have, for example, holidays, we would filter out, we would filter out when we had very less cycle availability and all cycles were damaged. So we would know that in the recent time, the equation is as true as possible. And in future, when we'll have more data, we'll actually build a prediction algorithm to predict the number of trips for the cycle availability for each station. So right now, we start with very less data in HSI layout, and that is why we, when we plotted the data, it looked like a polynomial equation. That is why we fitted a polynomial equation to it. We'll start with. I think we'll start off with the buff, but meanwhile, while getting the chairs on the dice, I think maybe we can take a couple more questions. Yeah, sure. Hi. Sure. Why do you take a start of the day and the end of the day are the only two relevant milestones? Okay, it's operation challenges. So why is the start of the day? I could do it at every time of the day that in the evening, you should have these many cycles, they're starting here, these many cycles, but we have limited resources. We live in a practical world. So when we talked to the business team, they said, right now I'm doing rebalancing once a week. So for us to convince them to do once a day was a big job. No, no, no, it's not about rebalancing. Say, what if in the middle of the day, the number of cycles at a station goes to zero, you'll not be able to collect data on what would have happened if the cycles were there. Yeah, so if it goes to zero, we cannot do much. See, but what we are trying to do is we try to build a optimizer in such a way that it should not go to zero. So we over index the cycles that we keep at the start of the day, right? And we try to, for example, one of the constraint you said, we said that the cycles at the end of there should be greater than equal to zero, right? So what we can do is we can do greater than equal to five so that it predicts to keep more cycles so that there's a minimum viability at any point of time. So these are things that we can tweak. We started with this, we can always improve. I think that was the last question. Can I call the participants in the one stage? Sankur Goyal from Swiggy, Jeram Kasi from Pickle, and then we have Aditya Madan who spoke before from Flipkart. So the next speaker, so the next speaker we have is Senthil Lem, who is director of products at Flipkart. And he's going to talk about supply chain and supply chain and using network intelligence and orchestration techniques as well. Over to you Senthil, you're all wired up. Can I remove this one? Can you hear me? Hello, welcome. So this talk is more about the, so far we have heard about conversations on localized problem solving around hyperlocal problem statements or in the lost mail delivery problems or even the vehicle routing problems in the lost mail. So this talk is more about the background before entering into the lost mail and it will be heavily on the context of the e-commerce or a marketplace kind of scenario. So my name is Senthil, a quick introduction would be. My name is Senthil, I'm a product manager in Flipkart. I have worked in several supply chain problems all the way from the very start of Flipkart having its own supply chain till now. A quick update on the flow of the conversation will be like first is about talking about the landscape that we operate, what are the dynamics of the landscape? What are the dynamics of the landscape that is coming outside as a demand and what is the dynamics of the landscape that we operate within the business? Second is about what is the key problem statements in this domain? So there is a bunch of problem statements that we look at it and how the network intelligence has a subject or a concept is well positioned in this overall scheme of things. And finally, if time permits, we'll also talk about an illustration, our particular use case or particular problem statement and see how we have evolved in that direction. That's a overall journey for this conversation. First, we'll start with an operating landscape. Step one is from a Flipkart or E-commerce marketplace scenario, the categories that we operate is humongous. So these are some of the typical examples that I can quote here is like books, which is very different from the way it operate for which we operate, it's very different from the way we operate books to furniture or it could be larger plants or it could be grocery and so on and entering into jewelry and new subjects like that. So the product categories comes with different dynamics altogether. The dynamics in terms of working capital management, the margin that it operates, the consumer demand it generates and the kind of consumer expectation that it generates is very, very different. Not only that, this product also has very different types of attributes. For example, if I want to deliver furniture to a house, it's not one product that need to be delivered, it could be multiple parts that need to be delivered or it could be some of the damaged parts that need to be delivered with this, which could be a replacement of a part rather than the whole product. Or it could be, some of the products could be fragile, for example, it's a wall clock. It need to be handled with care. Also the stacking norms of the product differs across the categories, right? And so on and so forth. The managing of products could be in lots or it could be in heaters or it could be high value and inflammation or it could be a hazardous product and so on and so forth. The third dynamic that we talk about is customer segments. The customer segment that we operate is pan India, with a variety of customer categories they come from. For example, we just launched a plus program, which is our loyalty program. So there's a difference between the plus and non-plus customers. There's a tier one, tier two, tier three customers. There's metro customers. There's million years that we operate. There's a women population that we operate with different age groups and so on and so forth. The important subject here is that these customers have different needs when they look at the e-commerce platform. And when you intermingle that with the products and its kind of dynamics the product brings in, the e-commerce platform is pushed to deliver a lot of new services. It's not just vanilla delivery that we do place and order, get it delivered. It opens up a variety of new supply chain services that is required. It could be schedule delivery, next day delivery or it could be faster speed like same day delivery, slaughter delivery. We talked about some of the in the morning conference, we talked about schedule delivery in the last mail areas. It's also could be installation. We do furniture and other stuff like it's records installation, reverse pickups, open box delivery, so on and so forth. It could be a very different thing if we enter into the house and deliver things, right? Compared to something else. So the variety of supply chain services not only that, it also can since some of the deliveries could be much, much longer in lead time, like it could be three days or five days and so forth. The customer needs also change between that order journey from the time of order placement to all the way to deliver. He can change the address or it could be changing the delivery location, date, place and so on and so forth. So the variety of supply chain services get intermingled. And finally, to operate all these things, the fulfillment models that we work in a e-commerce marketplaces, very different. It is not always an inventory and we deliver to the location. It is also about sellers managing their own inventory and partnering with the sellers to deliver the products. It could be brands who is managing the inventory for a variety of marketplaces, not only for Flipkart. How do you interplay with them? Are ready to make orders. For example, if we are launching jewelry, it could be ready to make orders that we want to deliver to the customer and so on and so forth. Homely channel, hyper-local. So Flipkart is in grocery these days. So we are in hyper-local. We are looking at hyper-local and homely channel business in a very close fashion. So there are multiple fulfillment models that can be operated for this kind of customer demand. The all this thing comes with the one important part of demand volatility. What a demand volatility here is that the demand variation that we see in a marketplace scenario or in a e-commerce scenario is very high, very, very high in terms of, so it is also induced as well as I received. The induced part is that we run a lot of sale events. We recently finished our big billion days and when we, so that the scale that we operate on a big billion is maybe five X or six X of the range that we operate on a BA scenario. So it comes with a very different demand volatility into the picture. The second, given this operating landscape, right? When we look at supply chain, how does the supply chain look like? We operate hundreds of warehouses with millions of products in the warehouses. We have hundreds and thousands of sellers who is part of the entire e-commerce story for us. These partners not only spread across the country and the customers also across the country. So we, the supply chain here is not catering to a very specific location. It is from all the way from warehouse to seller to any number of pin codes in the country, right? So it is from all the possibility, all the 20 K pin codes, the source could be all the 20 K pin codes and the customer or the destination could be all the 20 K pin codes. That's the kind of supply chain network that we are talking about here. So all these things are delivered through people, vehicles and those are the currencies of operations. So if I look at the people part, we manage a large fleet force, co-owned, yeah, partnered and so on and so forth, right? So the number of fleet force that we manage is also human. Among us with different skillset. It's not a specific, it's not only a delivery skillset. We look at installation. If you look at people etiquette, there's a gross delivery where we are doing delivery inside the warehouse, inside the house of the customer with the very different things. While a book delivery requires a very different etiquette in front of the customer, right? So the skillset of this field force varies a lot. And second, the next part is about the partners and vendors. The entire supply chain, when you connect all this 20 K pin codes to all the other 20 K pin codes in the country, it's not only a job that Flipkart can march forward. We also need the partners and vendors who can play a very key role in taking us forward in the journey, right? So Aditya also mentioned about the millions of packets that we deal in a network, right? So if you look at our network at any point in time, there's more than a million of parcels that's moving around. And from all these sources and all these distinctions, the routing of this parcel become much more important. We operate across multiple operating methods. So today Jairam was talking about the truck testing. So we operate on more than thousands in the scale of thousands of trucks that we manage, bikes, our own large line-all vehicles. We also partner with all the cargo movers through flight and train and all those vehicle modes, all the transportation modes is also part of the story. Finally, it's not only about the physics that we move around, there's also a lot of cash that moves around this entire network. It's in tens of rows of cash is getting dealt every day. It's also closer to a bank supply chain, right? So that's the amount of cash because of the COD or the cash on delivery nature of the business, that's the amount of cash that we deliver. So another interesting part is that this also comes with supply value volatility, right? So for example, a vehicle can get broke down or it could be an external climatic condition that is changing or it could be the absenteeism that we are seeing in a particular location or it could be festivals and holidays and so on and so forth. So it all brings up the overall supply volatility picture to it along with the demand volatility picture that we saw in the previous slide. What does all indicate to us? It indicates, so here, this is the point I want to take a step back and look at what is network. Network can be defined in various contests. In a supply chain, a network means it is the operator. So all the element that we saw in the previous slide, right? The sellers, warehouses, vehicles, people, partners, vendors, and all the stuff that is included along with our own facilities, the delivery hubs, people called dispatch centers and so on and so forth. So all these things form the part of the network. Now in the, as Flipkart grew up, Flipkart also had a very fast scaling of things. So it is like, when do you employ intelligence? Vis-a-vis, how do you fast scale? So that's a trade-off conversation we usually have. So in the initial days when we had this growth, what we did is that we operated mostly from the inventory model. So if the incoming demand in a variation form, when it comes to us, every order is pushed into a warehouse. So remember, we talked about millions of packet and this is very close to how our warehouse looks like. From warehouse, it has to get sorted all the way to the 20K pin codes that we talked about. So it's a sortation in itself, it's a very huge problem. So sortation, let me give you one snapshot about sortation. It's not about connecting one product to 20,000 pin codes. Every product has different nature. So we talked about furniture, we talked about grocery. Every product has, it requires a different level of sortation. Above that we also apply new services to customer, right? We talked about next day delivery. We talked about same day delivery installation, all those things when they couple together, the sortation in itself is a huge subject that opens up a lot of complexity to us. So if we push orders to a warehouse and we start processing the orders, the orders get piled up, right? Waiting for a transport connection to arrive at some point in time. And when you see the piling orders and when you design a transport connection, even a piling orders, your transport network looks much more complex. And if you're doing it in an instantaneous manner, it will always lead to a load. It's all in the industry term as LTL. We use load truck load, which reduces our efficiency and cost utilization, significantly down. While we are managing this, we also have implications because of not employing intelligence into this way of working, which is customers screaming at us and cost increasing exponentially. So there was a few snapshots about how customer concerns comes with, right? Most of the customer's problem is where is my order, right? And when you look at this piling network and we say where it is stuck, and if I'm not able to give him a right answer, he's going to scream at me back every now and then, right? The second problem is that most of the time it's also coupled with all the last-minute problems that Aditya talked about, fake attempt and so on and so forth. So managing the entire network is in itself a complex update, right? So then that's the first point I want to do. And second step is that when we step back and look at this entire network, which we talked about connecting all the hundreds of thousands of sellers and along with our warehouse to all the way to the customer. So it's not a push-based model where you get an order, you operate your push into warehouse and process this works. We need to step back and look at what are our push-ins or what are the problem statements that we need to rethink to develop an intelligent module about this. So that's Albert Einstein talking about how important it is to stay with the problem. It's more about thinking about the problem and getting in the right frame of the problem statement much more important than actually getting into the solving. Solving it could be much more simpler when you think about the problem statement of this kind that we talked about. So let me talk about some of the directional problems that we as an organization started thinking about. Step one, for every order, what is a full-fledged model that will be rightly operated? Should I engage a marketplace model here or should I engage into an inventory model? How does this frame of what full-fledged model need to be operated for an order comes out? The second is how should the inventory be reserved in a network? So I have a bunch of warehouses, I have a bunch of sellers, I have a bunch of inventory roaming around in my network. So where should I position my inventory or where should I reserve my inventory for a given order? If I am understanding the inventory reservation part, when should I release the inventory? Should I release immediately like the thing that we saw previously where we saw the inventory get reserved in a warehouse and it can immediately choke into a sortation or it can lead to a very inefficient transportation mechanism. So should the orders be processed in a real-time basis or should we batch it, wait for some information and start processing it? What should be the shipment flow path? So we have a variety of node, for example, if I want to connect from Bangalore to all the way to Guwahati, what should be the path that they should... Should I stop in Kolkata and go all the way to Guwahati or should I employ a transport from all the way from Bangalore to Guwahati or should we fly to Guwahati? There are a lot of options that we have, right? So assessing this shipment flow path is much more important for me. And then across, we talked about the exception that can happen, which is the variability. For example, a customer can say that I am, I don't want this order anymore. Or you can say that I want the order to be delivered to some other day or it could be a climatic condition playing a role in its part where we need to be answerable to customer in a more deterministic manner, right? So how do we take this exception? How do we build insights about this exception and answer the customer query in a more predictable manner? And other things would be is that when should it start a transport connection vendor? Should it start a tug immediately from a Bangalore or from a Bangalore virus to, let's say, on its expedition to Hyderabad? Should it start it now or should it start it later? What should be the transport connection? It opens up a plethora of problem statement along with and other could be is that what kind of flexibilities can I offer to the customer at given point in time or given point in time of the order journey? So we say just adding to that that could be other problems like which is beyond my conversation here. It'll be about forecast, demand planning, supply planning, inventory positioning, inventory planning and so on and so forth, which is not part of this conversation though. So how our journey started? The journey started with building workflows because we are a scaling organization at one point where our orders are increasing at 10x, 3x, 2x and so on and so forth. So one end we are building workflows. We are building our network. We are scaling our network in a very fast manner. So building workflows becomes a critical part, but while we are building workflows, the previous problem statement that we talked about orders getting piled up in a big network and you're getting stuck everywhere and your customers are getting a little bit angry about stuff of product not reaching their warehouse every time, right? So it's about extracting right data. So we've been, so looking at this problem statement, we started from workflows. We want to instrument a lot of information. Instrumentation, some of the instrumentation, we talked about the lost mail, where how a fee is traversing, what is the geo data that we are getting? Apart from that, how are we managing our vendor relationship? What are our vendors performing? How is our sellers performing? What is their operating hours? How do they model their information access? So we get into a, we were in a journey of modeling and a plethora of data across the notes or the factors that we talked about in the supply network. Then building awareness. It's an integrated awareness that become much more internet. Important to us. The integrated awareness is that from all the way from lost mail doing, in a lost mail, if you're allocating an FE, how it replicates and come back to when should an order be released in a warehouse. So that awareness has to be integrated across the network. And that was one of the key pointed. And then employ some control levers in the network, right? Well, if I am able to be awareness, I need to be employ some control levers in the network. And if we need to deliver a product on a particular day, if 30 days, 30 products is its capacity, that is some amount of levers and tools need to be provided to him for him to employ it. Otherwise, whatever that I do as an intelligent part will become shallow. Then the next part is about the building the intelligence itself, right? So we'll talk about more on the intelligence, but the overall goal of this team is first is to become more an orderly and predictable execution. That's a goal that we want to achieve while we solve all these problems. So that we will be able to provide right answers to customer whenever we, whenever customer walk in to walk to us. So now let me think about the logical view of it, right? So how does, how did we go about building such amount of network intelligence? Step one is that we were in stop building workflows. We stepped up and looked at building systems and processes in the, in the, in the, and model systems and process in the, in the language of lead time and throughput. Every processes in the supply chain is modeled as a service entity, as a service entity and above that lead time and throughput of that is constantly defined. And every time there isn't, there is a separate team that works on refining this lead time, making it much more efficient and feeding back to the lead time and increasing capacity part of it. Once we have this active activities and process and lead time in our pocket, the second step is to build this into a customer view. Customer view is the next day delivery. Varo's doing an operation of picking and packing a product. How does it reflect as in terms of next day delivery or same day delivery to a customer? Then about that business and customer intelligence need to be coupled. So what services should be offered to what customers? What business is much more pruned to what services? So it opens up that area for us. And having all these things lead to us in providing right service offerings and pricing to customer. So pricing is a very vast subject. We'll maybe may discuss in subsequent sessions if possible. Here the idea is that having services, understanding customer need, understanding business need will enable us to offer right services with the element of pricing into it. So that leads to customer choices. Now, when you look into the other thing that we talked about is about the events and disruption that can happen across the network. So we need to build a real-time engine which can publish all the events and disruption that can happen across the network. One could be predictable, one could be unpredictable or it could be scheduled like a band or so on and so forth. So all this information comes through us. We started building a logistic inside module which looks at all these disruptions possible in the world and funnel that information to any decision-making layer. So this is one important concept. It's about demand prediction, right? In the context of Flipkart or in the context of Flipkart when we take a decision for a given order, right? It is an opportunity cost for the upcoming orders. So it is not only taking a decision for that particular order. It has to be treated holistically. I can give you an example for example. Let's say we have one product in an inventory and there are two customers walking in at time T1 and T2 and first customer could be a new customer and second customer could be a very loyal customer to us because it's very high on the plus. How do we, if I am taking a decision at T1 for the incoming customer, I would have reserved that order to the customer but may not be the right thing to do. So keeping a demand prediction becomes much more important to us and that's one area where we are investing heavily. All these things lead to generating some real-time fulfillment decisions. In the subsequent slide we talk about or maybe we take a talk one subject and see how the real-time fulfillment decision has been designed. Then once you have a real-time fulfillment decision we also have to operate those levers that we generated in the network by creating an orchestration across the network. So what are orchestration elements that you have? You have arrows, you have people working on the floor or you have your lost mail hubs, your transportation nodes, your partners and all these people are players in the network. And once you take a decision or a plan when it's to create a plan for an order then you need to orchestrate with these players in the node so that this can be, the goal can be achieved. And that's an example of the execution system. The execution system here could be a VAROS management system or it could be a lost mail management system or it could be a transportation management system or it could be all the way to the inventory planning all the way. So this is the area what we call as network intelligence altogether. And the execution system, so as we progress in our journey, in the initial days we were heavy on the execution system, modeling those execution systems and as we scale now and we look at with the power of more data in our hands we, our investment goes heavy on the network intelligence part. So just to give a snapshot of how our systems look like. So there are a lot of building blocks in the entire plethora of FSG technology, which is supply chain technology in Flipkart. You will see a lot of other systems like VAROS and all those things. And also you will also see location platform, location intelligence, network intelligence. We talked about logistics inside. We talked about how to get real-time events and funnel that between that we have scheduled and unscheduled. We have people management, like we have a bunch of people who need to be skilled at various levels. So the skill management itself is a very big subject altogether. So all this information get funneled by various modules and come to a box called real-time fulfillment decision maker. And that's why I called this fulfillment decision cockpit. From there the decision and the plan for an order is created and it comes down through a network orchestrator to all the way across to the execution systems. So there's a high level system, quick view of it. So now let me design what is network intelligence and orchestration means. Given that we have this subject in the background. In a simple sense we have the constraint of our planning system. So what are the constraints? We talked about systems process lead time throughput. Those are the constraints that we hear and being able to plan on knowing this constraint itself is a big subject altogether. The second thing is that the decisions are need to be taken in the subject of visibility of the entire network of all the nodes, how they are moving, what are the things that's happening around the network and also a notion of understanding what capacity they can do. If an FE walks into our hub, how many shipments can he handle? If he's a new FE, he could have handled very less number of shipments or if it's a old FE, he can have much more shipments based on location, et cetera. That's a capacity notion. All this information is much more, it is much needed in order to take a real-time fulfillment. It also goes from iterative planning. There are multiple events and distribution and disruption that's happening either from the customer side or from the supply side. So all these events has to be funneled back to in order to take a right decision. So this defines what is network intelligence in English. Now if you flow the orders again, let's say we have this concept of network intelligence and if you want to flow this orders again, how does it usually look like, right? So when the incoming demand, like the same kind of variation, it can come with a high level of variation. But what we do is that we don't push orders to the arrows. We hold orders. We hold orders and take a decision whether it need to be filled at the real-time manner or it should be fulfilled in a batch manner. This is done along with the batch order is not only with the orders that is accumulated over till time, it also includes all the future orders that can be predicted based on our all changes that we are doing in the network. The second thing is about once we are able to understand the batch, the example of a batch could be is that I pick orders from arrows only for the next transport connection, right? Or I could pick orders from the arrows only for the particular zonal areas where the products are located very close by so that the picker's efficiency increases. So these are the multitude of signals that goes in before deciding a batch. And once a batch is decided, that goes for a planned picking. So the arrows executive knows exactly what he need to pick at what location, right? And when he picks, it forms a story for the entire order path. And the sortation become much more simpler because I'm able to pick the orders in a very sequence manner. It is for a particular sortation plan. If I look at the overall decision that I made, the decision is made for a overall order. The order scenario considers all the factors. So whenever I pick a product, it is going to need for a certain sortation plan that is going to get, the sortation plan gets much more simplified in this thing. And also I can, since I'm taking a holistic decision, the routing becomes much more informed and much more educated and I can always achieve a full truck load whenever I need it, right? The implication is always better customer experience and significant reduction of cost. So that is the story that we are marching towards. And in the next few slides, what we will do is that we will see how a barrows of an order get selected or a batch get formed. That will, so inventory is, so every, so we talked about a bunch of problem statement, what a fulfillment thing, for what a fulfillment system should take and all the subjects in itself is very deep. And we will not have time to cover everything. So I want to talk more about how we do inventory reservation and release in the barrows. So the question is which barrows to select given an order, right? So the first step that we do is that we don't take that point in time decision anymore. We hold order, we also have the orders that is accumulated till time. We also look at what are the future orders comes in. Then we prioritize among these orders. The prioritization comes from all the available barrows and inventory, all the available barrows and the inventory available in this barrows across the seller included. So look at the list of this thing, identify flow pod choices. So given a batch, given this order segments, what are the multiple choices that is possible? So it is only an, it's a divergent problem thinking here. So taking an order, we look at all the choices of barrows, from the choice of barrows, we look at all the choices of flow pod. So it is getting diverged. And then we look at, so every part has a cost as a SLA trade-off. Either I can minimize the cost or I can minimize the time to reach the trade-off across every orders that can be taken. And this orders has to be treated very specifically because one order, so for example, if you have ordered iPhone X in Flipkart, you will demand fast delivery as a customer. Or if you have ordered a T-shirt which could be, which could come in three, four days, you don't wait for a T-shirt to come into your doorstep immediately. So there is an order wise and customer wise, all the parameters that we talked about, the category wise, particular order wise, particular customer wise, gathering this context enables us to set the right priority for an order to take in cost versus SLA trade-off. Then there is a risk in the supply chain, right? So we may have new partners who are new to the system or it could be an evolved partner, they give better SLA, but 50% of times. Or it could be a well-established broader partner and they may give a longer lead time, but with the high reliability. So assessing the risk along with that and finally, before doing an inventory allocation, we also think about what inventory, whether to allocate the inventory right away or not. So one is that the case that we talked about, if my match is prepared for a future, maybe tomorrow, I don't have to allocate right away because I can still get more information and decide the later point in time what need to be allocated. Or it could be the other way around, it's a plus customer and I do not want to derail this experience. I will allocate the inventory right away for that customer. Or it could be pre-orders. Pre-orders is a concept in e-commerce where a book like Harry Potter book getting launched in future. Customers have placed order right away. So I allocate orders right away for them because there's no deviation in his mind in terms of whether we'll get it or not. Now, if this concept lead to VARO selection, how to batch an orders, right? But as usual, we determine the flow path for an order. Now the important thing is that once you determine the flow path, now there are various, there are areas where we have levers to place our transport connection on a network. So how do you balance your transfer? Should we, should from Bangalore, should I operate all the south zone, south destinations in the morning or should I operate in the evening? How do I balance this network? Once I do the balancing of the network from Bangalore, I'm connected to all the way across the country. So if I'm balancing the network, how will I group this network so that the sortations plan can be devised properly? If I'm able to do the sortation plan, now I do a backward propagation across the hall. So starting from the last mail, we think about when the customer, when the product has to reach the customer, then work backwards and figure out what is the latest time I need to release this order inventory into the warehouse. Once I determined that, I also do a forward propagation because releasing at the latest point means there is no queues in the network or no piling up in the network, but it can, any disruption can affect this order much more. And also, if there is no enough demand for a particular day, I am making people, I say tidal, right? So which means that utilization can go down. So one is the backward propagation, then also we do a forward propagation. Forward propagation, the goal of forward propagation is to figure out what is the earliest time I need to release an order into the warehouse, which includes where the goal here is to see how I can maximize the utilization of the current resources that I have, right? And also the minimized risk. So this is the pointers of pointing question to us about how to select a warehouse or how to batch an order in a warehouse. All these things lead to us building custom algorithms. So we have defined a lot of algorithms in every subject. In the next slide, we may talk about one area, what are the parameters going inside it, right? So this has led to us build a lot of custom algorithms in the network intelligence layer. Now, if I think about only modeling warehouse selection as one of the problem statement, what are, how did we go about solving that problem? Maybe talk about what are the parameters lead to selecting a warehouse. Now, if I think about, how will I model this in a real time scenario? First is that everything has a business value. So what are the operating levers here? Either it is speed in terms of delivery SLA or it could be cost in the actual opportunity cost that we are looking at. And it is also the risk involved in managing any of the product across our network. So all this thing has to be modeled to a common business value, right? An organization may take a step up, increasing the speed so that it increases and increases GMB or reduce costs so that I increase my CEM, right? All this parameter. So the business value is a point in time or a point in time strategic decision where the prioritization of any of these things happen and come to a common unit of business value. The objective function is to maximize the value of a batch of orders. If we're creating a batch of orders, it has a maximum value. The parameters that we consider is that for example, if an order is given an order, what is the cost curve of an order? If it flow through multiple paths, if I decide a path, what is the cost of serving the path? If I don't decide then if I take an order and if I ask the order to wait for some more time, what is the cost of losing that order or the future value of the order? So that is a cost curve of an order that we determine. And then cost of serving an order determined from the common one is that I can delay the order as much as possible so that I can increase, I can reduce the cost of serving this order. But there's a probability that the order can get canceled or it can derail some customer experience. The other end of this spectrum is that I can go only looking at order and look at the minimum cost of serving the order. Now there's a commonality between these two points and that determines our minimum cost of serving an order. The opportunity cost versus actually serving that order, that commonality comes as a cost of an order or flow path. Once we identify this cost of the flow path, we start modeling the risks associated with this. Then the second two steps are much more simpler, identification of path, which is just an orchestration of path and identification of arrows, which is again following the simple plan. So this is how we took, so just to summarize some of the conversations that we had, right? We talked about the landscape of e-commerce, we talked about different products that get operated. We also talked about the supply nuances, what is supply chain network means. We also talked about what are the, what is the problem with scale when we don't have an intelligent network orchestration. So we also defined what is the network intelligence and how did we, how a overall network intelligence has a concept sits into overall scheme of things in the technology stack. Then we also looked at a lot of problems that need to be addressed before serving an order. We zoomed into one particular problem statement, which is arrow selection as such. And then we talked about what are the parameters that gets into before selecting an arrows, right? So that's it from my side folks. So my question is slightly tangential, but I feel like you're the right person to ask. Is that in terms of the whole supply chain that you described, there's also the key element of people management that you touched upon at some point. Now, my concern or the question is that there's regulatory compliance in terms of labor laws and there's also a human resource need in terms of making sure that all of the people working for you are in a good state to work. So at a very basic simplistic level, if you could tell me how many hours are FE is required to work and how do you cap it? Like what drives the decision and what other kinds of people or human centered constraints are put into consideration? So let me start with the first problem statement that you mentioned about the human constraint that we face. So our arrows are kept at very different locations. It's not part of the center of the city. So it's all outside the city just because of the space cost that is involved in it. So everywhere for everywhere also that is operating outside. There is a working timing that need to be defined as per the compliance law. It cannot be 24 bar seven. If it is 24 bar seven, there is a rate the salary increase has to be significantly higher for the remaining part of the day. And also there should be transportation solutions should be there for this employees to go up and down. And second thing is about the safety nature includes employing medication facilities, medical facilities near the large varos that we operate. So that and also other laws applies in a virus scenario for example, that could be recreational facilities for a longer continuous work time. We not a law, but it is our way of encouraging to reduce the fatigue of this employees. We also have recreational facilities and other things that become considered the part of the layouting of a varos itself. Example of a varos. The second is about when you look at the employee timing in the logic side, it's a usual eight hour shift that we manage. We have a eight hour shift and there is a OT over time paid for the remaining two to three hours that we extend the thing as per the laws defined by various geographies of India. So we follow that thing. Like how do you measure an FE's efficiency for example? So given time, so it's about not about the number of products that we deliver in a particular day. It's about the number of products you can deliver given the time period of a day and also it was about the geographical nuances. For example, as our system knows based on the heuristics, we know that FE can deliver n number of products in a particular geographical locality given the address and given the location of these addresses. And then we use that as a metric to measure the productivity of the day. Not the other way. And relating it back to the mornings, talk about address failure and geolocation issues. Whenever there is technology failure in that sense, how does that fall upon the FE or how much does their feedback count and how do you improve upon it or what kind of cost does an FE bear for infrastructure failure? I may not be best question. Aditya, you want to take this question or should I go ahead? Hi, my question is on, you spoke a little bit about modeling risk. So I wanted to ask on what kind of risks are you looking at? And if you can give some detail on how do you attempt to model it? So the risk are twofold to start with. One is a product risk itself. So if I am taking a high value product, let's say it's iPhone X and for some reason the information is available that it's an iPhone X. So it induces any of the person who is handling the parcel to take his hands on it. So there's a risk on the product itself. The risk on the product is also under the second dimension of the risk is about the people who is operating that product. So if I have a high value product, let's say I'm operating jewelry for a hypothetical sake and I'm doing a diamond jewelry order, right? There is a significant risk involved in moving this gold item or this jewelry item from one point to another point. So who are the parties that need to handle this product? What are the delivery nodes it has to or what are the nodes it has to go through? It is based on the risk assessment of the subjects like people as well as the facilities, the risk assessment of that. The second part is about the risk of and people are a facility if you look into it from a risk perspective of a people, we know the history of people. We don't, so we have a background verification check across all of our executives. We also know the tenure of the executives. What are the remarks that he got from different transaction that he dealt with? So that information is all instrumented using those information we extrapolate and figure out what is the risk of this person handling this product, right? The second is that about the facility or the nature of this location itself, right? Should I, if you come to, for say, various geographic location, we don't deliver this product. High value products are not delivered to various geography level. There are pockets that we identify where we will not deliver the product beyond the cutoff, beyond the value of a product. We don't deliver the product because the risk on that area itself is very high where people know Flipkart employees, where they go and vandalize employees and collect the shipments. We understand those pockets and we keep feeding into that real-time engine that I talked about. We store this information to take any risk-related assessment. Cool, we're completely out of time now. So we'll break for lunch here and we'll reconvene at 2.10. So there are a couple of talks by Vahan, Vahan.ai and Swee and there's again another buff, pretty interesting I would guess on operation watch stories in between the two talks. So see you again at 2.10. Oh yeah, so there are firms near the exit, the feedback firms. So you can actually comment on the quality of talks and any feedback that you want to give. They are on the trace near the exit. Please fill them. Hello, hello. Wait for, can everybody make themselves comfortable? Cool, so I hope everybody had a good lunch and hopefully you're not feeling sleepy. We'll kick off this post-lunch session with the first talk on predicting order batchability by Sunil Rathee, who is a senior data scientist at Swiggy. So what do you Sunil? Afternoon everyone. Can we start? Myself, Sunil Rathee. I'm working with Swiggy as a senior data scientist. Today we are going to discuss predicting order batchability. So this is the flow of the presentation. We'll start with the important terms. These are important to be aligned with the presentation. Then we'll jump to the introduction, batching, traditional order flow, what should be the next version of order flow, machine learning batching prediction algorithms, first order and second order, then our machine learning model and then the experiment. Batching. So batching simply means to deliver two or more than two orders in a single trip by a single delivery executive. I think you all must have experience over poor all us here is a similar concept. First order, say you have n orders in your batch. If you sort all the orders by the order time, then the order which have the minimum order time will be the first order. And all the rest of the orders we'll call as second order. Predictive service level agreement or PSLA. So this is the estimated time which conveyed to the customer that how much time this delivery will take. Service level agreement or SLA is the actual time taken by the delivery to deliver the food. Compliant orders. Compliant orders means that where we meet our promises. Basically, where the actual SLA is less than or equals to the predicted SLA. Net promoter score or NPS is a business metric that we generally use to track how the business is going on. This is from the customer feedback. So before we start the presentation, let's first discuss why do we even need it. Any, every company is doing it. Patching any, all a share is doing, Uber is doing and rest of the companies are also doing it. But why do we need this prediction? I will tell you a short story so that you will be better aligned with the presentation that why are we doing it? This big why? So I have a friend, Jagmeet, who generally orders food from Swiggy. As usual, one day, he was hungry. He went to the Swiggy platform. He went to his favorite restaurant. He picked up the items, added to the cart. We showed the predicted SLA that we discussed. He was happy with it. We showed it 33 minutes. He placed the orders by looking at that. This is 33 minutes. Okay, I'm fine with it. Everything was fine. But the order was in peak hours and that too in dinner peak. At this time, we have a lot of orders. So to handle load at peak hours, we generally batch the orders such that they made the necessary conditions and customer experience doesn't hamper. So Jagmeet's order got batched with order O2, say from customer C2. Now the delivery executive has to wait extra to pick up the second order. He went ahead to deliver the C2 food first rather than the Jagmeet's food and then he moved ahead and then he delivered to the Jagmeet. So overall in all these operations, it took something around 46 minutes. We promised 33 minutes. We were unable to meet our promise. We felt bad. We apologized to the customer. We took all the actions which we generally take when we don't meet the promises. Next day, we were sitting together as a data science team. We were brainstorming that why are we not able to meet the promise? Then we looked at the data for the batched orders and non-batched orders and we find that this is happening more frequently with batched orders if you compare with the non-batched orders. If you ask the questions within the team, why are we not able to meet the promise? So there was a simple answer because at the time of prediction, we don't know that this order is going to batch in future. Fair answer, right answer. But is it a solution? No. So let's go through that how we build the solution. So let's compare a normal load scenario. So here we have NFDs in comparison to the number of orders is a normal load. So delivery executive can pick up a single order and he can deliver it to the customer. But what will happen at high load? At a high load, there will be delivery executives who are busy in delivering the food. So these two guys are busy. Now just consider simple scenario that there are two orders. Okay, let's first compare with the... So we know the pigeonhole principle, right? So it says that if you have M buckets and N items and number of items are greater than number of buckets then some buckets will have more than one item. Same here. We have less number of delivery executives in comparison to the number of orders. So some delivery executives has to deliver more than one order in a single trip. And that is batching. So this delivery guy has to pick up the orders from R1 and R2 restaurant and that he has to deliver to the customer C1 and C2. So we discussed what is batching. But why are we doing it? So there should be some bugs. So we discussed that it handles load at the peak hours. It gives a better utilization and payout to the delivery executives. So if the order are best, the simplest scenario, there are two orders in a batch and in a general case, if you're paying something like 50 rupees then for the same trip, he's getting extra money with the cost of extra time, little extra time that he has to move from customer C1 to customer C2. And if it's a multi-restaurant then R1 to R2 also. It's a small time in comparison to total trip. Now for the business side, it reduces costs per delivery. So if the orders are best, so say for first order you are paying 50 rupees and you said that if the order gets batched with this order, you will pay something like 30 rupees or 40 rupees, say 40 rupees. Now per delivery if the order are best, then you are saving 10 rupees. So it's a win-win situation for both business as well as delivery executives. It increases speed for second order. So at high load, what is the problem that we have less number of delivery executives in comparison to number of orders? So if we don't do batching then the order will wait at the restaurant until the delivery executives deliver the food to the customer and then come back. It took a lot of time. If you just batch it, then he will just pick it up with the cost of the preparation time of the second order that can be small. So overall it increases the speed of the second order. Let's look at the traditional order flow. So when you choose some items and add it to the cart, you are at cart where we predict the SLA. We predict the delivery time. So now if the customer is happy, he will place the order. Now when this order comes to our environment, we'll check that whether this order can be batched with some existing order or not. If it is, then we will assign a same delivery executive, else we will assign a new delivery executive. And then it follows the rest of the steps. So if you focus closely, this prediction of the SLA is before batching. Same Jagmeet story we discussed, right? The answer was simple, that we don't know at the time of prediction that this order is going to batch. So now look at the numbers that will tell you that where are we going wrong? So after deploying the batching thing, after a few weeks we compared the numbers between batched orders and non-batched orders. So the compliance of batched orders was 20% less than the non-batched orders. Batching was a major contributor to the bad orders. So bad orders are the orders whose delivery time is more than an hour. There were higher chances of customer to connect to the customer care because you are not able to meet your promise just after the estimated time they will connect to the customer care to ask, where is my order? So we looked at the numbers, how it can impact the business. First thing is repeat. If you are not able to meet your promise, then the customer will not be happy. And he or she will not order food as frequently as a happy customer. So the repeat rate can change. It can decrease and pierce. So when we compared the numbers between batched and non-batched orders, there was a 16 point difference between these. So with all the perks of the batching, still the business was going in the negative direction because of the customer experience because we are not able to meet the promises. Customer churn. If you keep on breaking your promises, there can come a point at which customer can look for the other platform. You can lose your loyal customers. So in a polite way, I will just say that we should respect our promises. So we looked at the previous version of the order flow and we found that there are some issues even with the perks of batching. So how should we do it? Now let's look at. So when we add the cart, we will check whether this current order can be batched with existing order or not. If it is, then it becomes a second order. So second order batching prediction will handle it. This prediction thing. If it is, then we know the environment with which order it is going to batch and then we can predict the right asset. If it doesn't batch with the existing order, then it becomes the first order. And here comes the rule of first order batching prediction. Our core machine learning algorithm. It will predict that whether the future batching is possible for this order or not. Because there is no other order in the current system right now that this can batch. So is future batching possible or not? If it is, then we will recalculate the SLA. Else we will throw the same SLA as predicted by our time prediction algorithm. So in the context of batching predictions, we handle differently the second order in the first order because they have different behaviors. Because second order, we already know the password with which order it will be batched with first order we don't know. So let's first discuss the second order batching prediction because that is much more trivial than the first order batching prediction. Second order batching prediction. So when the order is at the cart, you have all the information about your environment, all the orders, all the batches. Now with this current order, you can check whether this order can be batched with the existing order or not by turning all your conditions. If it can be batched, then you know that with which order this will be batched. So you write at this point, you know all the information that what will be the customer to customer distance, what will be the restaurant to restaurant distance, what will be the wait time, so all these properties. So you can use these information. You will pass this information to your time prediction algorithms and you can calculate the right SLA. If it can't be batched with the existing order, then you will throw it to the first order batching prediction. First order batching prediction. So now the order didn't batch with the existing order. It can batch with some future order. How will you identify that there can be a possible future order that will meet all the necessary conditions and it will batch with this order? So first order batching prediction mainly depends upon demand prediction, batching conditions, order level predictions and historical features. We will go through one by one. Before that, let's discuss that what is geo hash because some of these, some features are dependent on this. So we'll quickly go through geo hash. So geo hash is a public domain geo coding system. What it does is that taking a space, it will divide into grids and keep on dividing into it. So it shows that you will get higher precision. So these are some of the precision to which you can work on. So we are basically working on the precision six that is six, 10 by six, 10 meter for the first version of our first order batching prediction algorithm. So geo hash is just a string. So higher the length of the string, the higher the precision or smaller the size. Demand prediction. We have an order at cart. We want to identify that can there be a second order that can be batched with this order? So we are now depending upon future. We have to predict the future. So from the demand perspective, demand prediction perspective, we need to find restaurant future order. So this order comes at restaurant R1. So can there be a future orders for this R1? So our demand prediction algorithm will tell you that say that this order comes at time T, between this time T to T plus delta T, can there be future orders? If there are good number of future orders, then there are higher chances of batching. So this delta T you have to set. According to customer experience, you have to look at the data. This can be 10 minutes, 15 minutes. It depends on the predicted SLA of the order also. If it is only 10 minutes, you can say that for this order, we can deliver it in 25 minutes. So you can increase it by 15 minutes, something like that. Same for customer geo hash. So if you can identify that there will be good number of future orders, you can find the order density for that particular geo hash. Now we already discussed that we are working on the geo hash of 610 by 610 meters. So if the order comes from that geo hash, then it can meet the one of the necessary conditions that customer to customer distance. We'll discuss about it. What is customer to customer distance? Then customer geo hash to restaurant future orders. So from particular geo hash to particular restaurant, what will be the future orders? So if it's a good number, now customer to customer distance will be small, restaurant to restaurant distance will also be small. So there will be high chances of batching, batching conditions. So these are basically put up by the business or operations, such that we have a better customer experience. So it can vary from geography to geography. So what can work in say Bangalore, will not work in Ambala and vice-versa. So major contributions are from customer to customer distance. So customer to customer distance is that how far two customers will be such that their orders can be best. So if you say that a customer to customer distance is one kilometer, that means two customers will be, can be at most one kilometer. Max order delivery time. So this is the maximum time that should be taken by a batch, by all the orders in that batch. So if you said say something like 55 minutes, then all the order should be best in this 55 minutes. Sorry, should be delivered in 55 minutes. So this max order delivery time, customer to customer, you have to look at the data, that what is the sweet spot? What users said? It should be different for the high load. It should be different for the normal load. So it can, normally you can choose something like where the customer experience is right. So you can choose something like 45 minutes. Then there are item level conditions. This is basically like for some particular, some items can't be best. If you take an example of ice cream, if you batch that, then it can be meltdown. So for these particular items, you have to treat differently. Order level predictions. So now we know that we have a current order at the cart. For this order, we can find the necessary properties, like what will the prep time, preparation time, what will be the predictor SLA? So by using all this information, we can identify some of the batching properties because this is part of the batch. The major contribution is something from order preparation time. So same example, if it's ice cream, you'll just give you a scoop and then you can have it. So it's preparation time is less than a minute. So within this time window, within this minute, there are less chances of batching because the future orders will not be much. Order service level agreement that is predicted SLA. So if you set the maximum delivery time, say at 55 minutes, if this order, PSLA is something around say 52 minutes, then there are less chances because if you batch another order, there are chances that you'll not meet the necessary condition. Part of it is item level constraints. So these are again, basically, specifically for the items, like pizza can be batched only with pizza and something like that. Ice cream can't be batched. Ice cream can only be batched with the ice cream or it should be become the second order, then only it can be batched. So these are some of the conditions. Historical features, the history tells you about future. So if you can look at the history, you can identify that what was going on so that you can predict something about the future. So restaurant batching ratio. So if you know that a particular restaurant is popular, in this order bucket for this particular time interval, by looking at the some few weeks back or some days back or months back, then you can some guess that this restaurant can have a high batching ratio at this time. If you just compare that for Biryani, people generally orders at the dinner peaks. So restaurants who are serving Biryanis have high restaurant batching ratio for that dinner peaks. Same for customer G-Hash. So for customer G-Hash batching ratio, you can take an example. So say you had a customer lead long, from that you can find the G-Hash. Now if this G-Hash is falling into office complex area, then there are higher chances that this will be batched in lunch peaks rather than the dinner peaks because they will move to the residential areas to their home. Then customer past experience. We checked that what was the experience of the customer in the previous five for 10 deliveries. If we were not able to meet the promises, then our batching prediction algorithm will handle it by saying that you shouldn't bear this, assign a new delivery executive, let's give a customer a really good experience. So this slide is specifically for data scientists. So we experimented with different models, random fog boostings. We come up with the deep learning model. So the highlights of the model are basically like we had the data points of around 11 million. Features used was something on 58, which we already discussed and some extra features from our platform. Number layers used was eight. We used value as the activation function for hidden layers. Adam as the optimizer and the loss function was binary cross entropy. We checked the results and the results was good. So overall for the first version, it was precision 90% recall, 84% and F1 score 86%. So we were happy with the results. We deployed this model to some of the zones in Hyderabad and yeah, so for this version, we didn't do any change from the time prediction algorithm side. What we do is that say your time prediction algorithm for this current order is predicting say 30 minutes. Now your batching prediction algorithm is also saying that this order can be batched in future. So we'll just increase this by 10 minutes. So in spite of saying 30 minutes, we'll say to the customer that it will take 40 minutes. We integrated this model using a data science platform with our internal data science platform. So the initial question from the business side was, now you are increasing 10 minutes at the cart. So there are chances that there will be conversion drop. We told them that we are not increasing 10 minutes for every order. These are the highly probable orders that can be batched in future that we have to increase 10 minutes because we will not be able to meet promise. This will be a small percentage. They gave us a go ahead. We deployed it in Hyderabad some zones. We found that there was no drop in conversion because people can understand that this is a dinner peak. This is a lunch peak. It generally takes something this much time. Even when we were saying that it will take less time, they didn't know that it will take a lot of time. So when we are promising, there was no conversion drop. Compliance for batched orders improved by 15% because of better promises now. In spite of PSLA, for these batched orders, we are saying PSLA plus 10. The NPS and customer chain exercises are going on and we expect the results in some few weeks. When we get it, we can share with the community. We already have a second version of this model which have better results. And we are expecting that this can give us a jump of 10% additionally. We will deploy this in one or two weeks next. So that's all from my side. I'm open for questions. I think no one sleeps. How do you gather the data on like restaurant delivery and food prep behavior? Like how- So we are- And also item-wise, yeah. So writer is not item-wise, it's on the order-wise. So we get the items as a feature for our order and we get the preparation time for the order. So we have the in-house machine learning algorithms that predict these preparation time. If we already have the data, right? So we have the prep signal marked for ready from the restaurant side. So we know that how much time it took from the history. Now we can use this data to predict that what will happen in the future. So we told the restaurants that you are and the delivery executive that whenever you are picking up the order, you have to mark it. This data is going into our database and then we are using it. So for this new restaurant, we have the cuisine properties like. So for this restaurant, say we have, it will fall into one of the cluster and from that we can identify it. Trying to get interesting. Devans is for you. You have those 30 minutes and three orders, right? Or 30 minutes of free orders. So I ordered once. I felt like it came in like 31, 32 minutes. And I got a notification, your order was delivered within the 30 minutes. So next time I timed it. It's on the reach, not on the delivery. What is that? So say if you're living in a complex, then the first entry gate. But I'm living in an independent house. Then it's still with you. And next time I started a stopwatch, it came in 31 minutes and it was still not free. We can look into your issue. Actually, I'm at the other end of the spectrum. So I ordered a 30 minute delivery once and I got it free both times. We'll also look into you. We are mentioning about giving degraded SLA. Slightly degraded SLA for places where you kind of have to batch the orders. You told me that you did not see any drop-off in terms of the customers. That was a drop. Okay, there's slight drop. But did you try incentivizing those customers to probably- We are thinking about this experiment that if we know that this order can be batched in future, we will give some incentive that some discount. We haven't tried it out yet. We are doing this experiment. It's on the POC. Okay. Hi, one question here. So you were talking about increasing the promise time to your customers for orders that you're applying to batch. Have you considered the possibility of offering it as a new service offering for customers? Where in I would, for orders that you're willing to wait longer, I would leave a lesser delivery charge, no surge and so on. So that exercise also we are doing it. In couple of months, you will see that we will get some incentives to the customer who was ready for say one hour delivery or something like that. Yeah, hi. One more question. A key component of your SLA also involves the restaurant keeping their part of the deal that they also meet their SLA. Yes. But they might be on multiple platforms. They might have their own customers who just walked into the restaurant. And if the number of chefs, for example, for that day is less, so how do you account for that? So operations guys keep on discussing with these restaurants to tell them that if you get the signal by, you should admire the prep time which we are proposing to you. So we already discussed as Mark put ready some kind of thing. So we actually penalize the restaurants we didn't follow our instructions on the listing side you can see. Hi. So I wanted to ask a question about the machine learning aspect of your model. So you talked about the model parameters. I wanted to ask what kind of models did you try and how did you arrive to the current model that this is the best that works for you? So we experimented with random forest boosting algorithm deep learning model. As we have a lot of data, we have collected off for say around three months. Deep learning model was working fine with a good set of parameters. The parameters also we fine tuned with different set of parameters. We're closing for questions actually. You can ask me off. Thank you. Thank you Sunil. So the next session is a boss session on operations watch stories. And compared to the earlier boss, this is going to be more participative because hello. So the next talk is by Piyush Makhija who is a NLP engineer at Vahand.ai and he's going to talk about as displayed on the screen virtual assistant for hiring last mile workforce as in our last section above, there is a big human element to all of operations, right? That got emphasized. And I think Piyush will talk a little bit more about that. So what do you Piyush? I'll start. Hello all. My name is Piyush Makhija. I'm a NLP engineer at Vahand. At Vahand we are working on the problem of automating and scaling. High volume recruitment of frontline workforce for e-commerce and logistic companies. To give you an idea of why we are even required, please look at some of these articles that have come out in recent times and many more. So the on demand and e-commerce industry has created a new job market for last mile delivery teams. Let me ask you a question. How many of you guys have ordered food or shopped online in the last month? In the last week, is there anybody who has never used any e-commerce or online delivery service? The point I'm trying to make is that these services have integrated so much into our daily lifestyle that they have almost become a necessity. In some sense, the delivery guy that brings your products on your demand to your door is as close as we can get to a modern day Santa Claus. And the requirement or demand for these Santa Claus is much higher than the supply. To give some metrics to this, there is currently a 500,000 delivery boy demand in India, which is increasing at a 30% year on year rate. And most of the on demand or logistic companies are scaling their operations heavily. To give you an example, Danzo is increasing their frontline workforce by 10 times in just second half of 2018. So frontline workforce, blue collar workers or last mile delivery, all refer to the same stuff. So some insight into this particular kind of recruitment for last mile deliveries was a standard recruitment. The demand for per month hires is somewhere around 500 to 10,000, depending upon your logistics operations, the scale of your operations. If you feel that this particular number, 10,000 is high, Flipkart hadn't requirement of additional 60,000 delivery boys just for their Diwali sales. There's a higher attrition rate, somewhere around 75%. And this attrition happens in about two to three months. While in standard recruitment, one to 3% attrition happens over a year. The skill requirement for this job is low. So there's little barrier to entry. Anybody can start this job. And it's mostly customer facing. So the people that you hire actually affect your brand image. The hiring process is mostly manual and unorganized. It's mostly person to person interaction over telecalling and highly unorganized. The hiring demand is very seasonal and sporadic. Something like there's a high requirement during Diwali sale or IPL season. So what we understand is that particular job space is very fragmented, unorganized, and really hard to scale. Formalizing the problem statement that we are dealing with a little bit. The problem is to build an efficient and scalable system for a high volume recruitment of last mile workforce. The design considerations that you need to take into account while building a solution for this particular problem are as follows. Firstly, you need to reduce the manual effort by bringing in some automation to either the complete process or at least parts of the process. This point is to reduce the manual labor. Next, you need a one stop solution. You need one solution that accounts for your size and nature of logistics operation that can work across verticals and something that works at scale. So if my requirement is to hire 500 delivery boys this month and 10,000 delivery boys due to some sale in the next month, my solution should be able to reach that demand. We need to account for high churn rate and attrition. So high churn rate and attrition are inherent nature of this particular job segment. So we need to either circumvent that or account that into our calculations. There is no LinkedIn for blue collar workers. By this, I mean there's no social or professional platform where these users are these the people in the last mile workforce can connect professionally with themselves as a community or with the recruiter or build your career path. And when building a solution, we generally think, okay, let's make an app for that. But building an app has its own challenge. The first and foremost is the adoption challenge. Will people use this particular application? Will it be useful enough that they keep it on their mobile devices? The particular nature of the recruitment problem is that people are always going to uninstall the app once their work is done. Once you get them hired, they are going to uninstall it. It's not a daily use application. So these are the design considerations you need to take into account while working around this problem. At one taking this into account, we came up with our solution of automating jobseeker engagement on existing messaging platforms. Something like you build an account on a platform and where users can interact with you. So this number is a dummy number. This is not an actual number that you can call to. This is just to illustrate my point. So how did we address the design considerations? Let's talk some somewhere around those lines. The first is circumventing the adoption challenge. This is the biggest issue. So we circumvent the adoption challenge by going on existing platforms like Facebook Messenger or WhatsApp. So the adoption rate for WhatsApp or Facebook Messenger is 97% for smartphone users in India and most of these users use these applications on a daily basis. So we have a way to reach our audience or reach our users. Then we need to automate certain aspects of the process. The first is job pitch. Then you need to gauge the user interest. You need to screen the user and get them to the walk-in and we do this via a virtual assistant. Now a virtual assistant can be IVR based voice based or text based. But with IVR based assistance, the problem is that you have a very limited questions that you can ask and a very limited kind of responses that you can capture. The person on the other end cannot interact with you or cannot query you. Hence due to these limitations, it's not a very effective way to go forward. Next, a voice based virtual assistant. The problem here is that you need to respond to the user in real time and with the data bandwidth issues and background noise issues coupled with ineffectiveness or errors introduced due to speech to text and text to speech. It doesn't seem like the perfect way to go. However, a text based assistant something that our users are already accustomed to using something like WhatsApp or Facebook messenger where they are daily engaging with each other via a texting platform seems like a right way to go furthermore. We need to avoid hopping across different channels through the recruitment process to give you an insight into the current recruitment process. First you need to generate the leads and you do this via advertisements on classified snorkely quicker and whatnot. Then you process these leads on Excel sheets and in some cases even on paper. You need to engage with these users that you have processed and mostly it's done via telecalling. And finally you get them to your onsite screen the candidate get them onboarded. So there's a lot of manual processes involved and all of these are disconnected with each other. We need to bring all of these processes on one solution. Next, since it's a text based chatbot that we're trying to build, we can scale and interact with thousands of users at any instant and we can counter churn. We counter churn by increasing the top of the funnel. This comes from the fact that we are highly adoptable. We have gotten automation and we can scale and finally we can break build rich user profiles and lay the foundation of a social or professional platform where users can have a career path and recruiters can connect with them. So based on the design considerations, the thing that I'm talking about is actually a goal oriented dialogue system. We need to carry out conversations with our user to complete certain tasks and reach the final goal. The goal here being that we need to qualify the user for a given job and the task that we have are to pitch the job, gauge the user's interest, screen the candidate and get them to the walk-in. These are the tasks we are automating for in our solution. So a few key steps to our solution. So in our overall overall flow, when we post a screening question to the user something like do you have a bike? We provide them with option choices something like yes and no a valid response, but the user may in turn choose to reply with the text office on which can be anything. It can be a query or it can be an indication of interest in something. So the first step here is to comprehend the user's text. We need to understand the words that are provided in the text input and the problem here is there is high variation in the user input. For example, just the word yes can be expressed in more than 20 ways in English language that you and I can understand via texting platforms. Now imagine this being said in Hindi, Canada or Telugu. We handle this via data normalization. That is we map all of these possible utterances to a word. Yes, we convert the input of different variations of yes to the single word. Yes, and we do this via machine translation. Now that we have proper words, we need to understand what the user meant when he wrote that text and it can be a question on what is the job salary? What is the job timings? Where is this job or it can be an indication that I'm not interested in this job or I want to call you or it may be a greeting. I good morning or whatnot. So we handle this by intent classification or our intent detection system. And once we have understood what the user wants from us responding to the user is the easier part while building any solution with machine learning, your model is only as good as your data. So we need to understand the kind of data that we have. What kind of utterances user is giving out and the first problem here is that user mixes multiple languages. When you and I speak, we mix the languages that we know. So I am a Hindi and English speaker. I'll mix those languages while texting or while speaking out even something like call Marty, which is a English word combined with the Canada word. If I say this to somebody in Bangalore, they'll understand. But if I say this to somebody in Delhi, they won't understand what I'm going to say or what I'm meaning with it, which brings to my next point. This high variability in data introduced by a regional and cultural influences. Another example of this is yes, a native Karnataka speaker when he says yes, there's heavy emphasis on the S part something like s and the user types in that way. The last example. So we need to account for all these intricacies of our texting that we do in normal cases. And to do this, we need to train a good machine learning model and to train a good machine learning model. We need a comprehensive data set that accounts for all these variability and all these data sets or all these variations in data, but the problem is there is no good public domain data set available for such an informal language. The technical term for such informal languages is code mixed languages and it's still an upcoming research challenge. Hence, we had to invest heavily in the data collection process before building our own models. So the data collection exercises that we went through was follows we introduced the jokes and daily code spot because this is something that gets a lot of traction on WhatsApp and Facebook Messenger, right? So we entered groups where we put our bot and we send the users jokes and daily codes. It seemed like a reasonable approach, but the problem was that we were receiving a lot of spam. We did not receive a lot of useful data. Hence we moved to our next solution. We built an English translation bot. The user would provide us an input and we would do a Google translate API call and provide the English response. The problem was the usage was very sporadic. In some weeks we used to get a lot of data. In some weeks we grow out practically nothing. And this was not a good way to collect data. We moved on to an English learning bot. Users would interact with us and we teach them basic English. It seemed relevant to a use case or the kind of stuff or the kind of data that we wanted to collect. But the content was very narrow. We quickly understood that with this solution, we won't be able to build a comprehensive data set. Hence we abandoned this approach. And finally we struck gold with a friend finder bot. The inspiration is from chat relate. The idea is that you connect to users on Facebook messenger in an anonymous chat. Let them talk about whatever they want and we collect the data about what they talk. This particular bot was very popular and we were able to collect around two million messages per month with this particular solution. So once we saw the data collection challenge, we could get the data labeled and build our own models. So in the machine learning challenges, the first and foremost is that we handle. We had to handle nuances of the code mix data that we were dealing with a few examples of those. I can give out something like hi, I have a bike and may pass bike here. Just notice this utterance or variation of high the same utterance or the same letter sequence is being used. In this utterance and this utterance the utterance Miripaz bike here is basically Hindi for I have bike and based on the context this HII is getting translated to English high which is a greeting and HII which is Hindi for I have. So we need to account or take into account the context of the conversation or the utterance while translating in the entire classification part when we ask users a question like screening question like do you have driving license and provide valid responses? The user gives out utterance. I have LL only in this particular domain in this particular use case LL means learner license and the user will respond like this only LL or sometimes LL R. So our text normalizing model has to understand and expand this to a learning license. Then our intent classification model has to understand. Okay, the user says he has learning license, which means it does not have a driving license. That means his response is leaning towards no. Once we have understood this intent, we can respond that. Okay, get back to us when you get a driving license. A few more examples of a code mixed data set something like salary kithna rahega which is make mixing English and Hindi words in Roman script. We are able to normalize it. We are able to understand the users intent for a query of salary information and respond to it. So a few machine learning metrics that we have in the interest of saving time. I'm not going to cover all these metrics. I'm just going to say our neural machine translation method work much better than the statistical machine translation and our intent classification system has the coverage of 91%. The business metrics of this is that we are currently driving about 8000 hires for all of our customers. We are able to do this within five to seven interactions with the user and these interactions typically happen within 10 minutes time frame by that we understand that the user can if the user finishes the flow quickly is much more interested in the data or this particular job. The current ongoing machine learning improvement efforts at one are focused on getting more and more data and improving our models and making it much more better for the particular domain specific data. We are trying out new architectures and we are also looking into dialogue systems to make our conversations more human like a few key takeaways is that there's urgent need for a high volume recruitment solution. We need to build a social and professional platform for these users to connect and build their professional profiles. We need to build a solution or whatever solution we build for this particular problem has to be easy enough for these people to adopt and it should have some daily usage aspect to it with the machine learning challenge. There's a need for more research and development on code mix data sets and first and foremost in that is to formulate strategies to collect this data so that we can build better solutions for the Indian context. And finally we need personalization for the regional persona of the user so that we can take into account how the user or the person speaks or how he's more comfortable to interact in the broader term. The vision we have at one is that we just don't want to do the recruitment we want to help these people own their skills train them assess them and become a holistic platform for professional growth. In the short term with the kind of traction we are getting we feel that in the near future when your modern day Santa Claus comes to you bearing gifts that guy would most likely be onboarded or recruited via our platform. Thank you. Any questions? Yeah, yeah. So the two million messages data set which you received through the friend finder. How did you process that data? Can you give a brief about that? So this is raw data or raw text that we are receiving from the user and we use this data to build a comprehensive data set of the words that the user speaks. This is a very generic data set. So it will have utterances that do not deal with any domain. It can be something about interpersonal interaction. It can be something about traveling to somewhere. So we have this data set and we use it for data normalization. Basically like I gave the example of yes. If the person says yeah or yes with triple s is we need to label it as why is so we have our own annotation team that does that stuff. Hi, I'm Pramod from load share. My question is about how you went about the process of getting WhatsApp integration done and what is the price point around which this works? Any pointers for someone wanting to do WhatsApp integration? So I'm definitely not the best person to ask this question. I this is basically more operations and business perspective of the stuff and I do not particularly deal with this. But maybe I can connect with you or you can connect with us at info.com.ai and that you should get a response. So we have different sources from where we get leads. Sometimes the customer itself provides the leads and sometimes we are generating these leads by ourselves. We have our own referral system also that contributes a lot to it. So 2 million messages per month we were receiving. So I'm not sure exactly how many users they were, but we were like trending so much that we were in the top five apps in the region. Maybe around top three apps. In this India like South India region. Yeah, one last question. Under what pretense did the users with the friend finder app? What did they agree to let the messages be used for training your. Yeah, so basically when the Facebook has those service messages that you are going to let this thing be happen. So they'll get that indication that your messages will be connected. It's going to be an ominous chat and so on. Thanks. Thanks. Piyush. Thanks for a great presentation. We'll now take a break for now with that now now we'll take a break for evening tea and will be slightly this works. Let's start off the post the final session or the final set of sessions. We have Vibhav Khandelwal who is co-founder and CTO at Shadowfacts and he's going to talk about product development for fleet management. You ready? Yeah, so let's go. But I wait for some time people are entering in. You can start. Good evening everyone. Thank you so much for spending your Saturday for this session. It has been it is my first session to such an audience. So please do pardon if I do some silly mistakes or common unknown mistakes, but I'll be happy to hear your feedback. Happy to improve and hopefully happy to present again. So a bit introduction about myself. So I'm Vibhav Khandelwal. I'm co-founder and CTO at Shadowfacts Shadowfacts is India's leading last mile or express logistics network. We are present in 200 cities now we do around a lack and a 50 orders. I mean 1.5 lakh orders on a daily basis. We have a fleet of 10,000 plus delivery partners which which are on a crowdsource model working with us and it has been a good journey for us for the past three and a half years in terms of the clientele. We work with almost all the bigger enterprises that you guys have seen from a consumer standpoint. We are not a consumer facing company. We are an entirely B2B organization. So we work with McDonald's. We work with Domino's. We work with Swiggy, BigBasket, PTM, Snapdeal, Mintra. We work with all these clients and we do fulfillment of the last mile orders for them. The team is headquartered in Bangalore. We do have offices across the country as well. So now moving on to the presentation, what I want to talk about here. So as the title says, the product development for fleet management. So my agenda or my brief outline of this conversation or this talk would be on the lines that I would first want to highlight a bit about the persona of these delivery boys that what kind of people are they? What are their likes? What are their dislikes? And it will be more then we'll focus on the two parts of it. One is the engineering challenges and our learnings from that and second are the product level challenges that we have faced that we have learned things from them. The overall objective of this talk is not to say that this method works good or this method works bad, but primarily to share our learnings to share our experiments what we did in when we observe those problems and how we try to figure out a solution to them. So that will be the overall outline of my talk here. I'll be happy to take questions after the presentation here online as well as offline as well. So yeah, moving on to the first section, the rider persona or the delivery boy persona. So what kind of these? I mean, how do we describe these delivery partners? So bases are interviews bases are conversations. These are the guys who are the ones which are fulfilling the consumer apps or the consumption powers of the 50 million Indians. These are the second part of India or a couple of us also refer as Bharat versus India debate of sorts. Now these are the guys who have good aspirations. They they want to do big in life, but because of limitations in education because of limitation in opportunities, they are not able to do that. What do they? I mean, you talk to almost all of them and everyone would want to earn and make more and more money. There is no saturation. I mean, they would not come and tell you that we would not want to maximize our earnings. We'll try to figure out their ways. Then what are the difficulties that they observe? So finding the exact location having erratic consume customer experiences as well. So to be honest, we have heard case studies where so we do reverse logistics as well. So we have heard case studies where the customer has held the delivery boy's hostage and said that I won't release this guy unless Paytm refunds back my money. So these have been the scenarios and this is what these guys take care of. You see any good delivery experience image or a video. What you will see is a happy customer. What you will never see is that delivery boy who is just a plain simple guy fulfilling all these tasks for you and that is where these guys are. They are in the shadows. They will be penalized if they do wrong and they won't be appreciated much if they do good. So that is the kind of persona here in terms of motivation. Again, money is the major motivation. Recognization as we'll talk about more as well. The social recognition is also an important part here. What are the apps that these guys use? So share chat tick tock. So with the advent of geo actually the video consumption has actually taken off and to an interesting part. What we observed was that just after geo almost a large number of the phone numbers were starting with the number six, which was technically geo. So almost everyone flocked that into going into geo and these are the guys who want to maximize their savings and earnings. Another persona. So this guy is married. Education is 10th pass again. Not many opportunities available in life to do to earn big or become big then another goal and that comes with age is to have a stable and secure job. So very few people of us are very few. I mean, if you consider this outsourced crowdsourced economy the gig economy. So people feel that paying on a per transaction basis is absolutely fine because you are able to minimize your risk. You do not have to associate for a fixed contract with anybody whereas these delivery boys when they when they get into a family, they would want a stable fixed job. So just to explain it some in some numbers it is more like a delivery boy is fine. If you give him a payout somewhere between 12,000 to 15,000 versus giving that guy a payout somewhere between 8,000 to 20,000. So these guys they would prefer lesser of variance and more of stability. Now different personas have different identities. Different companies have different preferences about it, but just wanted to highlight some commonalities and differences in the thought processes. Among all these guys. So this will the reason why I'm highlighting this or the reason why I'm discussing is just to set up the context on what sort of people are we talking about? What actually is fleet management? How different are they? How similar are they from us? So now I'll move on to just the context is set up. I'll now move on to engineering challenges and the learnings. So as I told the conversation would be split into two parts. One is what engineering challenges you foresee? What are the engineering challenges one get to know? And how can or I mean some ways in which we have learned how to overcome that and then later on the product part of it. So the first first thing here is the locations. As simple as that with the advent of Uber and similar related startups, people realize that just by tracking locations, you can actually make a lot of decision making in the process. But tracking location in itself in is in itself is a very complex problem to solve. So what are what are the challenges? I'll just try to focus here. So one is patchy network. So even with all that 4G coming in still, there is inconsistencies in the network that are there. Now you would ask that Ola and Uber work perfectly fine here. What are the challenges that these fleet have in addition to that? So just consider where these guys have to travel to. So the majority of the transactions of a delivery boy, they happen at the customer doorstep. The majority of the time that the partner is waiting is near the restaurant. The hubs are generally outside the city with having lesser population and coverage in these areas. You would have also observed that in the doorstep point the your in the indoors in the basements, your internet connectivity is not that great and for this delivery boy, marking delivered or getting the online payment. Every transaction depends on these major hotspots only. So not a major problem. But yes, once you start seeing it at scale, you would figure it out that this is a major problem. Second is on the battery drain. So again, you would see in Ola and Uber. They have a USB plugged in into it similar adjustments. Yes, do exist in bikes, but they are not that successful and scalable and since locations are privy to the customer experience locations are privy to the earnings that a partner can make. Therefore, ensuring that the partner is alive for the maximum duration on your platform is of utmost importance. You cannot just keep pulling the location and allow the battery to drain out in four to five hours. You need to optimize that thing because that partner the delivery boy is actually on the move and the entire experience the entire earnings of that partner depends on that. So I'll focus on couple of learnings that we have taken in this process. So one one is on the MQTT protocol. Just a quick check on how many have heard about this protocol earlier or maybe use it. Sounds fair. So I think there is a there is a good audience that that knows about it. So primarily so earlier what we were doing, we were doing it over an HTTP protocol and we were transmitting location data using a socket that was created and that used to drain the battery at a good amount of in a very small limited amount of time. Then we thought that we would keep pulling the location and we will open the socket and send across the location only at a particular point of time, thereby reducing the battery drainage that is there in the socket opening and closing. Then we figured out then we stumbled upon this protocol that is MQTT protocol. So what the I mean this is this is I mean those working in the IOT sector. This is actually the machine to machine language protocol earlier was used for satellite communication as well. So it sends data as bytes and I mean it does not have all that I mean what I would say the confirmation of the proof of delivery like as of HTTP but it allows you to transmit data at even in patchy networks that includes 2g networks as well the battery drainage in keeping an MQTT socket open is lesser than the battery usage if you intermittently open and close the HTTP socket and this is not in single digits. It is a multiplier effect. So I'll be happy to share some links as I move along where people have done the benchmarking on the battery drain part on keeping an MQTT socket open and changing in the HTTP socket duration. So this was one of the major thing that helped us and what we created was a backup mechanism wherein the location would be first sent across the would be first sent across the MQTT Pub sub network and then would if failovers are there on a repeated basis then it would be taken on an HTTP protocol. This significantly helped us in getting locations even in the patchy networks. This significantly helped us in reducing the battery drain. Some numbers I have highlighted here. I am not sure how clear will they be to the audience but yeah, we have seen that we are now able to have even using our application even if you use it for the entire day the battery drain is 7% an hour. What that results in is 14 hours roughly of 14 hours of continuous app usage with continuous location data at the right frequencies that you needed for the right customer experience. Moving on next is on the location strategies. So again, so the important thing here is that it is not I mean you cannot you should not actually do keep pulling location at a particular frequency. You can look to optimize your strategy as to when you want to pull the location and when you do not want to pull it. Some trigger points may be so once the rider is delivering an order you definitely want your customer to have a real time tracking experience at that point of time. You can actually increase the frequency of getting locations from the partner when the partner is idle. You do not need it at a right. I mean at that high frequency you just needed for the decision making that the partner is in that vicinity or not. Second is when the you do get signals from the battery on the battery low percentage that is available. You can consume that you can have these strategies created basis the area in which the partner is working and the criticality of the task that you would want to track. So some and other configurations majorly the changes were done. I mean once you are able to do this decision making on the server you can just configure it pretty quick. Another important thing that I would want to discuss here is that the hardware configurations. So a few who have worked on Android and have tried to capture locations you would have said you would say that it's very easy to write a code for capturing location. It is just a single line of code where you say that I would want the location frequency at 30 seconds for example and Android will handle everything for you. But once you start going into depth of things, you realize that hardware does not behave in that way. So for example, if you set the same code in OPPO devices, you would realize that your location is let's say you have set the frequency as 30 seconds. You would realize that you are receiving your locations maybe after a minute. Once you set it in Samsung, you see you might get your locations even in 25 seconds as well. Now this was surprising to us as well. We were we were initially debugging our own code. We were initially trying to reach out to the Google fellows that what wrong are we writing in here? But then as and more and more when we read about it, we figured that this is how these different sensors behave. The Chinese OEMs they focus more on your selfie camera than on your GPS sensor. And if you see the distribution of what these guys have, you would figure that 34% is Vivo. So that is where that is where the entire gamut is. So what we did so what we did was an interesting bit here. So we had the past data on all the devices of different brands. We created a model on top of it. And what we created was we created two different frequencies. So one is the business frequency and the other is the device frequency. So the business frequency. Let's say is the business says that I need to get a location in every minute in every one minute basis. The model basis the past location data received for these different mobile phones. You then forecast you then predict what is the frequency that I need to tell to this device and that configuration is done basis the server. So technically I can sitting here modify at what frequency should I order this device to send locations on to and that forecast actually allowed us to save in more battery because some devices some hardware is really good. Whatever you tell them they'll figure it out according to them. So for example in Samsung, I could just say that share location in one minute and it will give me in one minute, but for OPPO, I would have to say that share location in 30 seconds that even if it be misses one beat, I'm able to get it in a minute as well. So that has helped us using our past data for different OEMs and that is one of the other thing that has helped us to save battery at the same. Moving on to the next thing on the routing engines and ETA prediction. So another crucial thing. So I mean whatever I think there have been talks by Divya on the run sheet automation. There have been talks from the Swiggy folks on the batching increase maximizing the batching almost every logistics problem or every last mile problem will involve routing. You need to do these optimization calculations at which is the perfect route for you to figure it out. Now there are multiple ways of doing it. I mean, I'm not going into the how do you row? How do you create the best routes? I'm going into the basic need or the basics the basic prerequisite for creating a route. You would need the ETA between two nodes for you to be able to make a decision. So let's say I have 10 orders. I would want that for all combinations. I should know what is the time that it will take from this note to this note only then I'll be able to create some sort of a logic for the routing engine. Now capturing this ETA can be done in multiple ways. The simple method would be taking the heversine distance between the two coordinates and taking a fixed velocity dividing it and you get an ETA or by ETA. What I mean is the time that it will take from point eight to point B the amount of time that it will take to reach there. Now what why this method fails because this is the airline distance is does not take into account the road distance or the one way part. The positives of this is that is super quick. You can get results pretty quick. The second method is use the Google Distance Matrix API. Google has an API wherein you can supply in your in input nodes and get the out and the destination nodes and it will give you an output in a distance metric. Now what I am going to say is that this is not the preferred method if you are working at scale as simple as that your response times for a Google Matrix API would be in the orders of 100 to 150 ms. Now when you are creating a run sheet, for example, of 30 orders, you would want 30 C2 combinations to be evaluated as distance and maybe when you are doing order batching in the case of swiggy or in the real-time orders, you would want this computation to be done even in a more real-time fashion. So if you just keep on adding it, it will limit on the number or on the complexity of the algorithm that you can run. Second is that it is really expensive. Once you are using it as an asset tracking tool. Once you're using it on a work basis, it is really expensive. Yes, it can definitely fulfill your costs that you are paying for it. But yes, it is decently expensive in the tunes of like 3000 US dollars on a monthly basis. This is the kind of calculations that we were expecting to do. So what we again stumbled on again thing and again, I would want to ask how many of you have heard about or used OSRM. Great. So OSRM again, it's open source routing machine. So this is basically what this engine is. This is technically a C++ library uses a star algorithm to optimize the route. What it will do is given any graph, it will find you the shortest distance between from going from point A to point B and it will give you in a response times of 5 to 10 ms. Now what we do is that you feed in the open street maps data onto this OSRM. That creates that gives in the graph data for one way that this is a one way. This is a road. This is how these different nodes are connected and then you run OSRM on it and you would get the road distance from going from point A to point B in a response time of 5 to 10 milliseconds and this you can host it on your own servers and with the volume that I am referring here I we are working at a T2 medium level and it cost you roughly 1% of what the Google APIs would cost you and not much of a maintenance hassle as well and this is something which Uber also uses it. What negative exist here? The negative is that definitely does not have the traffic data that Google has. How can you overcome this corn here? So OSRM allows you to give weightage to each of the edges you can upload weights to each of the edges and accordingly it will optimize and figure out the best path for you and what people I mean basis my discussion with data scientist at Uber and couple of other folks that I know there what we have figured out is that even Uber is also using their cab data to feed it into OSRM to actually create a replica of what Google Maps is that is an entirely 100% in-house solution very high very less latencies and really helps. The third thing that I'll focus here is the location familiarity. Now this is something that we realized after talking to delivery partners now. So these delivery I mean you could create a routing engine that is the most optimal that will give you the best results but there will be difficulty in adopting it on ground. Why because delivery boys they are familiar with a particular area and they would want to work in a particular area. They would want to get orders their last orders near to the area where they live and that is something which is a known conversation that happens in an offline manner. So if you see if you go to any of the e-commerce hub, you will find that these are the pin codes reserved for this delivery boy and this is some sort of an understanding that has built up. How do you take it out into your system? And that is what we have done. We have tried to incorporate the location familiarity into the routes that we create. We ensure that these I mean you basis the past history of the location data of these partners. You can figure out where is this partner most likely to know that place and reach that place and then take it up and then create the route so that these are preferred to that. So these are on the routing engines and the ETA again, broadly learnings what we have figured out that this is this can work. This will not work and now I'll move on to the product development learnings that we have had. So first thing is on the gamification. So again, the thing will stem from the fact that let's understand what is a life of a delivery boy. Now there is a repetitive task on a daily basis. He has to come in the morning, go to the outlet, take up orders, deliver it to the customer. It's on and on again and again. There is no brain involved. There is no new thing new surprises. There is nothing in that zone. Then there is no promotion or there is no career growth. If you see, I mean, even after even if a delivery boy has worked for you for two years, he's expected to do the same work. There is no entitlement or a senior delivery boy or something of that sort. And then there is erratic consumer behavior as well. I mean, they have to bear the rats and the negativities that some consumers do have and so and see in India in general we we want the balance of both. We want low cost and we want the best service. And if we are paying money, then we would want the best service even if that money is 10 rupees. So that is the kind of consumer behavior that these guys are expected to and that is some sort of a life of a delivery boy. So consider yourself being in those shows and you would figure that it is a very boring life. It's a very monotonous dull life. There is no motivation left. How do you create that motivation? That is that is the challenge that is the problem statement that we have tried to handle. So I'll talk about the two different methodologies. One is the social recognition part. The other is the Carrot and stick model. So the Carrot and stick model is that you incentivize on good performance you a dissentency why you find these partners on the bad performance or missing the SLA or doing the wrong part of it. While this works, this definitely works because the most as we saw the most impactful thing for partners is the money. So if you are starting starting to find them, they will definitely starting start to improve or maybe bypass the system. They are also known good for that. So but then again, that is absolute that will have impact on your attrition levels that I mean there are lots of psychological things that this might not be the best way forward for these guys. Then the other part is the social recognition stems from the fact that these partners they stay in a community. So you see the Swiggy guys, they'll stay together. You see Zomato. They'll stay together. Now what matters more to these guys is that yes, they are they are made to feel proud in front of their colleagues and that is something that you can create a virtual currency on that is something that you can actually take it up ahead and incentivize them and that even works for us as well. I mean, I'm not sure why people put their personal photos on Facebook, but they do get that instant hit that. Yes, people are recognizing that. Yeah, that this guy went to Paris. So I mean these are the kind of things that we have tried to figure it out the screen on the right shows a sample screen on how the delivery boy would see. It's more like a magic score would not want to go more into the details because something related to I mean very core to shadow facts and not many people would relate. But the overall thing is that we that it is more of a gamification, it is more of a way in which a metric in which you can recognize on a social level. So what we have done here is that we have created a leader board here. So a partner would be seeing his or her rank bases the vicinity in the vicinity of two kilometers, how the other partners are behaving now. It is more of a relative metric. What we have noticed is that there is a very high page visits of this page and people generally tend to compete here. There are different kinds of groups that we have figured out that there are some people who would want to be the best among their group and then there are some people who want their their group should be better than the other group. So for example, the Kormangala Warriors would want to be better than the Indra Nagar Warriors. So these these are the dynamics that you can tend to figure it out and can be used to motivate or incentivize these partners at different level. What our learning says that yes, these delivery boys, they like such a concept and they that adds some spice into their life. Next, I will focus on the loyalty program. So again, the thing here is so once you I mean in this market where heavily, heavily competitive, I mean, we have seen food Panda over eats, wiggies, omato increasing, increasingly giving a large number of payout on an ad hoc basis to these partners trying to capture as much as possible the market share. Now what you can do or what one can do to actually ensure that your partners are with you is to try to create a loyalty program. So one of the I mean outside this everything. I mean one advice that one of my senior who worked at Prathap, he gave to us before starting starting this job. So he said that till date these delivery boys are seen as blue collar people. They are seen. They are taken like laborers and therefore they respond to you like laborers. If you can transform them into white collar people, if you can transform this delivery job as a white collar job, then these partners will reflect start reflecting that ideology you start giving them if you start giving them transparency on the payout. If you start giving them joining bonus, I mean, whatever benefits that we get application for leaves paid leaves are there. There is a notice period associated. So all these things if you can get, they will also respond in the same way. So I like to quickly move and present a quick video of the rewards program that we have created. So again, a virtual currency. I think self-explanatory. I would just leave it to the audience. I'm not sure why the audio is not working, but yeah, the leaderboard part. So then moving quickly on the payouts part. I think almost everyone here realizes that the major part is in if you want to build a brand, you would want to ensure trust if you want to ensure trust, transparent and timely payouts is the key is the key here. So again, some learnings from our show as much detail as possible to the partner. So that the partner is able to clarify, verify and get proof that yes, this is a organization that I can trust. Have a support team for uns for helping in the pro in the payout related queries because that is the major thing that worries these partners. That is a major thing that will drive your retention levels then automate the calculation and execution process do not leave it for humans do not leave at the guidance of humans to calculate the payout to update the payout or even to release the payout. What we have done is that we have we have even integrated with the bank APIs and the execution of the transfer of money to their bank account happens via that that gives more visibility to the partner. What is the status of their payment? When will they get? How will they get and what we have also observed is that once you have automated all these calculations your issues also come to a lower point. So another point here that we were thinking and so this is not something that we have worked on but we have discussed it with or we have seen similar products in other geographies. So one of a similar player what they do is that they give the option to withdraw at will. So a partner is shown the live balance that this is the amount that he or she has earned at this point of time and you can withdraw it if you want to withdraw it right now. So some innovations some changes here but do focus a lot on payout if your payouts are transparent and are done on a timely manner it will create. It will be much easier for you to manage your fleet. Again, then I'll focus on the in-app training methodologies. So training is must training is important to build your brand. I mean build your brand loyalty. I mean these delivery boys. They like the fact that yes, the company with which they are working is present in 200 cities. So you would want this information to be communicated to these partners. It ensures good customer experience has best. I mean allows you to give give best practices. There are problems with the classroom training thing where you cannot scale it properly. There is no track of how good the conversation is or how good the training is and there is lack of personalization as well. Uh, what we have done is a small video here. This will so what we understood was that videos consume better. So within the app itself, we have allowed these videos to be a part and the delivery boy can. So we name it as Gurukul the delivery boy can go there do the chapters and earn a surprise money. So that was something related to what we learned from Google pay which actually worked good. And then vernacular content is really, really important for these delivery boys. If you throw, I mean, they can understand your keywords of the apps, but if you give them a very decent content, it will be difficult for them to understand and they'll generally skip that content. So yeah, a quick look on the, uh, I think I'll in the interest of time, I'll move ahead. Then fraud detection, uh, I guess that's a last slide for me. So, uh, build that pretty early. So a lot of people will feel key this is not where I would get my revenue. This is not where I'll optimize my cost right away. But when you're working at scale, it is important for you to have a module in place which takes care of the frauds. What these frauds do a obviously the economical impact, but the other important impact is that it creates the bad practices among other genuine partners who feel that those who are making fraud, they are able to get away pretty early, pretty easy. So do not have a, I mean, have a zero tolerance policy here. I mean, define bad apples, define your values that hey guys, this is not acceptable at any given point of time. And so another important thing is so we are a big proponent of other KYC. I mean, if this platform is of any help, we would definitely would want to sign up a petition that please enable other KYC for players, private players like us. This, this was a really, really good way for us to verify our partners to verify the genuineness of the partners very quick and reliable process to take it up ahead. But the reality is it's still not available for us. So we have tried to do a couple of things so as to be really sure on the documents on the items that we have. So we do the image processing on the pankard because pankard is an important document that needs to be taken there. We have integration with the pan API's then background verification again is important. It will generally help you when, when and ever. So it's more like an insurance. If things go wrong, this would be something that will be of help to you. Then bank account verification again, an important thing. I'll move on to the location intelligence part would focus a couple of things on this. How many of you have used Kepler pretty less interesting. So I'll bring a new thing to the audience. So please visit Kepler dot gl a wonderful, wonderful visualization tool again open source contributed by Uber and you will see really good visualizations there and you can and the interesting part is that it is a client only application the data that you upload. There will be on the browser won't be synced on the server and you can visualize data to the limits of like 200 MB is what I can definitely say that can work and then so one learning again is that location stream and task streams. So there is there are two streams. One is the location data that you are getting whether is the order data that you are getting. If you have these two data, you can actually replicate the ground scenario even in past what happened at that one particular point of time. But the important thing is that they need to be used in conjunction. They individually cannot help you. So one of the learning that we had was that earlier we were we used to find our partners basis the low order account that they used to do if all other partners were getting good orders and this partner was not getting a good order. We felt that this guy is doing something wrong and then we figured out that it is not the right way. Move down to the location intelligence part and now we are merging them both visualization definitely helps do focus a lot on visualization in fraud detection at least you never know what different kinds of frauds exist so you cannot code for them. What you can do is that you can create methods or you can create those graphics graphs for the operations team to see that data so that they can basis their operational know how figure out what is going wrong and what is going right and then again I mean building the right location infrastructure. We have spent good amount of time on that. I would leave that for another talk but yeah that's in itself is an interesting and a challenging problem to solve at an infrastructure level at a global level. So with that I'll take a note on the presentation. I'm open for questions. Please guys shoot. We're out of time. Let's take one question. Hi, hi, that was a good talk. Thank you very much. So you mentioned reverse logistics somewhere in your talk. We just explain what is that so reverse logistics is that part so as simple thing. Let's say you order a t-shirt from Amazon. Now the process of delivering that t-shirt to you is generally the forward logistics. Now let's say you did not like that t-shirt. You want to return it back so that process of picking it up from you doing the doorstep quality check whether it is the same t-shirt that you have ordered and it's in the right condition and then taking it back to the seller is the reverse logistics also if you have time and no one has questions I would like to understand more on the integration of with Kepler with you know your order system is what you mentioned. So I would like to understand the advantages that you got you know after that integration or what problem it exactly solved. So Kepler is generally and data exploration tool. It won't help you to create the ready visualization which you can just directly put it in your dashboards. What it can do is that if there is data, you can actually slice and dice and view different kinds of visualizations on that to figure out what is the metric that you would want to chase. So there would be there are in so if you go one level deeper so Kepler is built on top of deck dot GL. So you can use deck dot GL to actually integrate it in your portals DCK dot GL. I have also included some appendix in here. I'll keep updating on it so that the relevant links of the topics that I covered are covered there. Thank you so much guys. Thanks Weber. So can Rahul Jain come to the stage? So the next talk is so the next talk is by Rahul Chen from locust.sh and he's going to talk about sales for it or transformation and optimization. I'll set up my name is Rahul and I'm a data scientist at locust.sh. We are an optimization company majorly focused at optimizing largest decisions and logistics. Today I'm going to speak about optimizations for sales fleets and basically from a perspective of FMCG companies like this is optimizing the movement of salesmen or salespeople and and the in the next part of delivering the goods the last mile routing part. So the outline of the talk is basically I'll explain what is a salespeed. Why do you actually need a salespeed dig a little deeper into what are the challenges? Dig a little deeper into the stakeholders for the same. The salesman the shopkeepers the sales managers formulate the problem and then talk about our solutions and then some ground challenges that we face while deploying these solutions and for both from a quantitative perspective from a math model perspective and also from real-world soft constraints like what are the ground troubles of deploying such a solution? So the first thing is what is a sales suite? So let's look at a FM typical FMCG company a typical FMCG company would have like a to 10,000 stores from Kirana stores to wholesale stores in a particular city. Now what they want to do is that they have a fleet of salesmen like 20 30 salesmen and these salesmen need to go visit each of the stores and collect orders. Once the orders are collected, the company would sell our delivery send out delivery vehicles dispatching the orders. So basically it's a two-step process of once that you need to figure out which salesman should go to which store selling what product take what route doing selling that product and once that is done you want to send out delivery vehicles dispatching those orders the objective of this entire process is to get as much revenue as you can so send the right person to the right store selling the right product and also reduce costs. So you also want to minimize costs like transportation cost and also maximize the amount of time the salesman spends in the store as compared to on the road. A question that I get often is why do you need a sale suite like we are in a we are in the digital era. We are in the time of Flipkart and Amazon where people order products on a web application and once the orders are there you can always dispatch the order. So this is actually a bit inefficient that you are sending a person on the on the road to collect to physically interact with another person to get an order from a FMCG perspective. That image over there is a typical shelf that you would see at a store and it has products from all brands and companies. So shelf space is very very important given that it is a push sales model. So people are really working salesman are actually going on ground pushing products and if you are not selling a product someone else's and a code that I got from one of our partners was that increasing sales by 0.5% is a little bit more important than reducing cost by 2% that being said it's not true that reducing cost is not important. It is very important. But the reach is everything you need to reach the customer and you need to push your product on the shelf. That's why sales feeds and sending salesman or sales people on ground is very important. So I'll dig a little deeper into each of our stakeholders. One of the insights that from the entire exercises of exercise of building the solution and deploying it in India and Southeast Asia is that other than from a math perspective looking it as a math problem you need to appreciate the human element. These are real people who we are optimizing solutions for and they have some real contributions in terms of the in terms of their experiences that you need to capture in the math model. It cannot be purely done sitting in a lab. So we did a number of user interviews interacted with these stakeholders. The most important stakeholder in this is the sales person and you get a very interesting insight from from the person that the classical optimization literature would tell you that to solve to get the perfect route you minimize the distance and that's how you're going to reach the maximum amount of stores in the minimum amount of time. But a person's told me that you need to be at the right store at the right time. So over because he's work he or she has worked for the past 20 or 30 years in the industry. They know that if they visit a wholesale store at a particular time during the day they're going to get more money as compared to if they would have visited that that store during the closing hours or early in the morning. So these are regional heuristics that these people have figured out based on their experience and we were really interested in modeling these like we have a lot of historical data we want to figure out we want to try to add these objective terms in our optimization algorithms to understand and then optimize for these objectives that can actually get you more money. So if you can if you want to get more money you want to be at the right store at the right time. So making a sale is more about being at the right place at the right time. That is one of the insights that I got from salespeople the shop owner actually shop owners in this is a little bit more specific to Kirana stores. So this also reinforces the same fact basically the Kirana from Kirana stores perspective business for him or her is more about managing cash flows although they buy from every other company but they have limited and constrained resources and again if you are at the right time at the right if you are the right store at the right time of the day which might depend on number of factors like day of the week month of what is the month is their festivity are the festivities coming around you can actually increase your revenue potential from a particular store these numbers something that we got from looking at a lot of historical data also that revenue potential of stores is not static across the day and it can be very dynamic. So you want to reach at the right store at the right time. The third stakeholder is the sales manager. So the sales manager is a person who has to make these plans you would have 5000 outlets and all these salesman and now he needs to plan for the next month create these permanent journey plans as they say it. So that person says that this is not a perfect science that you can actually have an optimization algorithm and which will give us even if you don't use optimization if you use historical heuristics and intuition and make the beats it is still not going to be a perfect science and the other problem is that even if you build such a thing it has to it needs to be very repeatable because although it says permanent journey planning every month they get 500 to 1000 new stores. Now it needs to be re optimized. You need to create new routes. So permanent journey planning by itself is very dynamic and their point is that deploying or developing these plans would require more than just optimizing on regular objectives. You need to look at softer constraints figure out. How would you model these things in your equations? So from a bird eye perspective, I'll give a problem statement. What is the problem statement? So you have a set of outlets that need to be visited. These outlets would have different requirements from foods and beverages to personal products to detergents. Are there different people selling these products? Those are the salesman and there is a finite point duration for which during which these outlets need to be visited. So you need to take these outlets create tools for salesman. That is the routing part. Then assign these routes to different salesman. You assign them then you take these routes and put them on different days at the scheduling part. So it is a very convoluted optimization problem which does three interleaved optimization simultaneously routing salesman assignment and also scheduling. So from a computation perspective, it is NP hard. I'll get a little deeper into it in a while. I'll just get into each of these each of these boxes. So a sales week a salesman. I already explained it's a salesman tour. It is a sequence of orders that need to be finished that need to be visited. So one common objective of defining a good sales speed is the salesman spends more time in the store than on the road. So that is typical distance minimization or time minimization that you need to achieve two interesting objectives that we added to our equation. So a salesman fatigue that if the salesman is more tired, then he or she would it's then you basically want the salesman to visit the higher revenue stores when he's fresher during the day and lower revenue stores may be a little later and the other was so you maximize you minimize the salesman fatigue and also maximize the revenue potential. So this is interesting objective because your sequence of just minimizing minimizing distance might not give you maximum revenue possible. So your your you need to assign the right outlet to the right salesman. That is one thing. The other thing is they need to be visited at the right time. So it is a traveling salesman problem with time windows as constraints also a few graphs from this is abstracted data from from some of our clients different sales. So for the same this graph basically shows that if you look at a particular outlet and historically different salesmen have gone there with selling different products. So that is S1 S2 S3 different salesmen have different skills of selling to the same store. So a salesman who probably be selling products like detergents for the past 10 15 years would be selling detergents a little bit better than other products other products which is very intuitive, but it was just to it was just nice to see that intuition is being backed by data and also it again reinforces the importance of sales speed that people actually are actually adding real value in going and selling. You can actually get more money if the right person is going and selling. So so we looked at a number of other parameters from the salesman's perspective like is his or her experience. What are the outlet familiarity product familiarity? Regional familiarity and other number of other miscellaneous features to create to create this matrix of if a particular salesman is going to a particular store. What is its revenue potential? So it's a salesman cross outlet matrix in a way which is basically derived from historical data. This is a graph on the revenue potential of a store. Just two kinds of stores that we are looking at. One is a Kirana store the blue. It's not clear the red line is a Kirana store. The blue line is a wholesale store. So as intuitive the wholesale store is actually going to fetch you a lot more money as compared to a Kirana store. But there's a limited window where you want to go visit the store when he or she is not busy during doing business with their customers. So the idea is that for all for buckets of these stores for all the stores you will create these buckets of different revenue potentials. And we you want to basically the tool should be created such such in such a way that the highest highest revenue that you can get from difference. So you maximize revenue in such a way that you visit the bluer curves during the peaks and then map out the red redder curves. So if you look at this slide once again the who to went to what to wear mapping now we have outlets with all of these curves which define their revenue potential salesman with a salesman cause outlet matrix which defines this salesman's ability to do do these particular outlets selling those products and you have days and now we want to simultaneously create tours assign them to salesman and assign them to days and there are a number of other constraints that are real world constraints which can be these particular stores have to be visited by these particular salesman only they have to be done on these particular days only. So two kinds of products cannot go together. So those are very client specific. Some of the real world problems that I would show. So actually we saw that salesman usually spend like 20% of their time on road which can be improved a lot by better routing. This can be seen in this particular image. It's a in one locality three people are going and selling the same product. So that is just if you do not have hard time windows that is a reflection of that routing is not really good at different people are going and selling in the same area. So if you improve your routing that can be directly improved. This is a very interesting inside that we got from some of our clients that now once tour we have a there's a common constraint that different products cannot we cannot go together when you're going and selling like food cannot go with detergents. Okay now then it should and so these are different tours for different products. So it's heristically believed that if you go visit a store in the morning, you should not go visit him once again to sell another product because that person's revenue potential has reduced now. He just made a sale. He or she or just made a sale and you're just trying to make another sale. So you want to space these visits out. You don't want multiple visits on a particular day. So this brings to the importance of optimizing this optimizing the problems simultaneously that you're creating the routes and then you're scheduling them. So scheduling should be done in such a way that you do not may increase the same day visits a lot. Now I get a little bit more quantitative to for people who are in the routing community and everyone else was also interested in the same thing. What is the problem classification from optimization perspective? So it's a capacitated vehicle routing problem which is periodic in nature because the entire that the tours need to repeat. So the tours need to be optimized across time. There are multiple trips. So one salesman can do multiple trips. It's multi-day. So that's a scheduling element that the tours created need to be put on different day buckets also clustering is a very very interesting constraint which messes up a lot of optimization. So when you are putting these tools on different days, you want those those salesman tours to be close by. You want them to be close by because two days later or three days later, you need to make deliveries to those outlets and if they are spread out across the city, your delivery cost is going to go prohibitively high. So you want to create tours looking planning ahead for the deliveries that would be made in the future. So you want clustering multi-product and my heterogeneous fleets you have different salesman can choose to walk or to drive or to take a different vehicle with a different speed complexity of the problem with just 60 outlets in the problem, the number of solutions that are possible. Go above the atoms in the known universe. And the number of the batches that we would be looking at would have seven to 10,000 outlets. So it really requires a smart optimization algorithm to actually give a solution in reasonable amount of time. I'll point this out later also that time is a resource that is really important in real world optimization. You even we actually kept this as a KPI that we want to have a solution in 10 to 15 minutes because all of these date all of the data that goes into the optimization. It's a very complex optimization problem at the end. The optimizer gave an answer after three hours. People are not happy with it. You want to change the weights. There are so many objective terms. It needs to be quick and you need to run multiple runs. You need to run multiple simulations. So optimization needs to be very very fast. So for a batch of seven to 10,000 you need to have solution time as a very important KPI. So what was the solution approach looking at all of these factors of importance of need for speed the problem being really hard and past and still generate very good solutions. So as it comes out solving it simultaneously routing with scheduling is very very hard probably cannot be done with the current infrastructure available. We broke down the problem the two approaches. I'm going to talk about one of the approach only here. This this breaks down the problem in routing aggregating tours and then scheduling. So since we are breaking down the problem in two parts, there are some approximations involved, but we added weight of objective terms for the second part in the first part itself so that we get reasonable tours for the second step which optimizes and finds feasible answers for scheduling. So how we do it is that we have a routing engine with with specialized heuristics that generates a bunch of tools bunch of visible tools. We have a scheduling engine which is a mixed integer program which looks at all of these tools where the bunch of other constraints and objectives and then puts them on schedules the tours to different days. The downside of this approach is yes, it is not simultaneous. So there are approximations involved, but the upside is that you can get solutions in 10 to 15 minutes and which is actually very helpful even in a plan like permanent journey plan is actually very helpful because you have to run like 40 50 simulations for batch to display value and visibility because again, this is we're dealing with real people over here optimizing assigning stores to salesman to visit. So there are other a lot of other real world constraints which even though you attempt to model them, you would not be able to model qualitative factors like like salesman like it's a just a business heuristic that these people have to go visit the wholesale store at 5 p.m. It is a fixed constraint. So it is a lot of trial and error still, but the idea is always to have a scalable fast solution to generate to generate excellent solutions. The map visualization of several solutions. The picture I showed you before for overlap. The routing overlap for a particular product has almost been eliminated now just because a math program would can deal with constraints and it can be hard. You can always an optimizer can guarantee you something like this. So we were able to generate almost zero overlap for routing. Even the double zero with the zero double visits at a store were eliminated. So this was a major major major improvement because for a manual scheduling approach, it is very hard for routing and assigning it on different days to ensure that there are no zero visits because they're limited number of days limited number of salesman and a lot of stores to visit. So one of the biggest improvements was that the revenue potential went really up because of minimizing zero double visits just by minimizing zero double visits at a particular store some of the average improvements. This is across clients distance was reduced by 40 to 50 percent Transaction time was reduced service ability number of outlets for these beats became realistic. Common thing that I saw in some of the clients that sometimes the salesman would be assigned 40 or 50 outlets and a realistic number of outlets a salesman usually does is 20 to 30. So that is often a problem like they were assigned these outlets because of some business heuristic but when it cannot be deployed on ground. So actually this particular the optimization actually helped generate tours which were realistic to deploy and ground also given these constraints of number of outlets. It's a cut of it but it says challenges involved. So challenges involved the first challenge I think which a lot of you would relate to also is on ground deployment and acceptance even though you we tried a lot to model a lot of these complex real world objectives on ground deployment is still hard because it is just a change. There is a big change involved on how people function and you can't just go the next day and tell them this is a new beat plan. You have been servicing these outlets on the past five years. Now you need to service these. It's it's a big change for any person. So we how we mitigate that was that the multiple iterations that we run to show that yeah, you change the you change the weights. This is the as a scenario you run multiple simulations with different weights different combinations and show that they can actually be multiple gains. If you follow the optimized answer so multiple iterations of different scenarios then we added some some special constraints like for people who are really really for sales managers who were adamant that this won't work on ground. We added we gave a phased rollout. We gave we froze some particular salesman tours say froze like 50% of the tools and then optimized like that's a constraint optimization that 50% of this is okay. This all is frozen which is your critical business. I don't let's roll out the other part given this is frozen then optimize the other part then that is optimized. Then we slowly relax the frozen constraints to get more and more on ground deployment. We made separate weight configurations for different geographies that one particular so this is a multivariate objective optimization problem so you can have you can depending on the weights you can get a particular solution to change the weights you'll get the solution. So we bucketed it down to different templates that okay if you want clustering this is your configuration with which you should run if you want distance minimization this is what with which you should run. No zero visits they will always be a tradeoff between one solution to another and one way of showing that the optimization is actually working. It was to plot the solutions give them visibility in terms of if you change the weights. The optimization is changing the this part is always their noisy data in poor address sets. Actually, I can show you these are to these are batches. They're supposed to be one city addresses only. So they were outliers both in inters at an intercity level. So for addresses the image on the left is all the addresses are on in my suit, but a small subset of the addresses are also in New Delhi, which is clearly those are wrong entries their intracity outliers also usually sales speed data is highly clustered like there would be 100 outlets in 300 meters, but whenever you see one outlet like 100 kilometers away from the city or 50 kilometers away from the city, that's also an outlier. So address data is a noisy data salesman historical data can also be noisy some technical challenges were scaling up the mixed integer program. For those of you who are in the optimization industry makes integer programs can be really unreliable. If you give it more scale, it's going to blow up. It's not going to give a give any feasible solution and time was very important for us. So we spend a lot of time to actually make the program reliable in terms added constraints added cuts to make it robust to giving solutions. Uh, horizontal scalability of the product to different multiple use cases. Uh, so when we started with optimizing sales feeds, our constraint set was this soon realized that as this was working, the constraint set almost became four times that people said, okay, this is working like let's add this also. Okay, let's add time windows to this. So when initial design begins, always try to ensure that your product is going to be scalable and generic. It can. It should not be that you make a very, very specialized product which nails traveling salesman problem for time windows. You can get excellent solutions, but it cannot solve anything else. So being generic across different use cases is very important. Constraints change use cases change, but the optimization should be still reliable and robust. Uh, and need for speed and optimization. That is very, very critical. Something that a lot of people from academia often ignore this, the importance of this. You can't run a plan for 24 or 48 hours and come the next day and look at the solution and say, okay, I will change this weight and then run again. People are waiting for you. Uh, deliveries need to be made. Uh, things need to be rolled out. Plans need to get really quick and you need to generate. You need to demonstrate value also. So they need to be fast and they need to be fast in terms of changing parameters to run multiple scenarios. The it's cut of it. Uh, it says conclusions. One of the most important conclusion is appreciate the human element when you are writing optimization solutions for the real world. Uh, try to understand like, uh, what do why do people do and what do they do? So and try to get that learning using some machine learning model into or anything can be heuristics or machine learning model or business intelligence. Get them into the optimization because not only that will get it accepted on ground. The other thing is that a lot of times those people have a lot of important things to say, uh, like, uh, the maximization of revenue potential, like getting this insight would have been hard for us, uh, just making that optimization program in the, in the office. Uh, so these are the real objectives that you can actually add and enrich the literature also, uh, enrich the, uh, because as people in the industry, we have access to real world use cases. Uh, the other thing is that built fast, like, which everyone says that but, uh, build with customers, uh, iterate more, give them a solution, get it back, uh, add tweak it a bit. So if you build with customers, uh, I've seen that the learning and the speed of product development has been really, really fast. Visibility is very important. Uh, so we made a, so initially we were giving optimization solutions to our clients and it wasn't, they were not, uh, they did not have a lot of clarity why this is happening and why this is good because this is a complex optimization problem. You people won't be able to get a direct insight into, uh, okay. You assigned this person, uh, to do this order at 5 p.m. But why is that? I don't understand the reason for it. So, uh, in this particular case, we generated different configurations, as I said, and a lot people to have a flavor of different configurations in that, uh, generated intuition that, okay, I see that if you are, if you want really, really clustered orders, if you want really, really clustered tools, you cannot, you will have multiple visits on a particular day. Like if you want deliveries three days later to just go to one locality to, uh, for your salesman, different salesmen need to go visit that same locality only then, uh, so your double visits will go up, but, uh, so there is a tradeoff always in multivariate optimization. So visibility helps you, helps people or the different stakeholders to appreciate that straight off. Uh, and speed matters, uh, fast, real good solutions are important. Thank you. Questions, please. Very good talk. Thank you. Uh, quick question. Uh, are you using solvers or metaheuristics? So we use a combination. Uh, so a lot of this, uh, we, uh, for the, a lot of, a lot of the IP is in-house developed. So the routing part is used as a combination of metaheuristics, uh, heuristics and, uh, and, uh, and we use a commercial solver also for the integer programming for the mixed integer part. We use, uh, yeah, we use simplex and go Ruby. Thanks. Thanks, that was a good presentation. Um, the last session already before the closing remarks is a box session, uh, on solving last mile problems in logistics, fleet management and mobility. Uh, we have, uh, three speakers on stage and just like the other box, this won't get recorded and Chatham House rule supply.