 I'll start. Hello all my name is Piyush Makhija. I'm a NLP engineer at Wahan. At Wahan we are working on the problem of automating and scaling high volume recruitment of frontline workforce for e-commerce and logistic companies. To give you an idea of why we are even required, please look at some of these articles that have come out in recent times and many more. So the on-demand and e-commerce industry has created a new job market for last mile delivery teams. Let me ask you a question. How many of you guys have ordered food or shopped online in the last month, in the last week? Is there anybody who has never used any e-commerce or online delivery service? The point I'm trying to make is that these services have integrated so much into our daily lifestyle that they have almost become a necessity. In some sense, the delivery guy that brings your products on your demand to your door is as close as we can get to a modern day Santa Claus. And the requirement or demand for these Santa Claus is much higher than the supply. To give some metrics to this, there is currently a 500,000 delivery boy demand in India, which is increasing at a 30% year on year rate. And most of the on-demand or logistic companies are scaling their operations heavily. To give you an example, Danzo is increasing their frontline workforce by 10 times in just second half of 2018. So frontline workforce, blue collar workers or last mile delivery all refer to the same stuff. So some insight into this particular kind of recruitment for last mile deliveries versus standard recruitment, the demand for per month hires is somewhere around 500 to 10,000 depending upon your logistics operations, the scale of your operations. If you feel that this particular number 10,000 is high, Flipkart had a requirement of additional 60,000 delivery boys just for their Diwali sales. There's a higher attrition rate somewhere around 75%. And this attrition happens in about two to three months. While in standard recruitment, one to 3% attrition happens over a year. The skill requirement for this job is low. So there's little barrier to entry. Anybody can start this job. And it's mostly customer facing. So the people that you hire actually affect your brand image. The hiring process is mostly manual and unorganized. It's mostly person to person interaction over telecalling and highly unorganized. The hiring demand is very seasonal and sporadic something like there's a high requirement during Diwali sale or IPL season. So what we understand is that particular job space is very fragmented, unorganized and really hard to scale. Formalizing the problem statement that we are dealing with a little bit. The problem is to build an efficient and scalable system for a high volume recruitment of last mile workforce. The design considerations that you need to take into account while building a solution for this particular problem are as follows. Firstly, you need to reduce the manual effort by bringing in some automation to either complete process or at least parts of the process. This point is to reduce the manual labor. Next, you need a one stop solution. You need one solution that accounts for your size and nature of logistics operation that can work across verticals and something that works at scale. So if my requirement is to hire 500 delivery boys this month and 10,000 delivery boys due to some sale in the next month, my solution should be able to reach that demand. We need to account for high churn rate and attrition. So high churn rate and attrition are inherent nature of this particular job segment. So we need to either circumvent that or account that into our calculations. There is no LinkedIn for blue collar workers. By this, I mean there's no social or professional platform where these users or these the people in the last mile workforce can connect professionally with themselves as a community or with the recruiter or build your career path. And when building a solution, we generally think, okay, let's make an app for that. But building an app has its own challenge. The first and foremost is the adoption challenge. Will people use this particular application? Will it be useful enough that they keep it on their mobile devices? The particular nature of the recruitment problem is that people are always going to uninstall the app once their work is done. Once you get them hired, they are going to uninstall it. It's not a daily use application. So these are the design considerations you need to take into account while working around this problem at one taking this into account. We came up with our solution of automating jobseeker engagement on existing messaging platforms, something like you build a account on a platform and where users can interact with you. So this number is a dummy number. This is not an actual number that you can call to this is just to illustrate my point. So how did we address the design considerations? Let's talk some somewhere around those lines. The first is circumventing the adoption challenge. This is the biggest issue. So we circumvent the adoption challenge by going on existing platforms like Facebook Messenger or WhatsApp. So the adoption rate for WhatsApp or Facebook Messenger is 97% for smartphone users in India. And most of these users use these applications on a daily basis. So we have a way to reach our audience or reach our users. Then we need to automate certain aspects of the process. The first is job pitch. Then you need to gauge the user interest. You need to screen the user and get them to the walk in and we do this via a virtual assistant. Now a virtual assistant can be IVR based voice based or text based. But with IVR based assistance, the problem is that you have a very limited questions that you can ask and a very limited kind of responses that you can capture. The person on the other end cannot interact with you or cannot query you. Hence due to these limitations, it's not a very effective way to go forward. Next, a voice based virtual assistant. The problem here is that you need to respond to the user in real time and with the data bandwidth issues and background noise issues coupled with ineffectiveness or errors introduced due to speech to text and text to speech. It doesn't seem like the perfect way to go. However, it takes base assistant. Something that our users are already accustomed to using something like WhatsApp or Facebook Messenger where they are daily engaging with each other via a texting platform seems like a right way to go. Furthermore, we need to avoid hopping across different channels through the recruitment process to give you an insight into the current recruitment process. First, you need to generate the leads and you do this via advertisements on classifieds, not really quicker and whatnot. Then you process these leads on Excel sheets and in some cases even on paper. You need to engage with these users that you have processed and mostly it's done via telecalling. And finally, you get them to your onsite screen the candidate, get them onboarded. So there's a lot of manual processes involved and all of these are disconnected with each other. We need to bring all of these processes on one solution. Next, since it's a text based chatbot that we're trying to build, we can scale and interact with thousands of users at any instant. And we can counter churn. We counter churn by increasing the top of the funnel. This comes from the fact that we are highly adoptable. We have gotten automation and we can scale. And finally, we can break build rich user profiles and lay the foundation of a social or professional platform where users can have a career path and recruiters can connect with them. So based on the design considerations, the thing that I'm talking about is actually a goal oriented dialogue system. We need to carry out conversations with our user to complete certain tasks and reach the final goal. The goal here being that we need to qualify the user for a given job and the task that we have are to pitch the job, gauge the user's interest, screen the candidate and get them to the walk-in. These are the tasks we are automating for in our solution. So a few key steps to a solution. So in our overall flow, when we post a screening question to the user, something like do you have a bike? We provide them with option choices, something like yes and no, a valid response. But the user may in turn choose to reply with the text of his own, which can be anything. It can be a query or it can be an indication of interest in something. So the first step here is to comprehend the user's text. We need to understand the words that are provided in the text input. And the problem here is there is high variation in the user input. For example, just the word yes can be expressed in more than 20 ways in English language that you and I can understand via texting platforms. Now imagine this being said in Hindi, Kannada or Telugu. We handle this via data normalization. That is we map all of these possible utterances to a word yes. We convert the input of different variations of yes to the single word yes. And we do this via machine translation. Now that we have proper words, we need to understand what the user meant when he wrote that text. And it can be a question on what is the job salary? What is the job timings? Where is this job? Or it can be an indication that I'm not interested in this job or I want to call you or it may be a greeting. I good morning or whatnot. So we handle this by intent classification or intent detection system. And once we have understood what the user wants from us, responding to the user is the easier part. While building any solution with machine learning, your model is only as good as your data. So we need to understand the kind of data that we have. What kind of utterances user is giving out? And the first problem here is that user mixes multiple languages. When you and I speak, we mix the languages that we know. So I am a Hindi and English speaker. I'll mix those languages while texting or while speaking out even something like call Marty, which is an English word combined with a Canada word. If I say this to somebody in Bangalore, they'll understand. But if I say this to somebody in Delhi, they won't understand what I'm going to say or what I'm meaning with it, which brings to my next point this high variability in data introduced by a regional health cultural influences. And another example of this is yes, a native Karnataka speaker. When he says yes, there's heavy emphasis on the S part, something like s and the user types in that way. The last example. So we need to account for all these intricacies of our texting that we do in normal cases. And to do this, we need to train a good machine learning model and to train a good machine learning model. We need a comprehensive data set that accounts for all these variability and all these data sets or all these variations in data. But the problem is there is no good public domain data set available for such an informal language. The technical term for such informal languages is code mixed languages and it's still an upcoming research challenge. Hence, we had to invest heavily in the data collection process before building our own models. So the data collection exercises that we went through was follows. We introduced the jokes and daily codes bought because this is something that gets a lot of fraction on WhatsApp and Facebook Messenger, right? So we entered groups where we put our bot and we send the users jokes and daily codes. It seemed like a reasonable approach, but the problem was that we were receiving a lot of spam. We did not receive a lot of useful data. Hence, we moved to our next solution. We built an English translation bot. The user would provide us an input and we would do a Google translate API call and provide the English response. The problem was the usage was very sporadic. In some weeks, we used to get a lot of data. In some weeks, we grow or practically nothing. Hence, this was not a good way to collect data. We moved on to an English learning bot. Users would interact with us and we teach them basic English. It seemed relevant to a use case or the kind of stuff or the kind of data that we wanted to collect. But the content was very narrow. We quickly understood that with this solution, we won't be able to build a comprehensive data set. Hence, we abandoned this approach and finally, we struck gold with a friend finder bot. The inspiration is from chat relate. The idea is that you connect to users on Facebook Messenger in an anonymous chat. Let them talk about whatever they want and we collect the data about what they talk. This particular bot was very popular and we were able to collect around 2 million messages a month with this particular solution. So once we solve the data collection challenge, we could get the data labeled and build our own models. So in the machine learning challenges, the first and foremost is that we handle. We had to handle nuances of the code mix data that we were dealing with a few examples of those I can give out something like hi, I have a bike and may repose bike here. Just notice this utterance or variation of hi, the same utterance or the same letter sequence is being used in this utterance and this utterance the utterance. Me repose bike here is basically Hindi for I have bike and based on the context, this HII is getting translated to English high, which is a greeting and HII which is Hindi for I have. So we need to account or take into account the context of the conversation or the utterance while translating in the intent classification part when we ask users a question like screening question like do you have driving license and provide valid responses? The user gives out utterance. I have LL only in this particular domain in this particular use case LL means learner license and the user will respond like this only LL or sometimes LLR. So our text normalizing model has to understand and expand this to a learning license. Then our intent classification model has to understand. Okay, the user says he has learning license, which means he does not have a driving license. That means his response is leaning towards no. Once we have understood this intent, we can respond that. Okay, get back to us when you get a driving license. A few more examples of a code mixed data set something like salary, which is mixing English and Hindi words in Roman script. We are able to normalize it. We are able to understand the users intent for a query of salary information and respond to it. So a few machine learning metrics that we have in the interest of saving time. I'm not going to cover all these metrics. I'm just going to say our neural machine translation method work much better than the statistical machine translation and our intent classification system has the coverage of 91%. The business metrics of this is that we are currently driving about 8000 hires for all of our customers. We are able to do this within five to seven interactions with the user and these interactions typically happen within 10 minutes time frame by that we understand that the user can if the user finishes the flow quickly is much more interested in the data or this particular job. The current ongoing machine learning improvement efforts at one are focused on getting more and more data and improving our models and making it much more better for the particular domain specific data. We are trying out new architectures and we are also looking into dialogue systems to make our conversations more human like a few key takeaways is that there's urgent need for a high volume recruitment solution. We need to build a social and professional platform for these users to connect and build their professional profiles. We need to build a solution or whatever solution we build for this particular problem has to be easy enough for these people to adopt and it should have some daily usage aspect to it with the machine learning challenge. There's a need for more research and development on code mix datasets and first and foremost in that is to formulate strategies to collect this data so that we can build better solutions for the Indian context and finally we need personalization for the regional persona of the user so that we can take into account how the user or the person speaks or how he's more comfortable to interact in the broader term. The vision we have at one is that we just don't want to do the recruitment. We want to help these people own their skills train them assess them and become a holistic platform for professional growth. In the short term with the kind of traction we are getting we feel that in the near future when your modern day Santa Claus comes to you bearing gifts that guy would most likely be onboarded or recruited via our platform. Thank you. Any questions? Yeah, yeah. So the two million messages data set which you received through the friendfinder. How did you process that data? Can you give a brief about that? So this is raw data or raw text that we are receiving from the user and we use this data to build a comprehensive data set of the words that the user speaks. This is a very generic data set. So it will have utterances that do not deal with any domain. It can be something about interpersonal interaction. It can be something about traveling to somewhere. So we have this data set and we not use it for data normalization basically like I gave the example of yes. If the person says yeah or yes with triple S is we need to label it as why is so we have our own annotation team that does that stuff. Hi, I'm promote from load share. My question is about how you went about the process of getting WhatsApp integration done and what is the price point around which this works? Any pointers for someone wanting to do WhatsApp integration? So I'm definitely not the best person to ask this question. I this is basically more operations and business perspective of the stuff and I do not particularly deal with this, but maybe I can connect with you or you can connect with us at info at one dot a and that you should get a response. So we have different sources from where we get leads. Sometimes the customer itself provides the leads and sometimes we are generating these leads by ourselves. We have our own referral system also that contributes a lot to it. So two million messages per month we were receiving. So I'm not sure exactly how many users they were, but we were like trending so much that we were in the top five apps in the region, maybe around top three apps in this India like South India region. One last question. Under what pretense did the users with the friend finder app, but did they agree to let the messages be used for training your. Yeah, so basically when the Facebook has those service messages that you are going to let this thing be happen. So they'll get that indication that your messages will be connected. It's going to be an anonymous chat and so on.