 I don't need okay oh yeah hello are you guys like you all got your food you're happy okay so yeah so hi I'm Abhilash and I'm the creator of bus uncle it used to be like a very side project when I was still with Nanding and Lavadia trade gecko but now it's like our own company and we're building chatbots for other people as well so I'm just gonna run through with you a little bit about how I created bus uncle a little bit of the technical stuff as well and a little bit about the design stuff as well so just a little bit about everything but I want you guys to feel free like you can just jump in whenever you want and ask me any questions like anytime so let's just start with like how it all started right so I was at a bus stop one day right and orchard road just like all these people are standing there with my head turned right waiting for the bus and I just kept thinking to myself like how long do I need to wait so I was just thinking how long I have to I needed to wait but all these uncles and aunties had their own problems as well right they went shopping and then they're all saying they're waiting all day they have to go back and like eat food at home or something so it's a very simple problem I was just standing there and asking myself how long I want needed to wait and Singapore being a really advanced country actually has all these electronic boards at bus stops that actually literally tells you how long you have to wait for the buses so they give you a list of all these buses and their arrival times so on that day I could just look at the back and then I'd know how long I have to wait and there are also a lot of apps available you can just go in the app store search for SG bus or something you get a bunch of apps hundreds maybe that give you like these little interfaces that kind of tell you how long you needed to wait for your bus right but on that one day when I was just standing there I felt that all this information was a little too overwhelming right in my head I just wanted to ask one question I just wanted to I just ask myself I wish I can ask someone how long I have to wait so if I looked at this board here there's just too much information there's like I don't know you can't the number of numbers on this board you'd be like blown up right I mean you'd actually have to look for your bus and find out how long you have to wait and even with the app they all have their own nice little interface but it you there's a fair amount of scrolling and a fair amount I mean a lot of information in these apps that's very unnecessary in my head I just wanted to know this I had to wait seven minutes so that's when I decided maybe I can try to build something that can just tell me this seven minutes and that was actually what inspired the creation of bus uncle so for those of you who don't know bus uncle it's a Facebook messenger chatbot who's really really quirky tells you about your bus arrival timings and directions to go anywhere in Singapore if you don't want to listen to me talk I might get really boring you can just chat with bus uncle it's probably a lot more funnier than me you can actually go to this link and just talk straight to bus uncle I'll give you a little intro of what bus uncle can do today how people chat with bus uncle right so the moment you start your chat with bus uncle you just say hello and he'd reply with an app right and you'd have a button that called bus how long naturally you'd want to click on it and as you click on it you're taken through this little user journey while chatting with bus uncle and you can actually tell bus uncle like which bus you're waiting for then he tell you how long you have to wait so right here I said bus 10 and he said you got 10 minutes go get groceries so the chat you see on the right is actually a longer form of conversation that people have with bus uncle usually the first time they chat with bus uncle they go through a really long journey like this but because bus uncle does not really follow a linear flow people nowadays just say a uncle bus 10 or churn how long then he just say 10 minutes go get groceries it's a much shorter form but this is generally like a good introduction to how people use bus uncle you can also get like transport information so you can ask bus uncle how can I get to NUS it's very similar to what you can do on Google Maps and city mapper this actually wasn't intended as a feature in the earlier days but a lot of people when they're messaging bus uncle in the earliest days would always ask him this so eventually we didn't have a choice right our users wanted this so eventually we had to go and implement this so so now you can actually ask bus uncle how to get to wherever you want in Singapore and it give you like information like all public transport information including trains even though it's just bus uncle and what's what's fun is you can get like step-by-step directions on how to get anywhere with little maps for every single step of the journey as well and I guess what makes it a little more interesting than other apps is you can try to be natural with it right so if you tell bus uncle when you're searching for directions you're late he actually gets you the fastest way to get somewhere you can actually even tell him I'm feeling very lazy today or I'm very tired and he'll find you the way that where you don't have to walk as much or you don't have to make a lot of transfers so it's very natural in a way can also get bus location so one thing we realized from our users was when they're waiting for their buses everyone would be like where where right they just constantly keep asking where where where and then we realized okay maybe we have this information from API is available let's just put where so now you can ask where's the bus and he gives you a location and you also get transport updates so you Singapore being a very very small country always has all these reroutes that happened throughout the year for all these big events so one of this these events is called F1 formula one and whenever formula one comes in a lot of public transport has to be rerouted around the tracks and normally when they're reroutes these buses get intercepted so a lot of people can't actually use their public transport how they normally would so bus uncle actually gives you that information like you say I'm waiting for bus 16 then he be like a it's affected by f1 a and then he give you a like an affected bus list that you can actually click on and learn more about it as well yeah so that's a little bit about bus uncle like an introduction now I'll go through more of like a super basic introduction of how to build a bot or how to build a chat bot right how many of you here are technical people like know how to code okay quite a fair bit of you okay so that's good and what I'm gonna show you is like super super basic stuff so this is like really really basic compared to all the supervised machine learning and all that stuff you've been probably been dabbling around at home so you can ask me questions about that later so normally when people think about building a bot people think it's a little complex I need to learn machine learning I need to master my linguistics like I need to know exactly what vocabulary to use and stuff like that so a lot of people in Singapore are actually scared to build chat bots because they're scared scared people will talk to their chat bot in singlish right so people like I need to master my linguistics and stuff and finally I need to practice my algorithm so that the bot like fully is able to go through all these non-linear flows and give you the answers you want many people think that this is a huge barrier but the thing is when you build a bot you don't really need to do any of this actually because you don't need to do it yourself because there are hundreds and hundreds of tools available that do all this stuff for you give you a very simple interface to use and you can go build a bot in like five to ten minutes but but don't build a bot in five to ten minutes like when you read online how to build a bot in five to ten minutes those are really the worst chat bots you can build in the world so yeah so I mean you can just start I guess I'll talk a bit about like natural language processing right it's kind of this this subcategory of AI that deals with how computers understand human language and naturally when you build a chat bot your chat bot needs to understand human language so a lot of natural processing natural language processing is involved there are two tools that are available that are pretty good for you to start off with building chat bots one is called wit.ai it's got acquired by Facebook in 2014 and one is called Dialogflow got acquired by Google in 2017 so you choose Facebook or Google which one you like just choose one so just choose one and how it works natural language processing is very very simple as a human it always assumes that a human is asking a chat bot a question and the chat bot has to give a human an answer that's it that's natural language processing to you in its simplest forms and when you think about how this this works in deeper retrospect you'd realize all it has to boil down is the bot has to give you the correct answer and that's it that's all natural language processing has to solve so I'll talk about a bit of the basics about it I guess so that firstly we can talk about utterance right and utterance is kind of the free text of a question that a human asks a bot so the term they use for this in in all these NLP tools is called utterance and utterances are literally just text you send to the bot like raw text unprocessed that get that the bot gets straight directly from the user so for bus uncle some of the utterances that people send us something like a weather bus don't see anything like how long more like all this stuff some people even say liar right liar where you so these are like utterances then we have intense so an intent is kind of like the topic of a question there can be hundreds of thousands of millions of utterances but generally they're always categorized in in these in these classes called intense and intense generally just mean what's the intention of the user yeah it's not I mean it's quite obvious so when users say how long do I need to wait for a bus the intent in the back to the chatbot is something called get bus time so there can be a hundred utterances but they can be only two intense so one intent maps to many utterances and many utterances map to a single intent generally and then you also have this concept called entity and entity is kind of like a useful word or phrase from a question so when people message bus uncle something like I'm at 187 946 near Roche and all they asking bus 65 at orchard how long the entities are the useful phrases or the useful words or phrases from these questions that bus uncle would need in order to process the information so entities here are postal code which is 187 946 another entity's bus number 65 and another entity's place orchard so it just means useful stuff and finally when you when everyone's talking about AI machine learning again it's all about confidence levels right the difference between AI and normal programming is AI is dumb so you need to make it smart but normal programming is either yes or no so you have to make it smart but for AI generally means that you constantly need to train it and as you train it you increase its confidence levels so a confidence level is the probability that a deduced intent or entity is actually correct so one example is when a user is telling boss uncle a waiting them long la the intent derived from boss uncle was actually get bus time but it wasn't with a hundred percent confidence because waiting them long la doesn't know I mean doesn't very directly translate to get bus time so in the back this intent was deduced with an eighty seven point five percent confidence so that's kind of like the first part of natural language understand natural language processing the first part of natural language processing deals with NLU natural language understanding so processing comes after the understanding right so a user sends an utterance to the bot the bot deduces intents and entities and the moment it deduces these intents and entities it does something important that does something important is in a little black box and that's where that NLP stuff actually happens it processes that information so when a user said something like I'm waiting for bus what happened was the user send boss uncle something and now they're waiting for reply boss uncle deduce the intents and the entities from here and it took the useful information like bus number and station number and send this these useful entities to Singapore learn transport authority APIs right LTA has a bunch of open data sets are real-time APIs that are available so this stuff gets sent right to LTA and the correct information we need from LTA gets sent back to boss uncle so it's the waiting time is actually sent directly back to boss uncle and essentially now boss uncle can communicate this waiting time back to the user this is what natural language processing is right but one thing we did different boss uncle was we said hey let me make this sound funny right so we don't really want to tell the user two minutes three minutes four minutes so boring like we they'll be they'll just be like an app then right I mean if a user is chatting with boss uncle why not make it fun why not make it interesting so boss uncle generally always says let me make this sound funny and he replies back to the user saying still seven minutes go make soft-boiled egg right with something funny and that's essentially how every request to boss uncle gets sent and responses received back to the user so machine learning comes in to play when this goes to scale right so there are hundreds of thousands of users who use boss uncle every day and essentially when as these users you talk to boss uncle they send in a lot of utterances they send in free text to boss uncle all the time this girl can send 283 this guy can send 46 and as the bot receives more in utterances the bot kind of deduces more intense and entities using these systems in the back and as it deduces more intense and entities the bot increases its own confidence levels and that's when what we mean by when we say a chatbot is getting smarter or it's self-learning or it gets smarter over time you've probably heard like millions of people speak about AI mostly salespeople not really programmers who speak about AI and say it gets smarter over time or it's self-learning this is actually what they mean in the back so so boss uncle receives about like 60,000 utterances this was actually a while ago yeah but about 60,000 utterances every single day so gets trained quite a lot and the confidence levels of boss uncles is pretty high actually so that's there that's him thinking to himself I done smart hall and that's supposed to be like a nerdy glass emoji but I don't think the bar point is showing it okay so that's a bit about like the the technical stuff of it right I'll go a bit into the UX side of things and the design side of things as well so when we build chatbots we kind of have to build for people a lot of people a lot of developers don't really realize that most developers get super excited about AI they start focusing on things like I don't know recurrent neural networks and supervised learning unsupervised learning and they get super technical and stuff but most developers themselves aren't really exposed to how their products are used with people right and that's where product managers and designers and all these other guys also coming to help improve that experience so the characteristics of people when they're chatting with bots is we've boiled it down to like three main characteristics one is people like to be natural so when you're chatting with a chatbot you never ask the person to tell you exactly what to say so if a chatbot tells a person press cancel to do this or press stop to do this it's a bad chatbot because you don't really want to direct the person to use the chatbot you rather want the person to be natural so that's one thing you got to remember when designing chatbots number two is people are always inspired or they stick to some kind of culture around them so this is this is everywhere this is not only geographic culture like in Singapore like it's not only singlish or stuff it could be any kind of community that a person in that the person is in they're kind of inspired by this community when they use the language are there any gamers here gamers yeah so so normally as you gamers when you play multiplayer with each other you chat with each other right but the language you use to chat with each other is probably like a lot of swearing and and a lot of like small words and you just try to be like very on point you're kind of inspired by that gaming culture as well so culture just means they're inspired by some kind of aura around how they're supposed to use the bot and finally when people use chatbots they like to participate very simply put they like to add value to the conversation so if a person says hi and the chatbot says oh hey I'm just bought I like to do this I can do this I can do that I can do that as well the person's not really participating there the person your user will get turned off and then they'll leave the chatbot so always design your chatbot so that people can participate as frequently as possible and and then that's kind of how people actually use chat right they like to participate so people are not robots when you design for chatbots you don't really just think about what the chatbot can do you don't really think about what's its utility rather you got to think about how people use messaging apps in general people use messaging apps not only to ask questions but they also emit expressions and they also like to be cooperative a lot of chatbot developers you'll realize that soon as you launch your bot people actually start saying things like thank you and they start saying things like good morning or hope you have a good day or in boss uncle's case they even ask like have you had dinner have you had lunch and all this random stuff so they're not really questions they're not really they don't really just want to use the chatbot but they like to be natural and express themselves so they're inspired by feelings experiences and most importantly randomness like can never expect what people do they're very very random so when when designing the user experience or generally the flows of chatbots you got to remember what people can actually do so boss uncle actually took into consideration quite a lot of these things and now boss uncle's launched on I guess three messaging apps so far messenger telegram and the Google assistant will be launching some more soon right so some learnings from boss uncle what made boss uncle so successful we've been able to boil it down to I guess three points one is a unique personality so when we built the chatbot boss uncle we didn't really say this is a tool that helps people do things rather we tried to say this is a person that people already know of the super old grumpy uncle in Singapore who likes to sit down and copy the arms and drink copy allow hard drink beer and likes to like scold people as well it's in very very fluent English so we're super inspired by this this persona and we actually directly translated it into the chatbot secondly boss uncle has a non-linear conversational flow which is I think one of the most important things you got to remember when designing bots so normally I mean how many I mean have everyone here used a chatbot before yes kind of maybe okay so most chatbots you might have realized are very linear right they tell the user hi I can do this please click on a or B and the user has to click on a or B and if the user I mean that's after user clicks on a then the bot says oh okay now can you click on C and D I mean CRD and the user has to click on CRD this is a very linear kind of flow right it goes from one point of the conversation to another point and generally these are bad experiences because people are not really used to being directed when using the chatbot I mean if a chatbot does this to people that might as well just get the chatbot to be like automated voice dialer right like you know all those automated voice dialers you call they say please press one of these press to yeah those are pretty much what we want to stray away from very very far away from so you want to build for non-linear which means you want to try to make the chatbot be able to answer the person no matter what they ask so try not to launch a linear chatbot as a huge new thing and finally what made boss uncle so successful is he gives you answers very very simply put just gives you answers people ask how long they have to wait he gives them the answer five minutes they ask how to get to to I don't know Changi Airport he gives them the directions so a lot of chatbots when you speak to them you realize they don't really give you answers they they say oh I think you're talking about this and do you mean this or do you mean that or could you please rephrase your question those chatbots don't really give you answers a lot people stray away from those not really nice yeah so we're also building like many many more chatbots for other people so we've built heritage granny for the national heritage board we have a travel chatbot in Meidan called ketchup and we are launching our next chatbot in the middle of next month soon as well so that's us that's boss uncle this is our website you can just go here chat with boss uncle and ask me any questions thank you yes yes so in the so the question was the bot learns more and more over time by getting lots and lots of utterances so who labels these utterances right in the earlier days it's actually the developers or the product managers who actually kind of have to label these utterances but if you use the tools like with AI and dialogue flow these tools actually learn from what you're labeling right they learn from you labeling these utterances as well a month later after late after doing some labeling work the tool itself automatically starts labeling them as well yes yeah so it's definitely possible that the tool can get it wrong so you do have to keep monitoring your logs you do have to keep monitoring like what your systems are actually deducing as intense and entities but it's like as you launch it you monitor it for a while like you be very hands-on with it for maybe a month or a couple of months after you realize that the tool is getting it right for for some time you can just lay off like right now I haven't looked at the boss uncle code based on like a few months already I'm quite confident labeling correctly yeah yes yeah yeah sorry could you repeat the last part yes uh uh add more intent for NDD where it is simple, a period. Okay. So, yeah. So the question was, um, uh, as boss uncle deduces more intents and entities, what happens when they deduce like errors in intents and entities, right? Uh, so normally what happens is you go back to the tool and then you invalidate it. So you, you see what the, the chat board did use. You see what the system deduced and it might be wrong, but when you go, when you see something is wrong, you actually invalidate it. So you literally tell the system, this is wrong. Don't deduce something like this again. So you reduce its confidence level of deducing the wrong thing. And then you, by deducing, by reducing the confidence level of deducing the wrong things, you increase the confidence levels of deducing the right things in general. Yeah. Yes. Right. Yeah. So the question was, um, how do you deal with adversary inputs, right? Like people saying gibberish and trying to train your chat board wrong. Number one is you don't make your system 100% AI driven. The moment you make your system 100% AI driven, um, that kind of allows that, that, that opportunity to have take place. So I'm not sure how many of you know about Microsoft's Tay. Yeah. A chat board that was released like a while ago. Uh, so Tay was released like with really, really good intentions, right? To be like this person that everyone can talk to, but a lot of people started swearing at Tay like in the earliest days. So the next day onwards, Tay started swearing back at all these people because Tay was 100% AI driven. Um, so one thing you want to do is you want to kind of limit your scope. So limit your scope of the intense and entities that are actually possible for the bot to, to learn itself. You don't really want it to be able to deduce, uh, its own entities or, or intense directly from utterances yet. So what they did was, uh, they derived that what when people were sending like these swear words to her, she was creating more intense of her own. When people started with, we like with swears, right? Can I swear? Yeah. So when people start saying some things like, fuck you, right? Um, Tay might have deduced fuck you to be like a greeting, right? So, um, so, so the next time Tay started talking to people, she greeted them with that as well. Yeah. So, so you just, you kind of limit your scope. You say these are my fixed intents and entities in the start as it gets more successful. I gradually add on more intense and entities. Yes. Yes. So the question was, can you sense frustration and do you tailor the message accordingly? Um, when people send these utterances, they actually in its raw form, you can actually deduce sentiments, right? There are lots of tools available that allow you to determine the sentiment of a user based on a single message they send you. Um, so yes, we can always determine like, uh, their frustrations, people who are angry generally tend to use capital letters and, uh, swear as well. And, uh, normally when people are, I mean, are frustrated, we actually have an intent in boss uncle to handle them. So the intent in the back we use is called complaints. So the moment boss uncle deduces that a user was frustrated, boss uncle deduce the intent was complained. Then he would reply with something like a chill, just, just relax. It'll be okay. Your boss will arrive on time. Yeah. Yes. You know what was the original team structure like when you started working on this product? Roughly, um, how, what was the journey like the first version? Okay. Um, so yeah, so nothing's laughing cause, uh, the earliest version was just me. Like it's literally like, it was a side project of mine. So I coded out boss uncle myself, uh, but the face and everything I actually didn't do it myself. I'm not really a designer at all. So I got a lot of help from my friend at trade gecko. Her name is Kyle. She's like one of the best designers I've ever met. So I asked her, so what, how it worked in the earliest days was I actually coded out boss uncle properly myself, but without a face. So it's just like a blank face in the start, right? I told my friend Kyle, uh, can you chat with boss uncle for a day and then come up with a face of a bus driver and she was able to come up with that face in like two hours. So until today, we still use that same face that she created in like two hours. Uh, so yeah, the team was just me and Kyle was kind of like my designer for things. Um, as we got more users, started getting more business slowly, I had to like teach other people what I do. And then that expanded now. Now it's a three person full time and three person part time. Yeah. So now, uh, I'm kind of no more developer. Like I'm very sad. Like in the earliest days I was always a developer, but now I kind of have to manage the sales, the business development and fundraising and accounting and all this crap. So, uh, so now I kind of take care of more of the business stuff. And, uh, in my core team, I had just have one developer for my part timers. I have like copywriters and marketers and designers. Yeah. Yeah. So, uh, the question was, is it difficult to translate this to another language? So when you translate it to another language, you kind of have to create a whole new set of intents and entities just for that other language. Right now there's no technology that allows you to use the same intense map to multiple languages yet. So if you want to start training it for other languages, you kind of have to redo it from the start in a way. So it's, it's not really difficult, but it's very time consuming right now. If you want to have a chatbot that speaks multiple languages. Yeah. Any more questions? Yes. How much training do you have? How do you add an intent? Oh, okay. So the question was like, if you add an intent, how much more retraining do you have to do? Um, normally I just add the intent. I train it with about 10 sentences, 10 utterances myself, then I push it out. Then normally the users train it for me. Yeah. How do you monitor the quality? Okay. It means how do you evaluate the user's feedback? Right. Yeah. So the question was like, how do you monitor the quality and evaluate user's feedback and that kind of thing? Um, so yeah, so this is like super duper important because, uh, a lot of what we did was boss uncle was actually not our own ideas. Like we never wanted to add directions. We never wanted to, to, to add more jokes either, but a lot of users wanted it. And how we actually got to know this was, uh, we did, we tried to, I mean we tried to get two kinds of feedback, quantitative and qualitative feedback. So for quantitative feedback, we use an analytic system in the back. So we use Facebook analytics right now. Um, and in general, and I mean, if you guys have used analytic systems, you know how they work. You just fire off events and then you create funnels. So generally you learn about how people traverse through your user journey through this kind of data. Uh, normally with apps and websites, you fire off events at different points of times. Like for an app, a user might click on one button, you fire off an event saying user, clicked button A and then the user types in something in the field and fire off something saying user typed into a certain field. Then in the back, you'll kind of get a funnel showing like how many users first type the button and second typed into this field. Right. That's generally how analytics works for chatbots as well. It's very, very similar. So depending on the first thing a user says an intent is deduced, we fire off an event for that intent. Then they say something else. The second intent is deduced, we fire off another event for that intent. So that's quantitative. Um, and we measure like in the back, what are the most common things people are saying? What are the, where, where's the place that people like to drop off and things like that. So we kind of optimize the flows there for qualitative. We use our Facebook page. So every chatbot on, on Facebook right now actually needs to be deployed on a Facebook page. Um, a lot of people don't realize the value of this Facebook page, but I'm very glad that, that we're able to like look at the value of it in the earliest days. We use it as a place to get feedback from users. So we actually ask our users, what more do you want? What do you like? What do you not like? And do you like my jokes and, and, and the stuff. And we actually get users saying, yeah, I want this feature or I wish, I wish there was a way to store my favorite buses. And that's the way we get qualitative feedback. Yes. Any other questions? You guys.