 Say hi. Did you hear me? Is it because of this? Hey, Cortana. Say hi. Hi. Okay. There we go. Okay. Cool. All right. Hi, everyone. I'm Alyssa. And I'm a software engineer working with the commercial software engineering team at Microsoft. So what we do is we work, we code with customers and partners to build Epic stuff, basically. So today, I'm going to be talking about building bento bots with NLP models. And I was making my slides, just kidding. I wasn't making my slides yesterday, but I was working on them. And my friend sort of looked over my shoulder and he was like, okay, what the hell are bento bots? Can you please explain? So I was like, okay, the title is not very explanatory. Sorry, explanatory. So I'm just going to go through what these are. So who's used the bot before? Probably most of you by the end of this conference. Cool. So typical bot kind of looks like this, okay? So let's say I own a fried chicken restaurant. And my bot can actually handle a limited number of things in the domain. So for example, people can ask me what my opening hours are and they can place bookings in my bot. So how I would train the natural language model behind this using a third party tool like Lewis, maybe using Dialogflow. So how I would train it would be something like this. I'll make three different intents of what my bot can handle. And then I'll train some utterances under those intents so that the bot can classify them. So at its core, it's text classification. So when the user says they don't have to say the exact phrase. They can say I want to make a reservation table for four. And the bot should be able to classify that into make booking. So that's all fine and dandy. But consider this scenario. So the user says hello. Okay, so what do we map that to? Maybe if you're using Lewis or Dialogflow, you'll say hello intent. What if the user says how are you? What would you do? Map it to how are you intent maybe? What if the user says tell me a joke? Okay, one more intent maybe. But there's just so many things that they can say to you. So how do you actually handle this? The issue with using Watson, Lewis, Dialogflow is that they often have intent limitations. For those of you who have tried this before, you'll be familiar with this. So I think Watson and Dialogflow have about 2,000 intent limit. So the question here really is how do we handle small talk? How do we handle, you know, users love to test the limits of your bot. So how do we handle all the rubbish that they're going to throw at your bot? Okay, so the first step is to really abstract the relevant noise all into a different intent. So when you train your natural language model, you put, you know, all this other random stuff that the user could say, either in the small talk intent or I don't understand intent. And then that routes to the small talk module inside your bot. So today what we're really going to be exploring is what are the techniques that we can use to handle small talk inside your bot. All right, so before we begin on talking about the techniques, it's really important to understand really the different types of NLP models that you can have or the different kind of conversational bots that you can have. So this is what we can do currently in the near future. So that's the idea of open domain versus closed domain. Close domain means there's a limited amount of stuff that your bot can talk about and the scope is really, you know, controlled by you. Open domain means if someone is talking to my fried chicken restaurant bot, they'll be able to have a conversation about home loans at the same time. So yeah, your bot has to be able to handle anything out there, which is why it's so difficult. And right now we are looking at closed domain. So there's the idea of retrieval-based versus generative-based. Retrieval-based is when the responses of your bot are all predefined. So if someone asks me to make a booking, the bot says, sure, what time would you like the booking to be? So all the responses are prescripted by you. Whereas with generative-based, your bot is actually creating the answers on the fly. And we're going to be discussing how to do this as well. All right, so here are some of the options that I'm going to be talking about today. So under closed domain and retrieval, Google Home is still talking to me for some reason. Under closed domain and retrieval, we could use a third-party NLP service. So I'll be going through Microsoft's Q&A maker, which is like the easiest way to get set up. And then we can look at medium difficulty options. So that's custom classifiers like maybe a classifier from scikit-learn or node-natural package. And then there's a cool Python package called chatterbot that I'll be discussing as well. And then lastly, we'll be discussing the hard option, which is the RNN. Is this a clicker? You've got to plug it in. All right, so the easy option, this is using a corpus. So that's a body of text that you want to train the bot on. And Q&A maker. So Q&A maker is like a knowledge base as a service. There's no limitations on the intent, but it's a 20 megabyte knowledge base limit, but that's still a lot of text. So I'll just quickly demo how that would look like. Okay, so yeah, shameless plug, but never mind. Hey Google, be quiet. Oh, it keeps talking to me. Okay, so basically what I have here is a small talk module for the bot framework, but really you can use this for anything else. I've just used the Azure Bot Service. So there's this TSB file that will act as the knowledge base for your bot. Dialogflow actually has a small talk module, but they have intent limitations as well. So I actually took this out of Dialogflow and put it into a TSB file that you can use anywhere and train any model you want. Okay, so we have the question and we have the type of answer that it relates to. So let's go to Q&A maker and try to create a new service. So you can name your service. I'm just going to call it small talk demo. And then, so this takes 30 seconds max maybe. So everything is now loaded into your knowledge base. Okay, so I can say something like, how are you doing? Maybe I say it's my birthday. It's not that. Yeah, so there's like, I think 1,500 different utterances in there that it's trained on. So this is something you can plug into whatever you're building. So if I click on publish, you can just easily publish it as a web service just like that. And then in my bot, what I have is a reference. So these are the different responses that could match to this intent, for example. So if I go, okay, let's demo this. Who wants to say something to it? I can't think of anything. Tell me a joke. Yeah, so you can see now I've just, what I've actually done in the code is plug in the, just make a post request to the Q&A maker that I made. And then do a match to reply with the appropriate response. So yeah, that's how you can use Q&A maker. And the pros and cons of this are it's super duper easy to get started. So as I showed you, you know, it's really easy to get started with deploying a small talk model as a web service. The issue is that it's, sorry, and also control domain. So you know your bot won't go crazy and say weird stuff because you script those responses. And also it's NLP as a service. So Microsoft has researchers and development teams to literally maintain this as a service. Now the cons of it is that creating that knowledge base can be a very, very manual process. And there's no variation in your responses. So they're pretty static and there's a 20 megabyte limit to it. Cool. So with that, let's move on to maybe using a custom classifier. So this is also retrieval base close domain. You can really use the data that I provided that 1500 lines and train your maybe classifier on node natural or, you know, like use a base, a naive base or SVM on scikit-learn. So what happens is then when the user asks something to your small talk module, you will match it to the appropriate response. So same concept as before, but using, you know, an open source classifier instead. All right. So let's click in demo using natural. Okay. So I've written this small script over here that basically takes in the TSP file and adds it to the classifier and then trains it at the end. And then once it trains that classifier, it saves it as a model that you can load in at any time. Okay, Google. Be quiet. Okay. And then in the code, you can then load in your classifier. So this is my classifier that I've loaded in from the JSON file. And then you do a classifier based on text. And then you just respond using whatever intent that's being matched. So it's a same concept as before except using, yeah, except using node natural to do it instead. This isn't on my repository yet, but I'll be pushing this sometime today or next week. Hey, so for variation, let's try testing it on one of these devices. Okay. Hey Cortana. Nice to meet. Okay. Go away. I lost the thread there. Okay. Can you phrase it another way? Tell Tiny I said hello. Hi there, friend. So basically I've deployed a bot called Tiny and it's connected to this invoke speaker and it's actually calling the Azure bot service in the back end and running this code over here. Hey Cortana. Tell Tiny I said, are you a bot? I see. Hey Tiny. Sorry. Hey Cortana. Tell Tiny I said, nice to meet you. I think this is the beginning of a beautiful friendship. I think so too. Okay. Now let's try Google. So I've deployed the same Azure bot service on the Google home. You can deploy it to Alexa as well. So, you know, you can really create a bot using bot service and deploy it to anywhere you want. Hey Google. Talk to Small. Sorry, wrong one. Hey Google. Talk to Tiny demo. What's your name? Tell me a joke. I'm bored. I don't know if anyone heard that but that was a really bad joke. Okay. So yeah, you can see that you can really deploy these models on many different voice platforms. You can deploy it on Messenger, Telegram, et cetera, whatever you like. Cool. All right. So moving on. Okay. So the difference between the pros and cons of using your own custom classifier is that, of course, it's open source. So it's more customizable. Again, there's a controlled domain. So it won't go crazy. And you can really use whatever knowledge-based site. And the cons of it is that you have to really improve this model yourself. So when you build the model, maybe the accuracy isn't that good. And then you have to run, like, a parameter tuning, blah, blah, blah. You have to do all of that yourself and manage the deployment as well. Again, creating the knowledge base can be very manual since it's all retrieval-based. And there's no variation in that. It's all static, the stuff that you've hard-coded in it. Okay. So the third one, the third option we have was Chatterbot. So what Chatterbot is, is an open source Python library developed by this person called Gunther Cox. And he has a bunch of pre-built corpuses that, oh my goodness, they're available in many different languages. Let me put this on the ground. Sorry. I can't help with that, but I'm always learning. So yeah, in the background, this Chatterbot currently uses a combination of search and naive base. Because it's open source, you can customize this, but it's probably a little bit harder. Okay. So just showing you. It's always easier to show you than talk about it. Okay. So this is a Python library. And what I've done here is you can actually put in a, you can plug in, you can plug in corpuses like this. So Chatterbot.train, Chatterbot.corpers.english. And where this is coming from, so this is the main repository, but he's got a corpuses like this. So this is the main repository, but he's got a corpus repository as well. And this has a bunch of data sets, all in different, these different languages. And under English, there's all these different things that the bot can talk about. So yeah, you can see all the things that the bot is trained on. And you can also embed your own custom corpuses that you want. So just one line over here, this one line trains all, trains a bot on all of this. So let's test it. Who is Albert Einstein? I don't quite think so, but okay. What's the meaning of line? That's new. Okay. Never mind. You get the idea. So saying nice to meet you as well. Okay. Anyway, yeah, you get the idea. You can see the thing is that a response that doesn't make sense is sometimes better than no response at all because it's a bit funny as well. Okay, so the advantages of using Chatterbot is that it's more customizable, open source as well. It's really similar to the advantages of using the previous few options that we had. And the disadvantages is also similar to the previous few options that we had. So yeah, it just uses search and naive base at the moment. But I mean, you can tweak that on your own to improve the responses. Okay, so I think I'm running out of time. So we're going to go straight to the RNN. So this is a little bit harder. And this will let you create generative bots. So not just, you don't have to write in all your static responses that you want your bot to answer. You can actually get the bot to make its own sentences based on data that you've trained it on. So the idea behind Recurrent Neural Network, I should explain this. So one-to-one, that's not an RNN. That's a classic model. You have a fixed size vector, usually, and then you have a fixed output. So for example, if you're trying to do image classification, you put in one image and then you classify, is it a cat or a dog, for example. So put in one image and then it makes one classification. That's the one-to-one. And then RNNs are sort of this sequential hidden layers. So one-to-many, for example, you put in an image and it generates a caption. The outputs are many because when you generate a caption, your caption kind of has to make sense as a whole. So maybe like a person sitting on a chair. All the words all relate to each other in the same sentence. So many-to-one would be, for instance, if you're taking in someone's tweet and you're trying to predict whether it's whether it's positive sentiment or negative sentiment. And then lastly, many-to-many is kind of the thing that we'll be doing today. So let me just explain it through here. Okay, I'm running out of time, so I'll just quickly explain what this is doing here. So in an RNN, 101, you're trying to put in input, oh no. Okay, let me just duplicate this. Anyway, in a nutshell, your hidden layers each depend on the previous layers. Your hidden layers each depend on the input of the previous layers to come up with the next prediction in a nutshell. So that's why RNNs are good for if you need a context of the previous words to come up with the next word, or character in this case. Okay, so quick demo of the Cornell Movie Corpus. So this person over here provided a model, thankfully, so I didn't have to write it myself. But I trained this in an Azure N-Series VM for about 16 hours, so it's conversational exchanges from 220,579 conversations in a movie. So let's see what that looks like. So the results were okay. It was kind of hit and miss for some sentences. Okay, I need to go inside. It's using TensorFlow 1.0. Why is this name so long? It's going to take a while to boot up. So it's just restoring the model. Okay, so if I say hello, how are you? Okay, I'm not sure why it's doing that. Maybe I'll come back to it. Okay, let me just demo the second one. I don't know why it usually doesn't do that. Oh, well, live demo, so I took the risk. Okay, so that's the second one. That's a RNN trained on Reddit comments. Everyone knows this is a super bad idea, but never mind. Let's try it anyway. So warning, the chatbot may swear. It may say some inappropriate things. So if you can't, if you don't like that, then yeah, you can turn around or something like that. So these are some funny lines that I got from it. So it took about five turns or four turns before it started calling me an idiot. So that's the state of what Reddit is. Sometimes it can even be sweet. So it says waves and hugs you. So obviously people on the internet put actions around stars and the RNN has kind of learned how to do that as well. This one is interesting. I have a pretty good relationship with my boyfriend. It's like, okay, sure, good for you. And then this one is like, it kind of came up with something that didn't make sense. And then it was like the VEF and it said, thank you for the insight. So this is a pretty good one as well. And then this one was kind of disturbing because I was like, are you going to kill humans? And then the bot was, what does that have to do with anything? Are you even self-casting? And it kind of diverted the topic. So it's like, oh my God. Okay, okay, so hopefully this will work. Where did I put it? Sorry, I know I'm running kind of over time, but let me, this is quite interesting. So, oops, I have to get over this. But it doesn't say... Oh no, no, no, okay. Okay, let's start that again. No one put this on social media, please. No, but I cleared it pretty fast. So, okay. I was hoping I wouldn't say that. Okay, does anyone want to shoot something at it? Please don't say anything offensive because it's going to say something even worse. Okay, does anyone want to try? Anything. I'm not sure it's going to reply with something that makes sense. So you can see it's predicting character by character. Oh, no, no, no, no, no, no, no, no. If you weren't allowed to pay that, okay. What happens if you say this? Oh, this is, okay, does anyone want to try anything? Yeah. I think what? Don't be offensive, please. Oh, what do you mean? Who wants to say something to it next? Yeah, so yeah, interesting results. I better, sorry. Okay, let's try one more, one last one. Okay, it's asking what you mean by that again. Okay. Anyway, yeah. Definitely try this report out if you've got time. Okay, so quick bits of the last two slides. So the good thing about this is that it's generative. So you train it on a whole body of text, and it can generate some really entertaining stuff. A bad thing about it is that it's really hard to control. If you guys remember Microsoft's first experiment called Tay, that totally went off the rockers. So yeah, that's one of the tough things about using RNN. Okay, so to summarize, we explored these different options that you could use, close to main retrieval versus close to main generative. It's not either or. You can choose to combine them if you want. So just really experiment, get into it, and have fun with it. Cool, thank you.