 So, I assume that a lot of you wanted to become a sales rep as a kid. I certainly didn't, because it's really, really hard. No one likes being rejected, yet sales people are being rejected all the time, especially at the top of a sales funnel. So hey, I'm Christina Traeger and I'm going to show you how to use AI to exactly help those people and revolutionize the way your business is doing sales. Why do we even care? Because 80% of all sales reps are performing at average or below. We've seen that a sales career is hardly anyone's first choice. They kind of fall into their career after college and rather survive than thrive. I actually met Jessica Smith. She is the former director of business operations at Evernote and she, in our 101, she had confirmed that the biggest turn rate they're experiencing is from the position of a junior sales rep. So, what are businesses like Evernote doing about it? Businesses in the US spend up to 20 billion every year in total to train salespeople. And that number is even five times higher when it comes to hiring new salespeople. And that new hire then takes up to a full year to be fully productive. So we can see recruiting and maintaining a sales team of A players, because you don't want B players, is incredibly hard and very expensive. The good news is though that at the same time, we can see another trend. Gardner predicts that by 2020, 85% of all interactions between businesses and customers will be without any humans involved. Machines can now actually understand natural human dialogue. So finally, after decades of AI promises, it's finally happening. So I really encourage you to be part of that revolution and take advantage of all of its business opportunities. And we've actually found that especially millennials absolutely embrace the fact that they can just talk to an AI, beat themselves and use their own lingo and can interact with businesses on their own time and without having to talk to a human. And as I said, there's a real business opportunity here because you can now actually reach your customer where they already are on your website, on their phones, etc. And because of that, you can easily improve the engagement with that user because you've absolutely decreased the cognitive load on their side. They don't have to read text and look at your website and look for the answers. They can just talk to you and ask for what they wanna know. And because you're now in control of all interactions between your business and the end user, you can deliver a very consistent brand experience. And once your AI is trained, it works 24-7, no people problems, no turnover. You can experience actually tremendous cost savings and all of that while being highly scalable. Because if you have one bot, you can have many conversations at the same time. And let me tell you about Matt, Matt is this young gentleman over there. I know his emoji looks slightly more handsome, but so Matt and his two brothers founded a coding boot camp called Codify Academy. And they are doing great. They're increasing their revenue by three X every year. And the secret sauce for them right at the beginning was that they would get back to any new lead within five minutes after they had left our contact information on a website. Last September though, Matt and his two brothers were absolutely exhausted because of it and they just couldn't do any more calls. And this is when he contacted me. And together we wanted to solve their three major pinpoints. One is that no matter how hard they were working, it was impossible to get back to everyone at time because what happens is 2 a.m. in the morning, right? Second, they'd find themselves answering the same questions over and over and over again. And three, the majority of the people they talked to didn't even end up buying, so it's an absolute waste. And this is when we created Bob. Bob is a AI sales assistant. He talks to all of the potential leads and tells them about Codify Academy and answering all of their most basic questions. He lives on the Codify Academy website and all of their landing pages. He lives in a little chat box answering the visitor's questions 24 seven. He also follows up via SMS immediately after a new lead has left our contact information on the website. And because Bob is working 24 seven to only give the good leads to Matt, he was able to increase the conversion rate by 4x, leading to a 10% increase of the annual revenue made by Bob, within five months after we launched this. It's absolutely incredible. So we're actually currently working on integrating Bob a little bit more into Codify's processes. So not only for top final sales but also for in-person meeting follow-ups, for referrals, even for customer off-boarding, right? So everything that's repetitive can be automated and that's what we're using Bob for. But why am I here today? Bob is built using Node.js, otherwise we wouldn't be here together in this room. And we're using IBM Bluemix and of course Cloud Foundry. And the reason why we chose these technologies is, well, Node.js because of its non-blocking feature, we all know about that. Very important for an application like that, because it can have many conversations at the same time. And then we chose to go with Watson as our cognitive intelligence service provider because, I mean, we didn't fall in love with Watson a couple years ago when they debuted on TV, winning Jeopardy, hello, come on in. I actually met Bill Jones, who was the guy who lost against Watson on TV. And even he loves Watson, he's very interested in what we're doing with that. And then for all the fun part, like managing, hosting and operating our application, we chose to stick with Bluemix and Cloud Foundry because for me, it was really, really easy to deploy our application in seconds. I didn't have to worry about any infrastructure settings. And as I want to, I think I pitched it to Matt, he's a business guy, as like the iOS of all the platform as a service tag. And so I was able to build a fully automated pipeline without any experience in DevOps. And also, frankly, I do not have to worry about how to operate and set up a web server because when you build a startup, there's only three things to care about. If you want to iterate fast, means you have short development cycles, with an increasing workload free application. And because you want to be fast, you need very simple processes. And one of these processes is running and deploying your application, right? And then also, all you do should be scalable because that's the whole point of being a startup. I think a fun story here is also that Facebook used to have this motto that says, move fast and break things. And right after the APO, they switched it to move fast with stable infrastructure. And I told Matt, if we stick with Boom Mix and Cloud Foundry, we can already have some stable infrastructure. We don't have to wait for our APO. So for us, this is just a very ideal development ecosystem. Let's look at our design real quick. So we are a very thin web-based client. We are fully connected to a server all the time, no local processing, of course. We also are connected to a, to Google's Firebase. We're using Google Firebase because it's a real-time Firebase. It's very easy to use, can be set up in seconds. And this is where we store all of our conversations. We are pretty communication with the server. We needed a WebSocket because we are a chat application, right? So we need some real-time bi-directional communication. And for that, we're using a very popular JavaScript library called Socket.io. Socket.io comes in two parts. You can have a client-side JavaScript library or a more service-side Node.js library, and of course, we went with a lot of one. We're using Socket.io alongside Express. That's a very common combo, I'd say. And it works because Socket.io doesn't work with HTTP requests. It's event-based. And so we can still use Express to distribute our initial client script. And what I mean by client script is our client setting script is that every of our clients, like Codepy Academy, or customers, if you want to say, they have their own settings. They have their own login credential we need for them to authenticate with third-party services like Albin Watson and the Firebase. We also have different settings like the name of the bot. They have all different pictures. They might be using different collars, et cetera. So as soon as the user decides to communicate with one of our bots through our two user-facing components, which is the web browser or the VSS in our phone, we authenticated user and we create a room for them. That's how we call them. So let's look at how it works from question to answer on a very high-level bird's-eye view. That's me on the top left corner, so I usually wear a beanie. That's my version of business formula. And I'm asking the number one question that every user is asking in every conversation, no matter what the product is, how much is it? And as soon as that's happening, our app communicates with Watson and we tell Watson, hey, this is the question, and then Watson starts its magic. And Watson is using the training data that we've been feeding it before to kind of determine what kind of question the user is asking. And it sends us back in an array with different causes. And they're up to 10, I think we're working with five of them. And they all come back with the cost and then what's called the confidence level. And the confidence level is from zero to one or zero to 100%, whatever you want. And the better your bot is trained, the more likely you have that your top number one cost also comes back with 100% confidence level. So for example, for how much is it? Bob currently knows more than 300 different ways to ask for the price. So I'm pretty sure we would get back the top number one cost with 100% confidence level and this label is internally called a total cost. And then I take this label and connect with our Google database. Because from there, I retrieved the actual human readable answer. Because your user doesn't want to get an answer back total cost, right? They want to actually have information. And so we use this label as a key and retrieve the actual human readable answer and then send it back to user. So it's not a rocket science, it really isn't. And the key takeaway from here is the integration with IBM Watson as a cognitive intelligence service provider. And I will quickly show you how this call to Watson works. And that's actually snippets of our code. We're a Node.js project, so in your app.js all you have to do is requiring Watson alongside all of your other NPM packages. And then also, of course, giving Watson all of the credentials for that specific client. So we have a file called room.js, then you can see what's an object down there. You tell Watson what kind of service you want to use. In our case, it's the natural language classifier. You get a username and a password. So all of our clients get their own service wrapper and you get your own username and password. And then you need something that's called a classifier ID. And a classifier ID is something, it's kind of like a references to a specific snapshot or brain part of Watson, right? So if I talked to Watson, I have to tell, okay, use this ID and this is like the bucket that I want to retrieve the answer from. And this classifier ID also updates every time after you've trained it. So this is also something that's important when you start out and you build out your application and everything's manual. Make sure you update that classifier ID every time after you're trained it. Otherwise, you will refer to old knowledge. And then easy peasy, we create a new Watson instance using all of our client settings, we write it to a instance. And then on there, we call an Lclassifier.classify. Parts in the parameters, the parameters here again is the classifier ID. So I tell Watson, okay, please use this ID in this bucket of your brain. The text in this case is the question. And then define what you want to do as soon as you get back all of these replies. In our case, we do a little bit of logic around, has this question already asked before? Do I want to give a slightly different answer? How is the distribution of the confidence level? So as I said, you get back up to 10 different labels with different confidence levels. And if all of the confidence levels are kind of like below, let's say 70%. And all of the confidence levels are like 70%, others like 30%, 20%. You already know that the bot is not really sure what to answer. So you either can ask a more clarifying question or you decide to, for example, go with an I don't know, right? So that would all happen in your callback function. That's been rather technical or not technical. I'm still not sure what that technical level of the audience is. So I invite you all to go to codifyacademy.com and actually talk to Bob. So this is your opportunity to grab your phone without me feeling like you don't pay attention, so if you want to do that. We can also actually, maybe we can drop, we can do this here a little quick. There you go, you can see that all, there you go. So, let's see. So that's our codifyacademy website, and if you decide you want to talk to Bob, you can find the chat room here. So I obviously already have talked to him before. So that's what we store in our room, JS. You can see, okay, tell me about codify2. So a lot of information and I'm like, okay, cool, that's interesting. So how much is it? All right, awesome. So after, and this is actually a conversation that Matt used to have all the time. And from our analytics, we can see that literally you'll a call with one of the codifyacademy salespeople. And also if you see some of the answers, you could say that, I mean, kill all humans. It has quite a bit of sassiness and personality to it and we did that on purpose because we find that any emotional reaction, be it positive or negative, is something that we want to have. Because it's just absolutely incredible that people talked about like a human and they know it's a bot and it's just a few lines of code. And if there's one piece of advice I want to share is make sure your bot has some kind of human like personality. I can, I live in a house, that's actually a good example. I live in a house with 15 other people in Head Ashbury in San Francisco. We're all quite techy or nerds that work at Google or anything. So we have all of the devices, right? So we have an okay Google and we have an Alexa at home. So okay Google is in our kitchen and Alexa is in the upstairs living room. And you can just tell just because okay Google doesn't really have a name but Alexa has how differently we interact with the both devices. So with okay Google, for us it's just an extension of their search bar. It's what they want that. So we kind of like yell at it. It's like okay Google play brunch music and or we get even we get very angry if it doesn't get our request right. Whereas with Alexa, you can tell like we're like, hey Alexa, would you mind buying the new Apple HomePod? But so just make sure to use a name that people can remember and give it a personality and people will interact with it way more. Back into the presentation and giving it the personality is what I call the first of four building blocks. So this is basically your to-do list. If you want to walk out here today and know how to build a sales rep, this is exactly what you have to do. Step number one is creating the scope. So being very clear about not only personality but also what should my bot know, like what are the questions that I want him or her to answer. So if you already have an internal FAQ or if you have a customer's support chat or you have some emails that you can reference, great. The more questions you can feed your Watson or whatever service provider you want to use, the better. And that only talking about unique question but also different variants of the same question. After you've done this, you create what's called a training CSV and that file contains the question in the labels laid out how, for example, Watson wants, needs it in order for it to be trained. Then in the second step, you create a reply database. Because you have to somehow make all these real replies accessible for your bot. When we first started out, we didn't even have a database. We just had a JSON file in our application and wrote it into there. Cannot recommend it. So I would start out with an actual database right from get go. So after you've done that, all you have to do is train the brain. You either can use Watson's interface, they have a web interface for that. Or you can just use their node SDK and do it right from your application. As I said, after you train the bot, you get back the classifier ID and then use that classifier ID to actually build the rest of the application. Cuz you need that ID again to use their algorithms. And with the rest of the application, I mean, hey, what's the user interface, right? Do I want a web browser? Do I want that to communicate via Facebook? Do I want it to communicate via SMS? And also, make sure you build a second database to store all of your conversations because you want to read what your bot is talking about, right? Especially at the beginning. And that brings us to point number four, which is tracking the performance and building a dashboard. And I cannot stress out enough how important it is to have an easy UI for your non-technical people to follow along what your bot is doing. Because you want to get your bot up to speed as soon as possible. And so what we did, we asked our friends or we outsourced it to other people to really read through all of the conversations and figuring out, okay, what are areas we need to train it on more. And this is how our dashboard looks like from an example. I don't know, okay, one second. So you can see that we have a conversation. We just use Angular for that, an Angular material. It took us like a day or two to set this up. So you can just follow along all of the conversations. You can see what classifier it's been used and what confidence it came back. And then on the left side down there, you can see there is a part that's called training, and this is where you can actually feed it the human readable replies. Let's go back, and this is for example, this is the training file I was talking about earlier, very simple, literally everyone can create it. This is how Watson, for example, needs us to give the new answers to them. So in the quotas, you see the question, and then comma, and then just the name of the label that you want Watson to know and keep you back as response. Also something that's not talked about a lot of times when it comes to chat bots and AI is how important the training is. So four major best practices that I want to share with you that I want you to know is review the don't-knews on a daily basis, especially when you start out. You also want to update the confidence threshold, right? So when you build out your first bot, I call them baby bots, you want to have a very high threshold. So you don't want to go with an answer that the bot is suggesting that's like below 70%, no, even 80 or 90%, whereas now, for example, with Bob, that's a very mature bot, we have a confidence threshold of 0.01 or something like that. Because even the answer that comes back with 25% still kind of fits, because the base is just way higher, right? So you want to update at confidence level over time. And of course you want to automate a lot of your training. And do it in a consistent training cycle. We do it once a week now for all of our bots. And so the major takeaway also from the session for you is have an easy training UI to put in new data to track the performance. So we are tracking KPIs like how many conversations did we have, how many questions have been asked, what our customers are actually, where do they drop out, right? What is the last question before they decide to drop out? Or what are things that customers are asking for that we are not offering at, right? So running this data and having this analysis ready is incredibly valuable. And that's why again, it's very important to have this dashboard ready right from the get go. Yeah, and with Bob, we've been extremely successful and we wanted to share the success with other businesses. So if you have any more questions or if you want your own Bob working for you, reach out to me. We founded a new company called Cyber Industries. And if you have any more questions, this is where you can reach me at. Thank you. I was just wondering for the training, how often is it like you review one of the conversations and just wish that one or two of the questions was a little bit different? How often so? The answers I mean, sorry. Yeah, well, at the beginning it was a lot more. So I think it takes up, it took Bob I think to be fully productive about eight weeks. It really depends on how many conversations you have. So you shouldn't launch a bot without having at least like 100 to 200 conversations. And how you can do that is when you first launch a bot, you launch it in a sandbox and you send the thing to your friends or to people in Odesk and they just have a very basic conversation and then you can train it on that, on the questions. Yeah. Thanks. How many questions does he get now per day? I mean, yeah, we have about, we have about 100 for a codified academy. So 100 conversations per day? Yeah. That many people are coming there to talk and ask questions? Yes, yes. Because we only, so we do both things. We have the inbound side, we have the web chat on our landing pages and on our website, but we also do the outbound as soon as we get their contact information on any, you know, other side and whatever. So, yeah. Yeah. So I have three questions I would ask one by one. Yeah. The first question is that how do you really define a success of a conversation? That's a good question. It's up to you. So for us, it's, we have a, we have a flag in our training dashboard. And I wish I could have shown you that. Training dashboard that it's called excellent. And so we actually give only this excellent start when he gets all of the answers right and it happens more than 90% right now. And what's the average time spent in conversations like 15, do you measure the metric? You mean how many questions been asking? No, how much is the engagement time per user to the bot? So, we don't measure the time, but we measure how many questions the user asking, using or asking four to six questions. So the outcome of four to six questions, they get an output, what are they looking for? Yeah, yeah, because that's, and that's exactly the point that I'm making is that, especially for top final sales, like, you can almost predict what your users are going to ask about. Like, that's why you have your FAQ, right? It's like frequently asked question and come over and over and over again, which also made, you know, mad stop so painful in the first place. Last question, who designs the questions? So there is a new concept called conversational UI. So in your team, who's a designated person who keeps. Designs a question, we, we take them from users. So every time we get asked a question, it's not in our database yet, we add it there. Is that what you're, or do you mean design the replies? No, design the questions initially, right? Which path this will go? Yeah. So you have some assumptions, people who ask this question. Yeah. Then you go to a path A or B. We don't do, we don't do path right now. So, because this is, so the, the paths we're doing for the deep, a little bit deeper integrations now that we have. But for the first, like top final sales, when you use your lens on a website, I can assure you nine out of 10 times, there will be, what do you do and how much is it? If you can come back to the slide with architecture and elaborate a bit. I believe it is not the kind of initial architecture, but the final for the moment. If you can. This one? Yeah. Yeah. What was initial and why you choose this current? That's the first one. First and last one. That was, that was the first thing we did. So that's, I'm, when you're a startup, like you only work with, you know, what's new and inaccessible and all of these technologies are. So for example, Watson was very easy to use with BlueMakes because they're already integrated. Again, Google Firebase database is something that, because my skills in back end are very limited is something that I'm very thankful for. Cause I'm able to host and manage database without a lot of knowledge. It's literally just like another API that you can use. And socket data or express, again, it's a very popular combination that works very well. So, yeah. One last question. We have time. I have one question. Okay. It's not specific to your application, but generally all these bots, right? Yeah. If you ask some question and the answer is not satisfactory, we tend to ask it in a different way. We try to word it in a different way. But I've seen sometimes it'll still give back the same answer. So in such kind of patterns, do you, does the robot thing go, does the bot thing go? I'm giving the same answer again and again. Maybe this guy wants something else. Yeah. So that's why we actually prevent giving the same answer. So that's like what part of the logic that goes behind when we get back to replies we like compare if the user has been asking or if we have been giving the same again and then we often depends on what the distribution of the confidence of us, we tend to give the second cause, for example. So yeah, good question. Okay. Thank you very much, Christina.