 I'm Andrew Lewis, I'm going to be talking about talking to myself, but before I start I'm going to talk, if you've ever talked to me, you probably know what I'm working on. I'm trying to put my life into a database, so pretty much every aspect of my history I have imported this for, and they're going into this database. I'm not going to talk about that today, but you can read about it on my website if you're interested. But it's not just modern data that I'm trying to get my hands on. In high school I used to use MSN Messenger, and I saved all the transcripts. So I have this giant log of everything I've typed for last many, many years. Did anybody use MSN? Hands up if you've, yeah, cool, okay, that's better than I thought. So MSN was this cool software package that taught high schoolers how to type really fast. You could also use it to talk to your friends, that was one of the features. You know, obviously in high school you don't have too many interesting things to say. So most of my logs are things like this. You know, they're really not great conversations. I had this idea that I would save the logs and maybe have a reunion with some friends and read them in the future. This didn't really happen. So I have all these logs. I didn't really know what to do with them. So last year at BangBangCon I was talking to Sasha about these logs and this data, and she had this amazing idea. It's like, you should train a chat bot that knows how to talk to you based on these MSN logs. And then we got to jamming a little bit on the idea, and it evolved into this. So one chat bot on my MSN logs from high school, talking to a modern version of myself from more modern chat logs. This is a great idea. So yeah, I decided to give it a shot. So just to see what we're working with here, the person who I was in high school was quite different. I had some weird political beliefs, I had different ideas of a style. I was really into Pink Floyd. So I think if we put these two chat bots together, we might have some interesting conversations. So what we're going to do, we're going to prepare the data. We're going to train the chat bot to speak like I did in high school. Then we're going to get it to respond to questions. Finally, we're going to get the chat bot to talk to a modern version of me, and then who knows what will happen. So first step, preparing the data. I don't want to talk about it too much, but this is really the part that you underestimate. It's just going to be awful, and you think you're done and you're not. So just spend a lot of time on this. But I'm going to move on to the more interesting part, making it sound like me. So for this, we use a deep learning technique called RNNs. These are, if you've ever seen a Twitter bot, there's a good chance of using something like this. And what an RNN does is it creates a vocabulary of every word in a corpus. It assigns an ID, and then it figures out a sequence of likely tokens. So we can do this on our MSN logs. And one epoch is basically going through the data set one time. So after a little bit of training, we're basically getting random noise from my past. So this is basically just a random sampling of words that I've used in the past. A bit more training, we get a little bit more form and shape that come into the words. All your, if you've noticed, I had a typo in watching. So all your typos and auto-corrects are going to come back to haunt you. This is something I've learned. I was talking about this with my friend Max. He auto-corrected really badly. And then every time I see this now, I'm thinking, it's going to show up in a future bot. So after 10 epochs, this is not too much again, but the structure is emerging a little bit better. And after 100 epochs, we're getting something that seems sensual. It doesn't really make sense semantically, but it sort of sounds like me. So that's cool. So we're going to move on to the part now. We're going to teach this bot to understand and respond to questions. So what we're going to do is use a technique called sequence to sequence. This was pioneered by Google a couple of years ago, and they were figuring out how to do machine translation. So what you do is you take a sequence of inputs. You create a vector that represents roughly what it means. And then you feed it into another decoder RNN that gives you a sequence of output words that are likely based on the input. So you can use it for translation, like in this example. But you can also use it for generating conversations. So we feed it like Floyd, and we get out, of course. So this is generally when people are using generational conversation and chatbots. This is what they're using. So slowly a teenage Andrew emerges from hibernation and can start responding to questions. So let's have a look. Cool. Yeah. So it's slowly learning how to respond, a bit more training. So what I generally seek to seek will select for the easiest responses. And generally, it will learn very early as law works in almost every scenario. So I was talking to my friend, Andre, about this. And he started reinforcing this law behavior. And basically, my future chatbots are screwed going forward. So after a bit more training, we get a bit of semblance of understanding. So how do you say something I used to say a lot in high school? I thought it was a cool way of saying hi. I can ask you questions and know how to respond. Talks like a teenager. So it knows how to understand these kind of basic questions and give something that sort of makes sense in response. It has a little bit of an attitude, but it knows how to respond to questions, which is nice. So I think we're ready to get us to talk to a chatbot that's based on a modern version of me. So just a reminder, so we have Emerson Log Chatbot with young Andrew Lewis and talking to a modern Andrew Lewis. This date is coming from Google Hangouts from the last couple of years. So let's have a go at it. So these are generated, you know, not really the deepest questions, but they kind of have a bit of a flow to them, which is fun. Doing anything last night, no. Going to school, no. So I searched my training material. I don't remember ever saying there's a good lad. So this is like innovative. I think it's like come up with this phrase. I'm pretty proud of that, I guess. So let's have a look at some more output. You know, old Andrew was asking young Andrew about big plans for the future. I don't know how this happened. I swear this is all real output. I said, yeah. So you can kind of understand why some of this is happening, but it's really hard to debug these things. But this is the output I got when the two of them were talking to each other. I think this is the kind of level of conversation I managed to generate. So clearly, it's not the wittiest conversation. So when you're creating Chatbots, generally the ones you see when you interact with a Facebook bot are in the top-left category. So they're picking a response in a very narrow domain. So these are pretty easy to do, and they're also frustrating to use because they really don't handle a wide range of input. What I chose to do is actually the hardest of these. Is it generating a new response and it can talk about anything? And this is really the hardest type of Chatbot. So when I started working on this idea, I naively thought I would have a conversation, something like this. Young Andrew Lewis shows off the cool band he discovered. Old Andrew Lewis responds poorly. Old Andrew Lewis has a witty response about it, and then it shows some attitude. So this is how I thought it would go. So let's break it down. It's pretty hard to build a Chatbot that incorporates general knowledge into itself. It's pretty, generally the corpuses are available to train on, are pretty small people train on movie scripts, or Twitter, or God forbid, Reddit comments. And this is generally what's available to train these things. And they're not actually that big, and it's pretty hard to do really good conversational bots with them. It's hard to give a bot a stable personality, something that can respond to emotion and can be consistent over time. There's even a more philosophical point, which is that Chatbots don't have body. So it's hard to train them to respond in the way a human would with our arms and the way we perceive senses. And then it's even hard for something simple like this to have a working memory where a Chatbot can respond to something. When it can remember something it said in the past and then use it in your response. So Kamal yesterday had this talk about a problem that was unsolvable. I really like this title. If you could solve this world-telling problem, you could solve the halting problem. I think a title from my talk would be, if you could generate witty conversation with yourself, you could solve the Turing test. This is not really possible right now. So if you haven't watched Black Mirror, the Turing test is this test where you have two actors behind a curtain and one's a computer, one's a human. If you can trick the human into thinking that it's another human, you've passed the Turing test. Unfortunately, this is more or less equivalent to producing general artificial intelligence. This is years away. So this is a pro tip that I learned. Before submitting to a conference, make sure that you're not committing to solving a research problem. But I think on the flip side, I learned a lot from this. And plunging naively into a problem is a really good way to learn about that space, even if you don't actually solve what you were trying to solve. And then besides the point, sometimes it's really nice to have someone to talk to in the middle of the night. Cool. Sorry. Cool. That's great to say, yeah. That's a little bit of Canadian coming into it. Cool. Thank you.