 You quickly find that it falls apart. So the other thing that's big these days is chatbots, right? I think you see a lot of chatbots, a lot of frameworks, a lot of things with them. But there is a difference between what we do, which we call agents or assistants, and chatbots. And for the particular case that we've chosen to solve, here's the difference. So imagine you're busy excited, you're trying to get a meeting together. And if you were to use a chatbot, you want to say something like, hey, it's getting a meeting for us next week, right? And what a chatbot will say is, well, what day would you like me to organize that meeting? So if you try this with Siri, it will do exactly this. Then you say, well, maybe Monday or Tuesday. And then, okay, really in a quicker. And it can't decide, right? You have to give it very specific instructions. So you say Monday, and then it'll say, well, what time? And at this point, you're like, I just want you to get the meeting together, right? And that's the difference. When a real agent or a real assistant would do for you, you just take it off your hands, right? Take that ambiguous instruction of schedule a meeting next week, sometime next week. Figure out what's good, right? Confirm it, put it on your calendar, and send an invite to make sure that everything's done. Yeah. And that's the real difference between chatbots and agents that we see. Okay, so let me just quickly introduce Evie. She's an intelligent assistant, schedules your meetings, and how does she do this? She reads and writes emails, interacts like a real person, right? So it's like a CEO's assistant, follows up, books, meeting rooms, learns your preferences. Exactly what you expect from a real human assistant, right? That's really the bar that we're trying to hit. And did you know only 45% of your work day is spent doing your job, right? A lot of it is spent on admin tasks, and this is really the broader picture that we're trying to address and starting with scheduling meetings, yeah? And if you're a mid-level manager in a big corporation, easily six weeks a year, it's spent scheduling meetings, and that's what we're getting back for you, right? Is it a quick video, I think? No, let's get that, okay. So let's talk about speaking human now, right? So this is Evie, this is her assistant, it reads and writes, doesn't sound too difficult, we do it every day, right? But let's take a quick look at why it's such a hard engineering problem, lunch. I mean, think about the word lunch, you and I have a shared experience of what that means, right? A shared common understanding of what the word lunch means. It's a meeting, right? We typically involve a face-to-face meeting, you can't have a lunch call really, I mean, to extend more of a call, there's an actual face-to-face lunch. It's got a certain time range, right? Typically between 11.30 to maybe, and then the latest time you would end your lunch is maybe about 2.30, right? Anything later than that, you probably do a coffee instead. And it needs a venue, right? And there's typically food involved. So from that one word that you and I share, there's actually a whole deep level of understanding that the machine doesn't know about, right? And the machine needs to be taught about. And this is just one word. Imagine all the words that we use in the context of scheduling. So that's the first reason why it's hard. Then there's the whole syntax. Sorry, got it the wrong way. Syntax versus semantics. So 3 at 5, I have to run at 6. So again, it comes to the meaning of the words today. You've got 3 at 5, I have to run at 6, right? And implicitly, that means I want to end the meeting at 6. Then a very similar sentence. I'm busy at 5, but I'm available at 6. And to the machine, and if you throw this into something like word2vec, which is one of the vector space learning models, these are going to look very, very similar to a machine. Because in tactically, if you look at the structure of the sentence, they're very, very similar. But the meanings are almost completely reversed because of the particular meanings of the words themselves. So these are some of the things that make things difficult. A lot of neural nets and big data learning approaches focus on structure. And if the structures are similar, those are things that kind of tend to match, right? But in this case, the specific word, the detail really, really matters. Here's another one. That's me, Thursday or Friday afternoon words. Really simple sentence, right? But if you really break it down, there's a whole lot of processing that you do very automatically. Friday afternoon, pretty straightforward, right? But you know what, try to decide that afternoon applies to Thursday as well, right? And so you're going to have to multiply that afternoon now between Thursday and Friday. And then the other thing is, you got to treat Thursday or Friday afternoon as one element and apply that to the word works, right? And these are things that machines don't know. And these are things that you need to teach them or put in place a structure that you can use in order to get the right result, right? But ultimately, it's about people. One last example. Let's do coffee at Holland Village. Let's say Starbucks at Yapun, right? So first of all, relatively easy to pick out coffee, HV, Starbucks, Yapun, right? But again, you have a bit of this interesting context that's happening, right? In the context of Singapore, it probably means Holland Village. But someone in San Francisco would probably mean something else altogether, right? And then on top of that, you then need to graph the fact that Starbucks and Yapun are very specific places within a region called Holland Village, right? So it's not just the aim to look at the words in isolation, but you need to have this whole model of the world behind you in order to get the right meaning from the text, right? So you have to look a lot deeper. So that's some of the reasons why it's hard. And let's say you do get the complete understanding of the sentence, right? Let's say I'm in Hong Kong next week. Fairly straightforward, it's a fact, right? This person is saying that he will be in Hong Kong over this period of time, start time and end time. Fairly straightforward. But in the context of scheduling a meeting, you don't really know, this isn't like saying, no, I'm not free, or yes, I'm free, right? It's kind of an orthogonal kind of statement, just a statement of a fact. And you need to actually bring in some context around what is it that you're trying to do before you can decide what it is. And this, again, makes it difficult to apply very standard, simplistic mapping model to text, right? So where's the meeting? If the meeting I'm trying to organize is actually in Hong Kong, then next week's a good time. But if it's in Singapore, then maybe it's not a good time, right? Or was I trying to organize a call in which case I don't care, except I just have to adjust for the time zone. And did I have any other personalization parameters? And only then can I choose from a model that kind of sets a decision, right? Whether I prefer the time next week, I find another time, change the meeting to a call, yeah? So, literally, once you have the surface level understanding, you still need to go deeper, bringing the context around the task that you're trying to do, and actually reason from it to get to kind of the right set of possible decisions, yeah? Yeah, and so, you know, if you're ever curious about natural language and what a natural language pipeline looks like, this is a very high-level view of it. It starts with a whole bunch of preprocessing. So from an email, you take an email, you got to figure out, like, what is the actual text that comes out of it versus the five previous emails that were quoted. You've got to separate that part of it, then you have to take that and you have to go and find the signature and extract that because there's some interesting things you need to do with the signature versus the actual text. But then you get the text, right? So you do all that preprocessing, you get your text, and now you put it through this pipeline and you say, well, the first thing you need to do is to tokenize, right? So fairly standard, well-understood task, great things, and the machine just sees this whole line of text, right? You need to break it down into individual words, right? Not as easy as you might think, as you might expect. Then we tag it, parts of speech. Is this a verb, is this a noun, is it preposition, is it past tense, present tense? Is this a plural form of a base word? Then we enrich it with an oncology. That's really a fancy way of taking the word lunch and adding all the information we know about lunch into the pipeline. And then, so now we have these atomized words, now we start building it up again in order to get some meaning out of it, right? And so we take a semantic chakra, the meaning of the word. We talk about the Thursday and Friday afternoon, grouping those things together, figuring out Starbucks and Yakun and Holland Village, and then we start building that into a craft and then, since life is information, rout it to the right tasks, is this a response to an existing meeting on Friday schedule, is this a new meeting? Is someone starting a new email thread but asking me to reschedule another meeting? Do another round of contextual enrichment and then outcomes, structured language representation, yeah? So this is really the core pipeline for the natural language processing. And at a very high level, it's probably consistent with most of the NLP processing pipelines out there. And that's just understanding the language, right? So it turns out to do scheduling well like a real assistant, there's a whole bunch of other things you need to do, right? You actually need to figure out what a good slot is. So if someone said next week, so do I suggest Monday, Tuesday, Thursday or Friday, what time, how many people are involved, what time zones they are, is there a meeting room available, a bunch of factors going to that? We talked a little bit about the location modeling, Google Office, again, like HB, very, very context specific. In this case, it's personal, right? Or you could say Starbucks, and Starbucks to you means this Starbucks, and Starbucks to someone else means another Starbucks. Dynamic workflows. So we've seen meetings that were organized and started out as a lunch, and then someone had rescheduled, turned into coffee, a lot of companies that. How about we meet in another time and it just ends up in a call. So if you're an engineer, you know that you usually want to build for a spec, right? Say, well, my spec says A, B, C, D, E, and I'm going to build so it does A, B, C, D, E. But in the real world stuff is a lot more fluid, right? So another engineering challenge is how do you support that level of fluidity at scale without kind of just getting yourself tied up or not? And of course, at the end, you also need to output something, right? Not only does E.B. read emails, she also needs to write emails, and to write emails, you need to be able to talk like a real person. And a lot of it is actually being responsive to what just happened before or what happened right at the start. And so that when you then take an action, you're able to explain why you did something. And that's another pretty important thing that is critical for machines to be able to talk like people, right? I mean, we respond so naturally to events that we build that into the words we use in such a dynamic manner that it's hard for machines, right? So, yeah. So these are all the fun challenges. And if you enjoy hard problems, come talk to me later, software engineers and specialist data science experts. And now it's question and answer time. Because we have a broad audience. It's really open. What can I tell you guys about? All right. So my question is, how is E.B.'s understanding of language different from, say, plug into something that, I don't know, like Siri, right? Yeah. How is it different? How is it different? I think with Siri and with Trapbox in general, they tend to deal with one line at a time. And they really focus on, at the moment, more information, right? Show me the spot spot, or what's the time, or the actions that will be treated to voice actions, which is, do this. Play this song, turn on my music, and set my alarm. And these are very, I think it's not so much the understanding of language, as much as it's the scope of the task that they do, right? And with Siri, it's nice that if it doesn't understand and doesn't know what you want to do, it just throws you back to a website. That's the default, the fallback. Is that it? I'll ask you a question. How does E.B. deal with typos, or algorithms? So in case you want to hear, the question was how does E.B. deal with typos, or colloquialism? So if you notice, Google has a pretty good spelling in everything, in everything, right? I mean, it's probabilistic in some ways. Colloquialisms are hard, because simply because the surface meaning of the word is kind of different. And that, there's really no way around to say about very specific things. Oh, no, no, no. I mean, I think other than E.B. to be able to handle typos, reasonable typos, right? I don't think you can do it in an isolated state, because whether something is a typo or not, it's very difficult to handle typos. It's also got language, it's an active subject. It's based on the content. So that's what we're doing. Okay. Alright, Chris. Did you meet your own NMP system, or did you go with something a little bit longer? Like with AI or something else? So we built our own on top of open source departments, right? So the two problems that are fairly well solved are speech tagging, tokenization, and there's no reason to reinvent the wheel on that. Those are actually machine learning models. You can train them on your own corpus if you want. They come to you trained model. But beyond that, we built our own stack, right? And there was, I mean, there's one of the things I didn't call out there is that one of the things we do is we don't know single lines of text or combination. We have to read an entire email that's addressed not only to us, but to multiple people. So there might be a conversation going, and at the end, it's literally a line that says, Evie, do this, right? Just set up the name. But in all that text, there was a conversation, there was context around when, where, and who that we need to extract a lot of information from. Given the level of complexity, it's like we have to build our own stack. I'm wondering how much it is. Right. Or whether you have to build it on your own and also when you have to build it on your own. So I think it depends very much on what you want to do with it. I mean, if you want to build a chatbot, there are a zillion different chatbot frameworks right now, I think. I'm not sure if WIT or AI has a full chatbot interface, but API.ai is just a Google equivalent. Certainly a lot of basic natural language processing is there for you. But, yeah, I mean, it depends. If you wanted to build something like Evie, I think that's something you have to build yourself. What for? That's a problem for us right now. I mean, it's a question of, we're building a, we're not trying to build something that only does that. So it's trying to build it in a general way as possible. So being able to detect, you just need to be back where they write back to you and say, no, you got it wrong. It's relatively straightforward, right? You use the same foundation of the engine that you have to understand what you're trying to do. But that's, you know, it's just a lot of work. Does anyone else have something that I miss? Can I ask you, do you see that there's a specific difference between chatbots and new slides, right? I wonder how do you address the fact that it's learning? How is your, how is Evie learning? Because chatbot, you know, there's a different way it learns, right? Is it different roughly around the same program? So I'll repeat the question. The question was, does Evie learn any differently from chatbots? Is that a fair paraphrase of the question? You know, with maybe one or two exceptions, I don't know how much actual learning happens in the way that you and I think about learning, right? Any machine learning experts here? No, so I can say what I can, I want to say. Yeah, I mean, I think some of the deep learning machine learning stuff really has been categorized more accurately and we'll strip away the marketing as applied statistics, right? A lot of times that's what it is. They learn certain things within the framework that you have set up for them. So if you phrase the problem in a particular way, if you frame the problem such as it's a machine learning problem, within those bounds it learns, right? More likely it just waits to get to match your data set. But I'm not aware of anything that really is able to integrate new concepts today to actually take an unknown situation and apply reasoning to it. That's, I think, beyond what the state of the art is today. So when you talk about learning, I think it needs to be, are you talking about getting better at picking out certain parts of the text versus getting, being able to handle unknown scenarios that it's not been programmed for and that last bit definitely doesn't exist anyway. So I've got a question from a marketing perspective. Yeah. So this is something that's relatively new, you know, hasn't quite existed before, at least in Singapore. Yeah. What was your approach to going to market and after that, because I know you guys been live last year, in the last year. Yeah. And then after that, capturing market share, what were your approaches? I think what's interesting, one of the things we learned is that this isn't a product for everyone. It's not a mass market product because most people, especially in their personal and social life, don't do enough scheduling for this to make sense. So it turns out there's a fairly specific group of people who it appeals to. One of the earliest groups of users who really got into this, into EB, was people who used to have assistants. So they used to work in a big company and they used like a consulting company that had assistants, or at least a shared assistant. So they came out, they did a startup or they had a consulting firm. And then when they hear about this, they kick the tires, they schedule two or three test meetings. You get it and then like, I love it, right? And then they're off to the races. And it turns out like having an assistant takes some getting used to, right? You need to be able to instruct, give the right instructions within reason, right? Because even human assistants don't read your mind. So I think to answer, come back to your question around going to market, I think a lot of it has been finding the right groups of users. People who schedule a lot of meetings, recruiters, startups, resource-contrained startups, BD guys, salespeople. The other thing is virality, right? As EV schedules meetings with other people, they say, hey, what's this? And then they get curious and they can check it out. So it's still early days. There's one question about, we call it, the way EV learns, right? Does it learn from a one-on-one basis? Like, say, for example, it's actually worth talking with one particular person versus you have an entire organization which is actually using EV. Does it learn what you call phonetics or whatever the context of all the different people together? Or it's always a one-on-one basis? Or can it apply this learning from this person to a different person? Can you repeat the question? Yeah, actually, why don't you repeat the question? Because I'd like to hear it again. So what I'm saying is, does it have the capability to learn from different people in one organization and then apply that learning in multiple ways? I think there are things that you, I'd say you acquire information as the process of scheduling. And if you say, in the process of scheduling one meeting, you say, I'm going to be away next week in Hong Kong, whatever. That's something that EV will pick up and say, okay, this person's going to be in Hong Kong next week. Right? And we'll use that for future scheduling or other meetings, right? Certainly, that piece is, yeah. I think there's a lot of interesting work around figuring out, if you're sent an email and you ask to schedule a meeting, are there multiple organizations involved? What's the power dynamic? Is one a client, one a vendor? I think these are things that work in progress. There's a lot of subtlety, a lot of things that kind of happen immediately, but sorry, that happen naturally for humans. That for machines, they don't move. Probably doesn't quite address the question, but also because I'm not quite sure what, could you give me an example of what group learning versus individual learning would mean? So obviously, I come from more of a recruitment. It's a recruitment organization, we have like 100 recruiters continuously scheduling meetings all the time. Everyone has their own personal system who's basically scheduling all these meetings for them. Do you want to kind of share everything? Hong Kong could mean a room, Hong Kong could mean actual Hong Kong. So there are different contexts and there are different ways that these guys speak or there's standard terminology of these people who use all these different people. So does it understand from one person and has the capability to apply that understanding to a different person when he is scheduling his meeting? Yeah, so I think it depends very much on the link. So the link between the people, right? So if it understands Hong Kong is a meeting room within an organization, then if people within that organization speak about Hong Kong, then that's definitely, it's a probabilistic kind of thing you have candidates, right? Is this a meeting room? Is this a place? And then you have to figure out what the use other contextual clues to figure that out. Is Evee smart enough right now? So if I use a model that you should can she figure out a meeting? If I just say we're competing with me and Joe and get us a meeting room and she figure out a meeting room as well right now or do I need to specify a meeting room? No, she can pick a meeting room. If you're on a team plan. Yeah, if I'm on a team plan. So she knows the meeting rooms and she can just assign one to me. Yeah, so sure. So I made that caveat around first getting the meeting room integration set up with the organization for 10 years will be a choosing a room for you. Can you book her cell? I think that's fine. Not yet. Are you the process of identifying some other use cases? Yeah, I think we have the use cases I think we're waiting for the tech to catch up. The team based in Singapore right now we're about 8 people but we're looking into a problem. So are we just creating the team? Is it mostly that I say it is? Is it engineers or is it... Engineers. Mainly engineers. So it turns out like a lot of this is software engineering as opposed to data science or machine learning. And it's probably also I guess the phase of growth. We're building something from nothing. A lot of this infrastructure and plumbing and software engineering and then probably as we evolve and grow that mix will shift towards okay, let's look at the data as we find the models.