 All right. Hello, everyone. Hi. I'm Aditi, the head of design at Referralcandy. Not sure how many people have heard of Referralcandy. We're a small startup based in Singapore. So today I'm going to talk about designing for automated conversations. PS, it's not AI, so more on that later. So we have two products. We are working currently on two products, Referralcandy and Candybar. And we have actually built out automated conversations or bots for both our products. So I'm going to talk about a bit about, I guess, stories, things that we've learnt. But before I start talking about the actual bots and the actual conversations, let me explain a bit about what we do. So Referralcandy is an automated referral program for small and medium businesses. So online stores. And how it works is you install Referralcandy on an online store and then it automates your referral program. And it works with any online store. We have a wide range of merchants using us. So they could be selling clothes, shoes, food, tea, coffee, really anything at all. And it works. So it's already automated, but we thought, you know, and this product has been sort of live for five years. We are doing quite well. And we thought, can something like a messenger bot really make automation better? So it was more like an experiment when we tried it out. So these are a few images of our Referral Assistant. So you can have a look there. Pretty straightforward. I mean, our hope when we built it was, you know, we hope this will be a better experience for people rather than the sort of regular automation. We hope that, you know, it's more personal. It's a more personal experience. It's easier for people to, you know, get their referral code, share it with their friends, stuff like that. We have another product too. It's called Candy Bar. And this is for offline merchants. So Referral Candy is for online merchants and stores, and Candy Bar is for offline brick and mortar merchants. And this is actually how we got into trying out these experiments and trying out these automated conversations because Referral Candy is, you know, five years doing really well and Candy Bar is super new. We recently launched it. And so it was easier for us as a team to do something this new with Candy Bar. It's used by many, many kinds of small and medium businesses like donut shops, cafes, hair salons all over. So we have U.S. merchants. We have merchants in Singapore. We have merchants in the Philippines using us. This is Joel. He's from Mission Juice, and he's in Singapore, and he uses Candy Bar. I think it's important at this point to talk about how much our team actually cares about people like Joel. We really want to make a better experience for people like Joel. And that's why we started out trying and experimenting with automated conversations. This is an image of the Candy Bar assistant. So this is the first assistant that we built out. Pretty straightforward Candy Bar is an automated rewards program for brick-and-mortar merchants. And what you get is you get notified about, you know, when's your next reward? How many points do you have? How many stamps do you have? And stuff like that. So straightforward notifications. But we also built out a feedback feature where we asked people, how was your visit? So after you visit a cafe, we would ask them, you know, was your visit good? Was it bad? And we tried to collect feedback. So this was interesting, and this was sort of our first step into this whole field. So why automated conversations? I think it's good to question, you know, I think Prakriti also mentioned that in her talk, that you shouldn't just do this stuff for the sake of it. You know, why are you actually doing this stuff? So of course, you know, we want to improve the customer experience. Our products are both merchant-facing and customer-facing. So the merchants see the dashboards, and the customer uses the rewards program or uses the referral program, right? So yeah, it all started with an experiment, and it started with the SMS. So when we first started building out Candy Bar, we had a pretty straightforward SMS. Here's a very real example of our SMS. And a lot of my examples are all real. So very straightforward, as you can see, it's an SMS that is sent to a customer when they join the rewards program, saying, hey, here's your stamp, here's your reward. Please tap on this link. So initially, I was quite obsessed with trying to fix this SMS. I made it shorter. I tried to make the links neater. I tried to rewrite it many, many times. We did a lot of testing with the SMS too. We found out that people really like the unsub link, so we kept it in. So we did a lot of testing, but there are a lot of limitations with SMS. And it was an MVP product, so we let it go initially. I started looking at examples from my everyday life. So these are all real examples from my everyday life in Singapore. Body shops sending me these horrible messages. I'm actually in the loyalty program, and I do use it. But as you can see from this SMS, it's just like characters, and it's not really readable. It's a mess. And then, of course, I get spammed from some random number. And the Redmart delivery notification is not bad. You know, pretty straightforward. Here's your delivery. Apple's SMS also not so bad. That was actually about the time I got my Apple Watch, so I was really excited. And it said, Apple Store, today is the day. So I was like, oh, that's really nice writing, and maybe we should, you know, get a writer to help us improve our SMS. At the same time, I think when you look at the screen, you realize that all of these SMSes sort of look the same from a distance. And there's a limit to how much you can improve your customer's experience with just SMS. So we looked at alternatives to SMS. I started looking at, in my own life, what are the alternatives to SMS that I'm exposed to? So one really nice example from Ninja Van. Again, a delivery notification, but so much better than an SMS. And I was like, thank you. There are other teams experimenting with this and really trying to break out of the SMS situation. Netflix also texts me, which is a bit weird, but once a month, it's not a very useful notification, but it's only once a month, so I don't mind, to be very honest. So yeah, so Netflix also texts me. And I guess what I realized was when you are looking at alternatives to SMS, you're pretty limited, this is the range. The formats are limited, but let's work with what we have, let's stick to the goal, let's try to build a better conversation with our customers and make it more engaging. So that was a pretty sort of starting point for our experiment with Candy Bar. So yeah, why did we first build Candy Bar Assistant before we built Referral Candy Assistant? It's because Candy Bar was a newer product. It was easier for the team to experiment with Candy Bar, Referral Candy, five-year-old product. Most of our revenue comes from there and so we wanted to be more careful with Referral Candy. So after building these two assistants, I think the team and me personally, I think we learned a lot of things and so I'm going to share a few things that we learned. Hopefully some of this is going to be helpful for all of you. It'd be interesting to hear. Number one, take away that I had from working on Candy Bar Assistant especially was don't make it look like a human. Now this sounds a bit counter-intuitive, but hear me out. I think what's happened is we are surrounded by automation already and this is, I think, what I'm trying to say. Samantha is not a real person. This photo is from unsplash.com and is a free commercial image. This is not real. Don't make your bot be Samantha. Be truthful and honest about what you have built. It's an automation, it's a machine. Call it that and so call it Candy Bar Assistant. Use an image that is truthful, please. I think as designers, we should be a little bit more responsible here and we should make sure that we are seeing the right things. The reason for this is that trust is very fragile in an automated conversation and so it's very easy to lose it. It's very easy to figure out that this thing is a bot, so don't even try to act like you're human. Forget about it. This is an example, again, this is a real example from Referral Candy. We have a 15-person customer support team. They are all amazing and they talk to our merchants using Intercom. This is an Intercom conversation between a real human merchant and a real human customer support person. Now, this merchant has become very frustrated because a lot of Shopify merchants, this person is very successful, use a lot of SaaS services. A lot of SaaS services use a lot of bots. This person says, I feel like I've been in a chatbot all last week and this week trying to fix one issue, so you can sort of hear the frustration already in this chat conversation. Unfortunately, Tippi, who is by the way an amazing, amazing guy, has to say, I'm a real person, so no need to worry. This is already happening, folks. People are confused about whether the person they are talking to is a bot or a real person. You know, half the time, you're not really paying attention when you're chatting with some service person. And so it's really, and this sort of underlines this point that I'm making, which is please don't make it act like a human. And so, the conversation ends with Tippi sending a link to our Referral Candy about page where his photo is there and it's very clear that he's part of our team. And so really think about the impact that you're having on the ecosystem. You know, we don't do this, but a lot of other services do. And so as designers, try to pay attention to what's happening. This is a mistake that we made which I like to show people. So we send this automated message. It's not a bot, but Intercom allows you to send automated messages to people. And the photo here, which I've hidden with a smiley face, is somebody from our team and it's a real person. So you can make it seem like a message is sent by a real person, but this is not real, it's automated. So what happens is when this person does a specific task, like an event is triggered, we send this message. So it seems really personal because only this person is getting it at that particular time when you did that task. So this merchant thought it was real and thought it was a real time chat. And so I said, so in the automated message, it said, welcome to your free trial of Candy Bar. Have you tried your first stamp yet? And the person says, hey, sorry, that was rude of me. No, I haven't. So I was shocked. I was like, oh my God, people are so nice. And also I felt so bad because this person is feeling guilty that they didn't do it. So from a quantitative point of view, this is an excellent message. This message does really well. And right after this, we can see on full story that this person actually goes and does their first step of onboarding, which is trying your first stamp. So it's a success. But in the long term, it's not a success, right? We need to make sure that it's clear that this was not a real conversation. This was not a customer support person because we have real customer support people too. So we have to differentiate that this is not real. It's an automation. It's just a reminder and a nod saying, hey, try your first onboarding task and the real customer support is different, right? So yeah, trust is fragile in an automated conversation. Be honest, transparent. That is just a bot. And it's limited. This is not necessarily a bad thing. It's OK. It's just accept that and design accordingly. The other big takeaway for me was design basics are the same, but tools are different. So your design basics are basically the same. You know, iterate a product design, have clear goals, test, design, iterate. In the center of your circle, you have your user data, your content quality, user data coming in and you're sort of testing a lot and all of that. So your basics are pretty much the same. OK. This is a picture of one of the first rounds of testing we did for KaniBar Assistant with real people. It was very important because we were building out the feedback feature. When you ask for feedback, it's a very sort of delicate point. The tone of voice has to be correct. The timing has to be correct. And so we did a lot of testing for that. So here you can see how our feedback feature actually looks. But the tools are different. So initially, I sort of went the same way. I tried to do wireframes. So these are my wireframes initially when I was trying to design the candy bot. I designed them in Sketch and I uploaded them to Zeppelin. But it was a big fail and it doesn't really work to talk to your team about an automated conversation through wireframes. Obviously it doesn't work. And then I downloaded the Sketch files from Facebook. Facebook gives you lots of little UI elements. They look really nice. So I used them and when you're creating Facebook Messenger bot, these are some of the UI elements. So you have quick replies like these three bubbles and stuff like that. So I started making some really nice stuff. But again, this doesn't really work to talk about an automated conversation with your team. They can't really give you feedback on this. It's just like what? So yeah, I actually fell back to this conversation tree especially when I was pairing with senior engineers and they wanted to know the logic of it and so we just drew it out like this and we were iterating very, very fast. So every two days we were changing it as we tried to work towards the MVP. So when you're iterating really fast, you can't make really beautiful looking mockups. You have to sort of just stick to the flow, understand what is the MVP that you are really building. How do you MVP an automated conversation is something I'm trying to share here. The other thing that you have to do is write your MVP script. So that's, I mean, at least that's my recommendation. So think of the script like a movie script. So this is an example from Indiana Jones, I think, yeah. And think of it as a movie script with characters, actions, scenes, stage changes and stuff like that. And this is an example of the candy bar assistant script that I wrote for one of the flows. So what it helps you do is it helps you go through each and every flow, each and every persona and to check for whether this actually makes sense. So you can't only look at it like this, especially from a design point of view, it's very easy to end up writing something that sounds weird. So use a script at least to get to the MVP phase where you need to define what is your MVP script. Once you have your conversation tree and your MVP script then you can sort of start visualizing your little story elements. So for example that's like one of our elements for how the bot says hello. We have four versions of what we say when somebody gives feedback. So if somebody gives bad feedback, the bot mimics empathy and it says oh I'm so sorry about that. Do you want to tell me more about it? So we don't force the person to give us more feedback. But we give them the option. Do you want to add more? So that's how you sort of end up having your story elements and it sort of helps at this point as you build on top of your MVP. The other thing I recommend once you have your MVP script, your conversation tree and all of this in place is prototype it obviously. So we actually build the prototype afterwards because it went live pretty fast. So after it went live I ended up making a prototype. I've tried a few and I like bot society but feel free to explore. This was pretty easy to use and simple. You can share a clickable prototype with the customer support team with the product team and business so that they understand what exactly it is that you're doing. So yeah design basics are the same but the tools are different. And write your MVP script and of course test a lot. Testing is just basics but it helps to say it again. The next big takeaway is become a really good writer or get a really good writer. So I think becoming a really good writer is very difficult so if you can't do that then get a really good writer. And I can't underline this enough because a lot of people think of these automated conversations at a very functional level but the thing is even at the functional level the tone of voice is crucial for success. So it's not about what you say it so make sure you really put in those hours and that time in crafting the right words and something about cultural details. So we have some US merchants using Canibar assistant and I thought oh we had a bit of a brainstorming session in the office and we were like how should we say hello so everybody was like oh cute Jimmy Fallon Jeff would be nice and then we had people replying saying I hate Jimmy Kimmel it's not Jimmy Kimmel it's Jimmy Fallon and you would be surprised how many people hate Jimmy Fallon it's I was surprised I was like oh my god okay yeah so yeah cultural details is easily missable and connected to you know getting a good writer the words you use the details are important in your MVP especially in a script right so please pay attention to this after we got several complaints we actually replaced it with a more generic hi so and by the way no complaints after this it has just been all good so yeah so just be careful about the cultural details here the nuances are important when it comes to text only and chat you know people react in a surprising manners and then the next big take away is it's okay to say sorry it's okay to say I don't know your bot isn't smart it's eventually going to lead to a situation where it doesn't know anything so this is what we do in candy bar assistant but feel free to get creative so the person is actually asking a very relevant question how many points do I have and that's literally what candy bar does it tells you how many points you have how many stamps you have how far you are from the next reward that's all but it doesn't understand right and so you have to apologize at this point and that's okay don't try to write a vague thing that maybe I understood no just be very clear it's a failure point and people figure out pretty quickly so we did a lot of testing and if things don't work people find the right action pretty quickly so you can see this person has immediately tapped on view rewards to find out how many points they have so people can navigate around this sort of UI pretty easily last point and not the least it's not AI so this is the bit from my talk title and it doesn't have to be AI I think a lot of people feel like if you're doing an automated conversation it has to be smart it has to somehow understand everything that the person is saying but it doesn't have to I think as long as your basics are good as long as you're building this automation for good reasons you can still create a really good experience for your customers you can still make something that's enjoyable that's nice and still works and you can still hit your metrics you still do all of that with something that is a dumb bot I think it sort of reminds me of this sort of command line interface from when I was a kid that you used to load up games it's just a basic Q and A format you type in a query and the system does a thing if you type in the wrong query it's an absolute horrible failure and so that is basically what a lot of this automation is you're just making it nicer more human more friendly with emojis and gifs and stuff like that but it is limited and that's okay okay so when I started on this automated conversation design I was pretty much in this phase where I literally asked the internet how do you design an automated conversation then I got a bunch of links and I read medium articles so I hope that after this talk you have a little bit more knowledge at least from what we faced from designing these two very different bots so quick summary number one don't make it look like a human be very clear who is a human and who is not a human so that both can work together in a happy collaboration trust is fragile in an automated conversation so be as honest and truthful as possible of what limitations your bot has your design basics are the same so make sure your goals are very clear make sure a stakeholder alignment is there make sure your qual and quant metrics are balanced make sure you are testing a lot and all of that sort of basic stuff but your tools are different when you are designing an automated conversation write your MVP script and test it as much as possible and try as hard as possible to get a very very very good writer it's okay to say sorry and I don't know and last but not least it's not AI it doesn't have to be you can make a really great experience using something like this as a stand-alone product as an addition to your already existing product it just depends what you want to use it for that's it from me thank you so much thank you so much Ajithi can we get our Slido up we are just going to bring up our Slido can we take questions on the floor we are just struggling a bit with our Slido questions? no one has tried to do that but they do ask a lot of questions some people ask about the weather some people ask for the menu which is more relevant not yet but it's definitely in the plans for now we are very focused on getting it to work really well one of the challenges we are facing is the time out because the conversation should time out that's what people expect but the system remembers and so then it leads to weird there is this loss of stuff to solve before we get to the jokes alright so we have one question which is how does your team measure success with this feature and did you meet your goals? good question so we have metrics product metrics for each thing that we built so initially we measured success with the initial test experiment to see whether it performed better than an SMS pretty simple first step and then when it did perform better than an SMS then we could come up with a new goal can it do other things so initial experiment and then you have your MVP then you have a different goal for your MVP and then you keep building on top of that obviously keep your metrics user centric so for example in referral candy and user metric is referral revenue the more revenue our merchants make it's a win for us it's a win for our merchants and it's a win for the customers too because they are getting rewards so come up with the right metric which is aligned across and then it's easy how long did your testing phase for candy bar really take any tips on how to convince clients that want implementations ASAP clients ok so testing never ends whose question is this ok hi so testing never really we didn't have a testing phase we just constantly testing so how we do testing is we are a really small team we are 4 designers right and about 10 engineers so how testing usually works is I want to test something tomorrow so I send out the word that come into the office and we'll give you $50 Amazon gift voucher and people come it's fine I would definitely come for that yeah yeah participate in usability testing we are always doing it so something that I personally believe and thankfully I've been able to do it with the support of the founders is constant research constant testing so if you have a phase then the whole conversation comes up should we have a testing phase don't even let it get to that point just be testing constantly you just need 5 people anyway for a basic testing so keep doing it constantly so whenever you do need to update there's already so much that you already know sorry does that answer your question ok do you use eb test to test your automatic messages in production environment not yet but it would be nice if we could do that there are a lot of limitations with facebook and facebook messenger and the way it works so it's just more complex but definitely it would be a good thing to try out how do you avoid users frustration how do you avoid users frustration I've been talking about talking to a bot but at the same time make your bot limited to take care of your customers trust wow that's a tough question I think if people are frustrated with your bot and they say I've been talking to a bot that is not a good sign and you should probably switch off your bot for some time and think about it are you sure that you build the right thing because we've had our bot up for a very long time and we haven't had that yet also I think when it comes to the limitations at some point a human has to step in if somebody is having a really bad experience and so luckily we've had an amazing customer support team they do a lot of chat support so they are really familiar with how to say things the right way how to say a welcome message annoying and fake so really learn I think from your customer support team and absorb that and try to put that into your bot and then I would just ask the customer support team if you are having this sort of issue so we have one question from Menaka who is saying did you build a tone of voice system like a design system to keep your messages consistent I'm not sure what a tone of voice system is but definitely we whose question is this it's very difficult actually what I meant is like you have a design system to keep design elements consistent do you have any system like a brand tone of voice that you follow to keep your chat bot messages consistent it would be nice but no I don't think we do I think we should have one we did what I did what I did initially when we first started the experiment was I did a branding workshop with the marketing team there are some really good writers on that team and very good content people so we got them to input you know how should the bot say hi how should it of course that led to the Jimmy Fallon GIF bot I think in terms of branding sure we did that there was a lot of clarity also in terms of what we don't want to do we wanted to be very sort of designed for the metric that we were working towards so all of the decoration was later I'm not sure if I answered your question but we don't have a formal system but there is in a small team it's easier to have a shared understanding of what is okay and what is agreed in terms of our brand there's a lot of sweet because it's candy there's some words and things that we can use more often than others but it can also get annoying so you just have to be very careful I think these are all the questions I did notice during your talk a lot of dog lovers went like so cute are there any cat lovers in there who have any questions why no cats no alright awesome thank you so much Aditi thank you so much for coming in today