 Good morning. My name is Boone Sheridan. I'm Boonerang on Twitter. I am probably much funnier on Twitter than in real life. So I spent some time with me there if you can. But let's spend a little time together today talking about something that kind of connects to that idea of failure, because I'm going to talk about augmented reality, Pokemon Go, and all of the things that maybe we thought we were going to get with augmented reality, this idea of I don't know if you pay attention to augmented reality or virtual reality, but my favorite is every other headline is virtual reality is just around the corner, right? Last couple of years was virtual reality is it's coming, it's coming. It's almost there, it's almost there. I'm going to do this at least once, because this is one of the biggest stages I've ever been on. And I'm just trying to figure out how far I can actually go on this stage. And I can say hello to you people on this side here instead of looking at all these lovely faces here. But in talking about augmented reality, Pokemon Go, and virtual reality, there's a lot of disclaimers involved. Can we do this? Come on, work with me. All right, AR is moving so fast right now and that's actually kind of the coolest thing. We're seeing so many things change with AR and the companies that are building it that giving this talk has become kind of an adventure, because I actually don't edit the slides until the night before I give the talk and I see what the biggest stories have been and the new tools have been. For those of you that pay attention to Facebook and F8 or to Google I.O., there were a ton of new announcements for the tools that AR is going to be supporting in their tools. So it's kind of crazy. Talking about AR is sort of challenging yourself to never give the same talk twice. So if you're looking for space to do a talk on, this is it. There's crazy stuff happening. My favorite part of the AR VR experience is it has a little bit of that UX versus UI flavor. I'm not going to ask for anyone to raise a hand, but anyone familiar with the UX versus UI argument on Twitter? I don't know about your experience, but it seems like when the moon is just about full, every other person on my feed will get involved in a UX versus UI argument on Twitter. And it's kind of fun because you get to see the same images over and over again, the pathways and all that sort of thing. But it highlights some universal truths that people are discussing. And AR, VR, has this same kind of thing. They have this amazing continuum where you have virtual reality where everything is around you created by the computer 360 degrees. The computer is in control. It's not a depressing thing to hear. And on the other side, you have reality. Your version of reality may be very different from someone else's, so be aware of that. And then you have this space in the middle between augmented reality, mixed reality, and augmented virtuality. For the record, I have never heard someone mention augmented virtuality in spoken word. It feels like something that's getting written up in papers. It feels like something that certain studios are pushing. But it's just one of those words that I'm not quite there. But I do start seeing mixed reality used more. That's because the idea that you're bringing in things from the virtual world into your space. So again, you're using some sort of a phone. You're using a device to help you pull in more information, your partnership with the computer. The device you're using helps you float across the top. So I'm going to talk about AR. And I may say MR and AR as I go back and forth. I'm on my third espresso, so words will get a little funny. My experience with AR started not as a designer, but as a human being who got involved in something a little bit crazy. When Pokemon Go launched. Anyone remember when Pokemon Go launched? Yeah. Something happened. I live in an old church in Massachusetts. And my church was actually listed as a gym. And I made hundreds of new friends. When Pokemon Go launched. I woke up Saturday morning. And I had 20 or 30 people standing outside my home taking pictures and battling in a gym and doing things that I had to stop for a minute and go, I think I should look this up on the internet. And sure enough, I opened the app and there it was. And so what instantly happened was this wave of stories, but it highlighted the fact that this amazing game launched. And suddenly, everyone was talking about augmented reality in a way that they never had before. Augmented reality, virtual reality, all these new techs have some sort of catalyst event. Google Glass was a big one for better or for worse. Pokemon Go was a big one. And now suddenly augmented reality meant more than an academic type of technology. It now meant a game. And it meant people running around. Unfortunately, it also meant that people were trespassing, people were doing things they weren't supposed to do, people were crashing their cars because they were too busy looking at their phones. It was kind of this really weird wave of enjoyment and destruction, which I'm kind of for, because talking about chaos and ball of death, like it was Pokemon Go, Pokemon Go, Pokemon Go, fiery ball of death. But since Pokemon Go launched, something really changed. The game got everyone's attention, but then more companies started getting involved and more things started getting built. And so it gives me a chance to use my favorite Pokemon characters to talk about augmented reality is actually evolving as a meaningful platform, something that you can build with, something that you can design with. But something subtle has happened about how augmented reality is talked about and how it is used. I want to talk about that, but I want to touch again on augmented reality and virtual reality has been with us forever. This might not be familiar to too many people. This is an old TV show from the US called Murder She Wrote. And this is in 1983. Virtual reality was supposed to be right there with us in 1983. That's hilarious to think about it. And again, it was always just around the corner. And even as 1995, anyone here that was playing a lot of video games like myself, it was, this is a cool headset. The games stink. And I can't believe I just spent four months of an allowance on this thing. But it was right there. But then it still never got to mass market. Then you've got the famous cover here of Lucky Palmer when he sold Oculus Rift to Facebook for about a billion dollars. And suddenly, again, it was supposed to be right there. We were all going to be doing this. It was something that was going to happen. It never quite showed up. Yesterday, pulled for photos, the new Oculus Go, virtual reality, and these headsets are not ubiquitous yet for so many reasons. Not the least of which is the fact that if you wear one of those, lots of people love to come up behind you and tap you on the shoulder and scare the crap out of you. I am permanently paranoid from putting on those, because every time I put one on, someone always crosses the room and goes, don't touch God. But as much as Oculus Go was supposed to be the future of VR, something really powerful happened at last year's Facebook F8 where Facebook said, we're going to still make virtual reality. But we're focusing on augmented reality, and we're focused on the idea of using the camera. Everyone around the world with a feature phone, with a camera, can now work with augmented reality. And that's kind of the key to thinking about AR today, is that it's not equipment. It's not necessarily new tools that you have to strap in and power up and pull on a wetsuit and jump around. It's the idea that hardware has reached this point now where almost everyone has a device in their pocket with the hardware, the software, and the controls to deliver AR tools and deliver AR experiences, which is kind of powerful when you think about it, because now it's democratizing AR in a way before it wasn't possible. So instead of a tool for someone who has the disposable income and the willingness to walk around in public wearing something really crazy on your head, now it's you can take out your phone and do something. Because when you think about it, all you really need for AR is a camera and a way to display what you're taking in with a layer of information on it. So now instead of people thinking about AR and VR as big tools, I don't know if you're familiar with Topshop having virtual fitting rooms where you can stand in front of a smart mirror. Anything that has a chip in it is smart now. So it's a smart mirror, but it's basically a big display and you can pick clothing to wear over it. And you can see what a suit looks like, what a dress looks like. A number of fashion companies are doing this making what they call virtual dressing rooms. And I've seen one at a Uniqlo store and asked the person if they liked what they were doing, they said yes, and I said, would you call this an augmented reality tool? And I'm not going to say they looked at me like it was an idiot, but they went, are you a cop? Why are you asking me? I said no, but do you think of this as augmented reality? I said no, I just think of this as trying on dresses. I think of this as seeing clothes that I want to buy. So the words augmented reality have started to fade, but the tools are showing up in more and more places. And it's letting us do a lot of cool things. It's letting us do some silly things as well. This is Samsung's augmented reality toothbrush. All those of you who had AR toothbrush on your scorecard, please check that off and pick up your prize on the way out. This is the weirdest one I've come across so far, so that as you brush your teeth, kids, and it times it, right? Because you're supposed to teach children to brush for a certain amount of time. So I guess someone thought, if you put a camera there and you mount it on the sink, which is the craziest thing in the world anyways, what's the last place you want to put your phone up precariously? Bathroom. But they decided kids will want to brush their teeth if they can have goofy lenses and they can pick up points. I want to say something really mean about Samsung, but I can't bring myself to do it. So where are we now? We've gone from this hardware situation that's kind of crazy to democratizing this information and tools and hardware platforms that we can use. What does AR mean, both for designers, for technologists, for those of us that are just interested in something that we might be able to build for ourselves? We're at an age of usefulness in a way that I think we've never quite had before, and it's becoming ubiquitous. This is an app for an airline that lets you walk through the airport and see the wait times for security, where your gate should be, that idea of if you've ever visited an airport and tried to guess which security line will take the least amount of time, right? Everyone has that decision-making process. But think about that. Real-time information, lay it in over the world, just pulled out from your phone. It's also great for teaching. This is a sketch, AR Sketch app that teaches you how to sketch, if you can hold your phone over the paper and begin your sketch and start to see how it comes out. It's incredibly cool. It's not something that you would think is, this is an augmented reality. Oh, this is going to help teach me how to draw. This is going to help teach me how to put together sketches in a way I never have before. So we're reaching usefulness. We're reaching the stage where it's not just games. We have Google Maps. They just announced at I.O. that Google Maps is going to add an AR overlay. So if you've ever tried to use Google Directions when you're walking around somewhere and you have that moment, every time I open Google Maps, I have to do the same thing. I have to walk for a second to see where the dot goes. Somehow Google thinks everyone has a compass, and it's like, oh, I'm clearly facing north. So when you're in a city and you see someone take out their phone and they do, one, two, three, OK, this way. That will save you time. But it's kind of cool now because it's overlaid with the Maps. It's not necessarily thought of as this is an augmented reality navigation tool. Maps is now more useful than ever. And lastly, to sort of show it all coming together, this is a video of a scenario that I really, really love. I think we have sound, so let's give it a go. One thing we're really excited about is the ability that Tango gives us to be able to do indoor, in-store navigation. And it will just drop digital breadcrumbs that will take you exactly to the place that you want to go. You can use your Tango phone, and it will show you more information about the actual product, and those kind of things that really help you to be able to make the decision of what you want to buy when you want it. It's incredibly cool. Now my dream is for Ikea to launch that. I'm not scared of going into Lowe's, but I walk up to the front of an Ikea and I get cold sweats. But think about that. The idea of now you're just, I know what I want to buy, I know instead of asking or trying to find someone to help you, just pull up your phone. It'll walk you to the thing that you need and you walk out with it. So again, the usefulness is helpful. It's not so much games anymore. Pokemon Go still makes a few headlines, but now we're seeing real useful stuff that people can actually get value out of and not even think about it as AR. So as designers and developers, the AR ecosystem is stabilizing. The fun part about that video I showed you is it uses something called Tango, which was a development kit from Google. And about a week after that video launched, Google discontinued the Tango development kit. And they launched another one. That's how fast it's moving. Google built something amazing and said, nope, not good enough, toss it. Each launch of iOS has a new AR core in it. And depending on what type of phone you have, if you have a 10 or an X or an 8, you can see more and more things happening with the AR core being built. So the tools are there. You don't necessarily have to worry about a fractured ecosystem as much. It's sort of every six months, more phones come out. It doesn't mean old phones leave the ecosystem. That's the important thing to recognize, is every time someone buys a new phone, it doesn't guarantee that an old phone disappears. So there are some challenges of technology there. But it's expanding in a way that now you can think about everyone in your customer group, everyone you build for, could use it. It's on the to-do list for so many companies. As part of my research, I spoke to a couple of companies like Forrester and Gartner, read more reports than I probably ever should have. And AR was on the to-do list for lots of companies, which is interesting because they hadn't picked what to do yet, but they knew AR was on their list, which is kind of amazing. It has that smacks of that VP of, get me an AR app tomorrow. What do you want it to do? I don't care. Just make it happen. Someone goes, crap. OK, better go to the internet. That increase in interest from executives is kind of the best and the worst part of it. There are CEOs and CTOs that know how to ask for augmented reality. So depending on your role, no offense to any CTOs or CEOs in here, it means the rest of us have to try to play catch up and figure out what it means. It's like, you said AR, but let me ask you something else. What if we built something that did this, sort of change in the conversation instead of answering the ask, that terrible idea of a CEO gives you a very specific thing, and there's that phrase I hear a lot in the US. What's the ask? Which is a terrible use of English, no matter what your native language is. It's a terrible phrase. So all these things are happening, but the shift from that CEO ask to people building them is great AR is customer focused. It's coming from a space where you have a customer need and AR can solve it. So because this is UX and because we are thinking about customers, I want to talk a little bit about what it means and how you could build one yourself. Because I think every team and every designer has a chance to build an AR app because there are customer points along the way that you can influence. The challenges with AR, I find that people really struggle to start it. It feels like a tough thing to get started. So I want to give you some strategies before you think about an AR app or before that request comes in. And this is key. You don't need AR tools to prototype AR. You can do a lot in paper prototyping. You can do a lot in Google Web Designer. If you've got something like Bounce.js, you can build something on a screen that will show you some interactions and give you a sense of what you're doing. And I recommend that instead of stressing out and going and getting a team that does AR, start by prototyping the idea. Stay with paper and pencil. You might have seen sometimes you make a frame in the shape of a phone and you have a scroll of paper that you're moving up and down. That sense of prototyping is so easy and so quick and it saves you that pain and trouble later because you're going to need to do a lot of prototyping for AR. Getting to a nice, clean AR app requires a lot more time on the front and the back end. As you're building something, this sounds funny, but I have to keep saying it. An AR app has to be built for people that are already mobile friendly. Already people that have a device that they're willing to spend time in because AR, by its very nature, starts to get really interesting with layers of interfaces, interaction with objects. In a great AR app, you can poke, prod, twist, and turn lots of things. And if someone isn't already comfortable with using a phone, it's a nightmare. I've done testing for an AR app with some older folks who didn't necessarily love their phones, but they got to the AR app. We're testing out something where you can take furniture and put it in your home. And as soon as they started playing with it, I could just hear them go, mm, mm. That low level on the back of the throat, when you're unhappy, mm, mm, mm, mm. And just they walked away from it. And we did it with some younger people. And we did it with some kids. And they were like, oh, cool, poke, turn, twist, poke. And they filled a room with IKEA furniture. So depending on the device and the viewport, you can mitigate that, but build and think about people that are already going to love their mobile device and use it. This is a huge one. If you're going to build something for your company and you're going to do an AR app, think about how you'll measure success. Because AR is a beautiful, fun, shiny toy right now. But if it doesn't connect to some success, you'll see support for it really vanish. This is sort of a business culture aspect of it. When you're doing something in augmented reality, are you connecting it to something that your company or you can measure and sort of get business value and connect it to the people that want to do the projects and do more? You see so much of AR connected to buying something. You see it connected as a way to help manage a purchase process or do, as I'll say it later, post-purchase, like help with customer support. Talk about that early rather than building something and then figuring out, oh, and by the way, it has value this way. When you start with those conversations, I think you come up with better ideas to start with. And you build better bridges to the people who you're going to need help with later on building it. That is weird. During the process, if you go through those first steps, you start to come up with some ideas. Maybe you have some paper prototypes. Maybe you actually have built some stuff in code and you're starting to look at it through a phone. Narrow your scenarios and embrace your constraints. The fun part about augmented reality is you've got a camera and a viewport that you can do anything with. And that is also the most frightening thing in the world. That open sense of we could do anything we want. Just imagine this is a phone. This is a really bad image. But imagine this is a phone. Imagine I'm 6'3". But having that sense of we're building an AR app to do these one or two things and do them well, and embracing those constraints. Because when you get further along in the process, you're going to find out that the interactions are crucial. And you'll spend more time than you think figuring out the actual interactions on a screen. Because AR, in a lot of ways, it's interfaces within interfaces. This is the simplest one to show because it's Pokemon Go. Most folks have seen this in person or somewhere else. But you've got another layer. So you're seeing the world through a phone to begin with. And now on top of that viewport, you're adding more interactions and more interfaces. So the amount of space you can actually view something with is getting smaller and smaller. And so if you're not careful, you wind up filling that space and you have to give people that sense of what will this do, what actions will take place when I tap something, when I twist something, when I turn it. AR gives you the ability to do physics with objects in a way that you can't in real life. And that's kind of awesome. In a lot of AR scenarios, you can pick something up, you can throw it, you can move it, you can turn it. And so now you have to think about tap zones. You have to think about motion. The interactions become critical. This is an app from Amazon, an early version. Amazon is doing something like other companies. You can take something from their catalog and see it in your home. I want a new toaster. How will it look? Will I put it here? Will I put it here? Maybe a new kettle. Where will I put this? But as you can see, look at the slab that they just use for a menu or for a display space. They took a whole chunk of the interface and just filled it. And so you've shrunk your camera port even more. You've shrunk the space to work with even more. To me, this is almost like, this is a mega menu for an AR app. This is the equivalent of a mega menu where you're taking up all this space. And that's going to change. As this evolves with Amazon, I'm sure it's going to get smaller and smaller, and the cues will get a little smaller. You'll see interaction, but that amount of space is huge to take up in an interface. And the thing that I've heard over and over and experienced myself was, you have to plan for so much testing afterwards. When you've got an AR tool built, it will take you so long to figure out how it will work in the real world. If you go to Medium, there's an article by the good people at Wayfair who built an AR app, and it talked about how testing just almost demolished their budget. And when I've talked to other companies about the apps that they built, the horror stories were we should have budgeted half the time to build it and half the time to test it. Because you're working with new interactions, you're working in new spaces, and this is where usability testing comes way to the front. And testing, it's instead of that contextual interview where you're talking, it's that contextual interview where you hand someone a phone and just watch what happens and see how they react with it. And suddenly you see the differences in what I call arm style, how you move something, and how you move that gives that impact on whether or not the interaction comes through. Do people understand the interactions that you're proposing with them? I've been playing Pokemon Go for years, and I still discover things that I didn't realize because I never took something and twirled it four times. Or I never took something and tried to expand it all the way. So discoverability and interactions become a huge part of testing. And the worst part is, if you don't plan for it, you've got executives and people going, that's my app, and you're like, yeah. So funny story, everyone who saw the flower ignored it. And the flowers, the thing they touch, and then it brings up the thing and they buy, right? Yeah, no one sees that. So we're going back to the drawing board. And again, you used to call it the CEO tap, like when you're talking to an executive, and not all of you in the back can see, but my foot is doing that slow tap that's starting to pick up speed. And you know you're okay if it's slow, but as soon as it speeds up, you have that, oh crap. Okay, so we're going to deal with this tomorrow and I'll talk to you then. After building, after launching, this is actually my favorite part and I think the most important to talk about is considering the personal impact. Augmented reality does something very powerful. Everyone get it? I always love to see what makes people bring the cameras up and when the cameras go down. Augmented reality, because it's often camera-based, you can inadvertently pull people into an experience. You might be walking around with your phone, catching Pokemon, and then you realize that you almost just walked into someone who's just minding their own business. Or you might be doing something with your camera and you're taking a photo, but if I take a photo of this fantastic gentleman, love the beard, if I take this photo, I'm catching the people behind. What am I doing with the data that I collected when I took that photo? Am I letting those people, are those people now part of the data I've collected? Have I stored it? Am I doing facial recognition? All those types of things. Augmented reality means you're hoovering up lots of data. You're generally using GPS coordinates, you're using cameras, and how you use that personal information is critical because once things get into an ecosystem of technology, it's hard to get them out. It's a favorite sign of mine in Hawaii, right? People were getting turned down a private road and an interview with one of the people said, you know, they've tried everything. They sent emails to the executives at Google and nothing, they could not get that changed. So as you're building apps and as we're reaching a point with virtual reality and augmented reality where the data just gets slurped up, and I feel like we're at the stage now where data's starting to live forever. You know, even just the photos we're taking now and we're tagging and putting on Twitter, those things may never go away. So I hope it looked good, right? But that idea of what you build for augmented reality, think about the data that comes in and ask questions such as should we store this? If we store it, do we tag it? If we tag it, are we doing it appropriately and connecting it? And more importantly, what could be done to hurt someone with this? You know, if we're collecting all this facial recognition information and then we're tacking it's on, I like to ask there's a great book called Technically Wrong and in it, the author asks, you know, when you create a product, ask how it could be used to hurt someone. An augmented reality by using a camera, by using location, you're collecting information that could really be used against someone. That was the down moment, I'm sorry. When I talked about location, right? So that was with cameras, but in locations, where are you taking people? If you're using augmented reality that uses locations, you have to think back to all these Pokemon Go scenarios, where people will cross physical barriers to do things in virtual locations. It's sort of the crux of my question when I had Pokemon Go launch at my house. People were coming to my house. I was happy to meet them. They were all really nice people, but it was really weird to have 200 strangers hanging out in front of my house. And lastly, this is the one I think about all the time. As the virtual fitting rooms become more common, think about what that will mean, that there's an always-on camera that's taking pictures of you and showing you all these different things. And at some point in time, what if you're changing and you're naked for a moment, right? Does the camera turn off? I'm sure there are some people that will even get a little bit of a thrill out of it. Like, no, I'm just gonna be naked and I wanna see clothes on me. And it's sort of like, what's gonna happen to that data? It's gonna happen. Someone I know who's a much smarter person about computer security said every company is a data breach waiting to happen. And the truth is, augmented reality captures the sort of information that could be really terrible if it gets loose. So again, I'm being slightly paranoid, but when you build with AR, you need to think about that. But on the plus side, when you're building in AR, you can do some things that are fantastic and you can solve real customer needs. And what I've found is building AR that goes to something that your customers have already talked about or a cycle in their needs. For those of you that have done any of the workshops around experience mapping, road mapping, journeys, I'll do a journey session tomorrow, like think about the life cycle and the journey your customers are traveling on and find a place where there's grind and find a place where AR can really help them. So rather than starting from scratch, think about how to expand options for consideration. If they're trying to buy something from you, if they want to get something from your company, AR is the perfect way to do that. You think about IKEA place, right? Again, I get the cold sweats before I go into an IKEA. So the idea that you can take an app, you map your room, and then you can pick furniture from the IKEA catalog and put it in your house and see how it looks. And that's fantastic because you also think about the time it takes to get to an IKEA. You know, IKEA's in the US are generally sometimes just outside of a main city. They like to have lots of space and so getting there can be a challenge. So think of all the time you can save by actually having the product, dropping it into a room and seeing if it fits and knowing when you get to IKEA that this is the thing that I want. You'll probably still get a bag of tea lights and a hot dog because it's there, but this is the same as what Amazon was doing, showing you how to have a product there. Warby Parker. Warby Parker used the iPhone X and the face mapping to do something amazing. You can hold it up, it will map your face and then pick glasses that they think will look best on your frame. So it'll give you the shape of your face and then it will give you the glasses that they think look best. They'll still send the glasses to you, Warby Parker sends you up to three pairs to try on, but you can instantly go to, these are the ones that we think are good. Sephora, right? They're showing you doing virtual makeup artists, how it would look. If you can't get to a Sephora, if you don't have the time or you just want to play around with a look, you can mess around and see what it looks like. I still look pasty as all heck, I don't have no idea how much foundation I should be using. Another one I love is preventing post-purchase problems. I use alliteration and sometimes it bites me in the end. What I love about post-purchase problem solutions is how you can do things with things like customer manuals. This is from Hyundai, or Hyundai, and the idea is that you have a manual on your phone and you can move it around the car and it will show you the different parts of the car and how it operates. So my experience with getting a new car is you get a book about this thick and anytime something goes wrong you go and you take it out of the glove box and then you start flipping through and it takes four or five minutes to find 13-2-4-3 if this light is blinking, your car will blow up, you should get out of the car. But here, you can sort of motion over and it seems like that's no big deal at the dashboard level but think about at the engine level. Think about looking at your car engine and having this overlay of I need to change my oil. Where's the oil? Where's the filter? Where does a brake fluid go? You know, cars are beautifully complex machines and this helps simplify it. So if you have a product that needs a little bit of support, this type of data becomes the perfect overlay. So those are two flavors of things that can happen and supporting the customers and doing something that I think is really meaningful as opposed to catching Pokemon Go, which is still fun, but it doesn't help. Thank God that wasn't me. Microsoft has this great concept video you can find where they're using augmented reality and it's getting that whole sort of movie screen effect where you have multiple things that you can use the HoloLens for and it's a future that I think is reasonable. I don't know that I'd have so many screens up but the idea that these World World overlays are coming but I think the truth is the reality is a little more like this where you're setting up a table and seeing if everyone likes it and other people in the room are going, what are you doing with that? So I think we're a lot closer to this future than this future and I'm totally okay with this. Thank you for your time.