 Alright, cool. Well, uh, we're here to talk about WebVR, what, why, and how, so I'm gonna start, uh, talking, oh, yeah, introducing myself, obviously. My name's uh, Wes Rufalcaba. I'm a senior front-end developer at Lollabot. Uh, I have a BFA in illustration, just so you kind of have an idea of where I, where I'm coming from, what my perspective is. Uh, I've worked a lot in, uh, WebUX as, uh, a designer, uh, was, had that as my main title for seven years, and uh, uh, yeah, now I'm just a, I just write code. Got sick of designing. Uh, and then I own a Google cardboard, which I forgot to bring with me, uh, and then a Google Daydream and an HTC Vibe at home. So if I down anything that's not those, then you might know my bias. Uh, so, yeah, I worked for Lollabot. Uh, we're an awesome company. You should work with us, because uh, we're all very cool. That's, yeah, what you want, right? And uh, we do strategy design and development. Uh, so content strategy and, and CMS strategy editorial stuff. Um, all sorts of awesome things. Yeah, so uh, first I'm gonna start by talking about kind of VR in general, and I've added some AR stuff to this, but that's not necessarily what the talk is about, but just kind of, so you understand how it fits in here. And then, uh, we're gonna go over, uh, terminology, uh, what the user experience is like there. Uh, hardware, cause that has a, plays a huge factor in what you're able to do and what kind of experiences you're able to deliver. And, uh, what on earth should you do or can you do with VR and what are people doing right now? Uh, and then we'll get more specifically into web VR and where that's at and where it's going. And a demo. Uh, cause this stuff is actually pretty approachable. So first, terminology. Um, so this is, I have no idea who Milgram is, but I thought this was a handy spectrum to have. Uh, so on the left you have reality, right? IRL. And then, uh, or yeah, and then on the right you have, uh, a virtual environment. So you can't see anything that's real, you're just looking at a bunch of 3D generated nonsense, right? Uh, and then in between those two, uh, you've, you've heard of augmented reality, uh, which is where you're mostly seeing real things and then there's some virtual elements and then, uh, virtuality is just that but flipped. It's, uh, mostly fake with some real stuff breaking in. So, uh, some examples of virtual reality. Uh, this is an HTC Vive. Here you see, uh, what that girl is seeing. Uh, she's, so she has two controllers you can kind of see in her hand. Uh, so she can see her own hands in, in the application as they're moving and she's petting a little robot dog and when she crouches down, you know, uh, she gets closer to the image, or to the robot dog which she feels like is in real space. Another example also at HTC Vive you're defending a castle. Um, so one of his controllers you can see, uh, is showing to him as a bow and then the other, his other hand is pulling arrows. Uh, and he's trying to prevent these little bathroom icon guys from, uh, attacking his gate. Um, so yeah, just, this is obviously not real at all. This is a very, uh, you know, kind of computer generated looking example but you know, it's, it's completely immersive experiences people feel like they have a bow in their hand and that they are, uh, doing these things. It's completely immersive. So augmented reality, some examples of that. Oh, and actually don't look at the man behind the curtain. Uh, I wanted to un- comment something. Is that option? No. Sorry. I meant to do this before. There we go. Alright. Back to my presentation. Uh, so augmented reality. Uh, so you're mostly seeing the real world and we're putting, uh, virtual elements into the world. So a popular example, uh, not that far back was Pokemon Go. Um, you had the option to kind of look at the world through your, uh, camera, uh, show up on your phone and then they'd try to overlay a Pokemon and a Pokeball, uh, on the actual ground so it looks like you were, you know, that was actually in front of you and only you could see it. It's your own magical little world. Uh, here's another one that's come out with this is, I believe the, uh, new iPhone. Uh, being able to, uh, measure things. Uh, this is, I love this. This augmented reality is really exciting because of things like this where you can, you know, they tap the screen to start measuring and tap it again to stop. Um, so you can measure out a whole room and then, you know, export that to some system where you can then start arranging your furniture. Uh, another example, this is, uh, that's an actual room in theory. This is probably some demo and then you can see what different IKEA furniture looks like inside of there. Again, really exciting. Um, it's not going to be perfect but it's better than, you know, imagining it purely in your mind. Hmm? Yeah, better than nothing. Uh, one that I really liked, uh, that, uh, Hyundai did was you could pick up your iPad, point it at your open hood and you could click through some tasks there that you want to be able to do. So hey, I want to change the air filter. Hey, I want to change the washer fluid. You click on it and it plays a little interactive, or not interactive, it might have been, but a little video where it would actually overlay a 3D thing of, of the part that you need to interact with and exactly what you need to do in the steps. It's incredibly useful, uh, as opposed to, you know, having to follow diagrams or whatever. It actually has markers in the, uh, hood so that everything should be very accurate when it, when it shows you where the air filter is, it should be right there. Um, so user experiences and interaction types. So it's very different, I mean, you're not using keyboard and mouse or, you know, whatever other kind of interface some touch, uh, or any kind of accessible interface, accessibility, uh, tech interface. Uh, so first there's kind of how do you, how do you interact with the world you're in? Uh, the, the lowest bar is the gaze or the look and press. Uh, so gaze is just, you'll generally have a reticle, uh, that's in the center of your vision. And if you stare at something for a little while then they'll assume that you wanted to interact with that. And so you'll enter it or, you know, something will happen. In this case, this was a team photo we took, it was 360. And if you looked at people, you would get their name, uh, they'll pop up above their head. Um, and then some of them if you, uh, can still find this thing, I think it's on vr.lolobot.com. Uh, if you click, some people have an Easter egg, uh, thing that you can, uh, find. Um, oh, and then also one thing with augmented reality is it has, uh, at least, at least with the new Apple stuff, uh, when you have, when you're looking through your device, you're able, if there's a 3D thing that's, you know, fake in the world, you can manipulate it with a touch interface. Uh, so it's similar to look and press, but a lot more, uh, nuanced. Uh, you also have traditional controllers, so a lot of them will have kind of media controller looking guys. Uh, or you might have a game controller or a flight stick if you're playing some sort of piloting thing or something like that or a driver's, uh, wheel. Um, but these things usually don't appear in, uh, in virtual reality except for some of these new, like this one, uh, kind, does, ish. Uh, and then there are VR controllers. Uh, these one, these two specifically are very well tracked where you can do very, uh, small interactions and they are tracked with, uh, great precision and very low latency. So it actually feels like your hands. Um, and Vive is actually coming out with something that looks more like the Oculus ones here. So the idea is that with the Oculus one, I believe, uh, that if, if you want to reach out and grab something, you go like that, you let go of the controller and the halo around it will hit the top of your hand and it's touch sensitive so it knows you've opened your hand because it's no longer touching the bit in the middle. So things are meant to feel more realistic like you're, you know, instead of you press A to pick up because that's not exactly, uh, you know, how cavemen did it. Uh, you actually open up your hand and grab something. So here's another thing that's getting big. Um, some of the more complicated AR things, not the apple necessarily, then maybe you can get into this, uh, would be gestures. So you have something that's actually able to distinguish where your hand is, even maybe what, which fingers which, which could be really useful. And then you're able to actually interact with things in space. So this is a demo with, um, if you know the technology, it's, um, the Oculus Rift and then they have a leap motion, uh, attached to the front of the headpiece, head mount and display. And, uh, they're a, the leap motion VR is able to distinguish hands and fingers. So, uh, a gesture could be pinky specific versus pointer specific or hand specific, uh, which is very cool. So that, they're just showing in this demo that they do know exactly where all of the fingers are, uh, while this person's messing around. So the next thing with VR specifically, AR, this isn't so much of a thing, uh, is body interaction. How, what, what can you do with your body that'll actually translate into the, into the virtual world. The lower level, uh, experiences. So, uh, if you're familiar with the devices, we're talking cardboard, daydream, uh, Oculus Rift, um, you, it's sitting or standing, which means that, um, you are able to turn your head. If you weren't able to move your head at all, it's not very convincing virtual reality. Uh, but you can, uh, also, you know, stand up, look around, look behind you. And, uh, there's, you'll be immersed in a, in a complete world. Uh, for something that is more complicated, uh, right now, the, the only way that this is getting done is, uh, so for room scale is, if I walk this way, the world moves with me. As opposed to if I, if I have this and I'm playing a game and I walk, my body says I'm moving and my eyes say I'm standing still, which actually caused some nausea, because that's a little odd. Uh, but with, uh, room scale, things like Vive, you can actually wander around and there's demos where you're like on the bottom of the ocean, you can mess around with a fish and kick a sea-in enemy. Actually can't kick it, they don't track your foot, but, um, you can, yeah, mess around things in the real world and it actually gets a lot more convincing than. Um, there is possibly going to be things where, um, the headset would be able to track the room instead of external sensors having to track you, uh, which would let you wander around and, and interact inside the room. Uh, yeah, virtual room. Hardware. So hardware is a big factor, obviously it's hard to render all this stuff, uh, takes some pretty solid, uh, gear. So, um, these are all sitting standing only, so, uh, Daydream, this is, uh, it's on the very bottom left here and the Gear VR, those are the two like mid-range ones. Uh, they're in the realm of $100, um, and then, uh, the Google Cardboard is a specification, uh, anyone can make it. It just says, okay, if you want to make hardware for Google Cardboard, it has to fulfill these things. If you want to make software for Google Cardboard, you have to do these things and that way you can kind of match up and, you know, uh, in this case it's, um, oh, I can't even remember who makes this headset. Someone's headset with, uh, you know, Joe Schmo's VR app and they'll just kind of work together nicely. Um, these aren't as high fidelity as, as something like this, but they are quite good. Uh, it's completely limited by phone hardware. So, um, the, the newer phones like the Pixel or have, have bits in it that are specifically for VR, um, whereas, uh, the Gear VR originally had extra hardware in it to, uh, to kind of supplement what wasn't in the phone. So, uh, that's going to be changing a lot, especially now that Apple's gotten into the game. Finally, it's been years, they bought a lot of VR companies and they finally came out with something, so that's good. Uh, be seeing a lot more interesting things. So, um, the higher end headsets, uh, you're talking about having space to be able to wander around probably. Uh, you're talking about having, uh, VR controllers, they range from, I think it actually went down, I should have updated that. Uh, just the headsets and the controllers and stuff, uh, those, that package is about, uh, I think 500, or maybe 400 to 700 now might be a little off. And that's all American, sorry. Uh, those are the, that's the currency I'm talking about. Um, and it requires a powerful computer. So, that's in the realm of probably about $800 if you're starting from scratch, uh, to get the computer that can power this without causing issues like motion sickness. Uh, motion sickness is usually caused from, um, uh, a bad refresh rate. So you're not getting enough images to your eye quickly enough. Uh, or, uh, bad, uh, UX design where somebody could do something that, that just really confuses your biology. Um, oh, and then the other thing that can be caused by is, uh, if the lenses aren't aligned right, all of the lenses are made very specifically so that you can stare at a screen that's an inch from your eyeball. Cause if you just try to do that with your phone without anything, it just, it just looks like a blur. So, uh, the way the lensing works, you want to have your eyes right in the middle of the lens. Um, and if they are way off that can also cause some motion sickness. Um, so yeah, applications for VR. Uh, so, uh, gaming obviously a lot of enthusiasts right now are into that. Uh, there's also, uh, along with gaming there's a kind of experience, kind of stuff that's less like a game and more like, you know, being somewhere that you couldn't be normally like I mentioned bottom of the ocean or defending a castle from bathroom icon people. Uh, photos are really cool. I, uh, had the chance to go to Iceland and, uh, some other places and took some 360 photos of my phone which is kind of like taking a panorama, uh, on Android but you go up and down as well as left and right. Uh, and then I can show people, you know, really cool things that are immersive and they can look around and, and kind of pick out the details they want to look at. Um, and then as well, uh, 360 video which you actually need a special camera for. Uh, they also make special cameras for 360 photos but, um, they're less common. Uh, you can also demonstrate micro and macro scales of things that you can't really, you can't shrink yourself down. Uh, like I've seen some educational cartoons do when you go through the human body but in VR you could do that. You could get someone into, uh, the blood flow and show them how red blood cells work. You could also demonstrate, um, the, the planetary system and all the moons and all those things. So macro scale, you know, going out and seeing things that are enormous and being able to zoom in and out and, uh, examine them. Uh, tour museums, distant locations, event reenactment, things that, again, you can't do necessarily in reality for whatever reason, either it happened long ago or it's very far away. Um, and one thing that I find really cool is therapy for PTSD, phobias and brain injuries. Uh, PTSD and phobias are exposure therapy. So, uh, for instance, there's some apps that'll, that'll put you in a room full of spiders. Uh, you can kind of control how horrifying that is depending on how scared you are of spiders. I didn't kind of get used to it. Um, and it's, you know, it's not going to be, you know, you're probably still not going to be buddy-buddy with them but, uh, that kind of exposure therapy does actually work which is really cool. Uh, same thing with heights, there's heights apps and, uh, all that kind of thing. Um, and, and brain injuries would be, um, that they're actually able to focus the, uh, therapy very specifically on certain experiences. It's, it's a much more controllable environment. So I can focus on specific parts of the brain. So I find that very, very cool. Uh, and also training simulation. So it's a lot easier, uh, to, if you're going to train someone to work on a nuclear reactor, there might be multiple ways consoles could be set up. Having to set up physical consoles for that could be very difficult and costly. Um, same thing with maybe, uh, certain kind of flight applications like NASA or, you know, uh, military or whatever. Um, if you can, uh, have a convincing virtual reality experience, that might be a good way to get people trained up on all sorts of different things, uh, as well as having them train on physical devices. So, that was all just kind of VR in general. And AR. Uh, so web VR. Web VR is very interesting. So the, the biggest strength of the web as usual is, uh, you don't have to install software. There is no app. You just, you know, most of your devices come with a browser and you go to a thing and there's some new software experience. Uh, but you can do that in virtual reality, which is very cool. Um, so most VR is the biggest thing going on in this space. Most VR, uh, is an effort from Mozilla, obviously, to try and get, uh, VR things into the W3C specification. What this means is that there'll be an API that you can interact with in JavaScript that'll talk directly to the hardware and you'll be able to do things that, uh, um, maybe you don't want JavaScript to be doing because it's too much heavy lifting. Um, so lower level system features like, uh, I think augmented reality, I would just, I don't know yet, but I would assume that, uh, you don't want JavaScript to crunch the layout of the room. You want something that kind of just, uh, so like, AR basically would have to say, okay, there's a line here and a line here, so I'm about this high off the ground and this, that and the other thing. Oh, now I can place this 3D thing there. And if you have, uh, underlying hardware in the phone that crunches the room already and then my JavaScript says, well here's my 3D model, can you put it 10 feet in front of them on the floor? Uh, that would be a lot less work for JavaScript to do, which is not, you know, uh, something that you want to be doing a lot of, uh, heavy lifting as opposed to low level system kind of, uh, stuff talking to the metal. So, yeah, that's gonna be, so it'll be able to talk to controllers, the headsets, uh, be able to figure out location in the space potentially, uh, all these kinds of things and, uh, not only are they trying to get out in the browser long term, they're also creating libraries that let you do this today, which is very cool. Uh, and kind of assembling open source things that are assembling and helping open source things to make like, uh, things that you can just get started with now. So, uh, one of them is WebVR boilerplate. So this is, uh, it's a starter kit, you know, it's just a GitHub library, uh, where, you know, it has your package JSON and, you know, everything set up so that you can create a custom application. Uh, very JS heavy, very developer centric, uh, which is fine. Uh, if you wanna do something custom, this is a great way to get started, uh, if you're looking to do WebVR. Uh, A-frame is the other thing that they're working on. A-frame is kind of like JQuery for VR, it just makes it a lot more approachable. So it's, uh, a markup language, uh, where all the tags start with A dash, so there's A dash box, A dash camera, that kind of thing. Uh, and then JavaScript will, when the page loads, JavaScript will take it over and make it into a VR experience with two eyes, uh, two images, one for each eye, and give you, uh, all sorts of other cool tools, like, uh, being able to move around or being able to have a pointer or, uh, also gives you a reticle and, and the notion of click and hover and all these kinds of things. Um, so, uh, the other thing is that it can be further mod, modified with JavaScript, it's not like this is a, out of the box untouchable thing, uh, somebody who, uh, you know, if you have a, more of a, uh, a design light front end kind of person start something, a, uh, more focused JavaScript developer could pick it up and do more complex functionality with what the more design-oriented person started, uh, which is very cool. And then the, the, the project goal for this in general is to enable more creators. So, have people that, uh, that, you know, more creative, kind of artsy types that may not know so much code like, so they can come in and do something crazy and, uh, uh, yeah, see, see what they come up with. So, A-frame tutorial time. So, uh, good news is, A-frame docs are fantastic. Uh, it's, it's a single JS file. Uh, you can set this up on code pen, which is usually what I do. They have an official like code pen, uh, pen that you can fork. Um, and really it's just including the one JavaScript file and then giving you some boilerplate markup that you can get started with. And no compiling necessary. Um, but there is also, uh, libraries like React VR and probably some other ones that I can think of right now. Um, so to get started with it, this is literally all you need. Um, I have my, this is linking out to the A-frame website and getting a release that is probably old at this point. Uh, and then you, uh, put in an A-scene tag and scene is, you know, kind of a room, a thing that you want to start building on, right? Uh, and from there you just start putting in elements. So, instead of just talking about that, let me just show you that. So, wait, uh, I went here. Alright. So, uh, I'm gonna cheat a little bit. I have a bunch of stuff commented out so I don't have to remember all of this. Uh, so, uh, here's an example of a box. Actually I'll just do that. So, uh, actually before I save that, let me show you where we're starting. So, this is, this is our scene. Isn't it wonderful? It's beautiful. I just, I see everything. I see all the possibilities. I see life, death. Uh, alright. So then we save. And now there's a box. Alright. Cool. Box, right? Uh, oh, another cool thing about this. You can't inspect it like that. Uh, it's not that cool. But, um, this is all very, uh, straightforward to work with. So if you, where to go? If I change things in here, like, you know, you do with, uh, html, it just pssht. Oh yeah. I totally do that. So, you can write just vanilla javascript to interact with anything here because it is a tag in the DOM, um, that is being watched by, uh, the code that's rendering this giant canvas element here. That's actually becomes the VR experience. You'll also notice here the little goggles. The goggles are, if I had a headset attached to here. Oh, it actually tried. That's interesting. Uh, usually it tells you that you don't have a headset. But maybe with Linux it's getting confused. Uh, but yeah, if I brought up this up with my phone and pressed that button it would put it, it would come up with two eyes and then I put it in my headset and then I could walk around this space. Uh, you can also, if you're a gaming dork, WASDA, W-S-W-A-S-D, you can walk around in the space and then, uh, click to move your virtual head so I can, ehh, ehh, ehh. Look down at my beautiful cube. Alright, so we'll refresh that. It's no longer blue because I didn't actually make that change. So, uh, yeah, so we'll start with a great cube. We can also do things like, uh, change the width. Uh, so now, you know, it has width height parameters and if you're wondering like, where did I find all that stuff out? Well, you just go to the A-Frame docs. Like I said, very good. Um, so here's a nice code example. Uh, apparently there's a color tomato. You can probably use chartreuse, that's, uh, another web color, color that you can use, uh, in CSS. Uh, imagine it's here as well. And it's so depth, height, width. Uh, and then there's all sorts of, if you want to get really nerdy about there's all these attributes and you can kinda figure out what they're about and, uh, what they give you. So, uh, go back there. Uh, and then we can also mess around with the, uh, positioning of it. So, uh, moved a little over left. So the way these coordinates work is X, Y, Z. So, uh, X is going to be left-right. Y is going to be up-down. And Z is this way, that way. Uh, by, so you, you notice by default, um, right here, zero, meaning cubes on the ground, right? Um, that I'm looking, I can see the top of the cube. And that's because by default, uh, you get a lot of things out of the box where they frame. So you get a camera, uh, and the camera is me. And the camera is 1.6 meters off the ground. Other fun fact, uh, being American is a disadvantage, cause I don't know what meters are. Uh, but you all have a head, uh, a leg up there. So, congratulations. Uh, and, yeah. Uh, you get other things like a default lighting source. And, uh, whenever, if I just declared a box and I didn't give it any attributes, there are a bunch of default ones that are there anyway. Um, oh, it's probably white by default. Ha, ha, ha. It's there. Trust me. Alright. So, uh, another thing we might want to do is we might want to put in a ground. So, uh, giving you a line for each attribute here just to make it a little more, oh, is that big enough in the back? Yeah? Okay, cool. Um, actually I'll, I'll bump it up. I don't need that much. Uh, so, uh, yeah. So we have a nice green ground. It's 500 meters wide, 500 meters tall. And if you're thinking grounds are usually laying down, you are correct. Uh, so it's at 0, 0, 0, but we rotated it along the X axis to lay it down. So it's negative 90 degrees, so our ground is laying down. Other thing you'll notice, if you, uh, ever try this out, if you go positive 90 degrees, you won't see anything because of weird 3D reasons. If you look at the wrong side of a plane, it's not visible. You're supposed to look at the right side. The correct side. Not the right side. Anyway, there's my cube. Oh, my cube got shorter. But my cube still 1 by 1 by 1 by 1, I think. Or is it 2? Yeah. It's 1 by 1 by 1 by 1 because that's a default height width dimensions and I specified that. So that's weird. Why do you think that is? I'll tell you. Uh, so it's because the positioning is set from the center of the object. So we're actually seeing, uh, 0.5 meters tall because the other 0.5 is underneath the ground. So if I do a little bit of this, bam. Now it's perfectly touching my ground plane. Hooray. Alright. And then, uh, let's say we also want a sky. Uh, so sky is an element, uh, that's a concept in, in 3D things. Uh, sky is basically, by default, an A frame, I can tell you that it's 200 meters away. It's a giant sphere and you generally put a, some sort of color or texture on it. Uh, usually clouds or something. But in this case, I, I came up with a clever blue, uh, before. And so now we have a sky. Um, other fun fact, um, you want to cut down on planes as much as possible. Planes meaning like surfaces in 3D. You want to cut down on, uh, those as much as possible. There's a better word for that, I forget. Um, and skies, since they are sphere, have a whole lot of faces, faces, that's what it is. So in order to make a sphere, they have to put on a whole lot of flat shapes to make it feel sphere-ish. And that actually does slow down your application. So one easy way to cut down on, uh, performance issues is to try and figure out how to not have a sky image. Instead do like a cube around you, like if you're in a room or something, that's a lot less faces, uh, for the, for the rendering to deal with because it's having to cast light off of that and figure out how that should look and where it doesn't space and bleh. Uh, okay. And then, but yeah, for what we're doing it's fine. Um, so, let's say that, um, we wanted to do an animation. Let's see what that looks like. So, huh? We're out of time. I don't know. Okay. Well, just this last little bit. Animation looks like that. Uh, yeah. So I can, uh, post, actually I think the link is in, um, the notes but it's in my github, uh, and you can take a look at it and the docs are great. Um, but yeah, there's an animation. It looks like, kind of like SVG where it looks like HTML kind of, um, and you're just specifying things like that. And there's all sorts of things you can control. So, yay. Okay. So I know you don't have time for questions, but I was just going to state one thing which is if you haven't looked at it yourself, the A-frame inspector, which is just a property you apply to the scene, which allows you to have a actual external view of the scene. So Do you remember the keyboard shortcut? It will be worth demonstrating because it's probably one of the key things that actually helps you build your scenes. Yeah. Do you remember the keyboard shortcut? I can't remember. It's control-ult-i, but you have to have the property attached to your scene. There we go. Yeah. Yep. That's awesome. It's the best. So you can actually drag it around. Ah. Yeah. Uh, it is, oh, uh, if you go to, yeah, github.com slash westrove slash reveal.js and all of my presentations are in there.