 Hello, I'm Alex Krasner. I'm the GA at the virtual environment studios at the libraries of Virginia Tech and Today we're going to be learning how to make pass-through Applications for your Oculus Quest headset. That is almost like an augmented reality experience Inside of the quest using the cameras to show us the outside world inside of virtual reality we're going to start today by Getting unity set up for our pass-through applications. We're going to Go over hand interactions. That's including grabbing and distance selection. So a couple of disclaimers There's not going to be any scripting today. Everything will be no code whatsoever however This workshop does not Cover the basics of unity if you're coming here and you want to make an app But you know nothing about unity or you're gonna want to go to my other video that I've already made that just covers the basics of unity kind of tells you how the layout works and everything and Yeah, that should be available on a similar site Or it's potentially if someone shared you a link to this Maybe they shared you the link to the basics one as well. So anyway Without further ado Let's get on to the workshop, okay So the first step before we can even open unity is that you need to make it so that you can Connect your quest to your computer so that we can push apps to it Unfortunately, this requires you to put together a developer account and To get a developer account you're going to go to developer dot oculus dot com and sign up on the right there should be a button to sign up you'll sign up and Once you have signed up, you're probably gonna want to scroll down here on the same site developer doc is calm click submit and you should be brought to a page here where it will ask you to give it some verifying information a phone number or a credit card and If you Would really like to preserve your privacy. I would recommend Maybe using a website like privacy comm to create a fake credit card that You can access with your real credit card and providing that instead as your verification One thing however is that you will need to make sure the same account that you are using your oculus or meta headset with is The same account that you sign into for your developer account, so it will probably ask you to sign into Facebook With an account if you've already linked your thing with Facebook, you have to use that account if you haven't then maybe you'll be able to make a side developer account After that you will need to make an organization. I Would just name your organization It'll be up here on the left and I would just create an organization with the same name as yourself and That way you shouldn't have any conflict every organization has to have a unique name think of it almost like a username It's the name that would show up in the app store if you were to publish your app, so Do with that information what you will you need to have your developer account verified and You need to have an organization that you are within Organization can kind of just be whatever you make up though And once you have gone through making all of that your account will be ready You will now next need to go on to the app on your phone and enable developer mode You can probably find tutorials for this online But it's as simple as going into the options going to devices And you should see a developer mode that pops up as soon as you as soon as you have created a developer account so Once all of that is done Your headset should be set up such that you can actually develop for it, and I'm gonna close this now and Let's get into unity So when once you have your headset set up for developer mode, we can start developing so I'm gonna open unity Opening unity hub and with unity hub now opened We can see, you know all the projects that you have You want to make sure for this that you are on 2020 preferably the latest version of the LTS or long-term support version of 2020 today. We're gonna use 2020 you can most likely do all of these things We're about to do today in 2021 however Due to stability reasons, I'm going to stick with 2020 for the tutorial today Anyway, once we have this one installed though, which I've already installed if you don't have it installed You'd go to install editor and You would choose it from here One important thing is when you're installing the editor a page will pull up that looks like this and If you have already installed it you can just pull up that page by clicking on the gear next to your install I'm going to add modules It is critical for building to an Oculus Quest To have Android build support and all of these other tools SDK and the open JDK You got to check that box. It's about two extra gigabytes of download So you need to make sure you have those if you don't have that you will not be able to build to Oculus Quest. I Would also recommend turning on Windows build support IL-2 CPP because if you have this on you can preview the view of What your apps going to be however, we will not be doing that today due to pass-through restrictions When you use pass-through video You need It does not show up Whenever you are recording it and that includes through the debug screen in Unity So anyways, make sure you have this installed and you will be good to go So with that installed I'm going to go to projects and make a new project let's go with 3d always and Name it whatever you'd like. I'm going to call mine Quest to workshop pass-through You can name it whatever you'd like make sure it's saved somewhere that you can keep track of and Create the project So when unity finally opens You will see something like this. Maybe you've changed up your layout. Maybe it looks slightly different But you should see something like this again. I have gone over all of the UI components here and what all of these buttons do in a previous video If there was a way for me to link it I would ask that someone adds a link to the video right here so that you can go and click and check it out so Anyway What we want to go to is the asset stores the first thing we want to do we need to get the Oculus integration and To get the Oculus integration, we're going to go to the asset store It's a tab right here The tab may not show up if the tab does not show up. You're gonna click window asset store and the tab will show up But in the tab all it does is just opens your browser. So Here we are at the asset store To get the Oculus integration, we just want to type Oculus integration I Have found that this site has a tendency to be kind of laggy So when you type something in and it doesn't immediately load do not worry. That's kind of common this case we just searched it and And it didn't even show up. So that's exactly as I'm saying sometimes this site just kind of glitches out If it does that type it again Oculus into creation And here we go. So the top thing that shows up will always be an ad ignore it Here it is right here Oculus integration. You'll see it says I've already purchased it, but it's free So when you get to this page, it should show to you that that It'll say add to my assets or something to that likeness if it doesn't say that one important thing to make sure you have is That your unity account You need the same unity account signed in on the browser and In unity itself. So if we check up here in the top corner, you'll see that you can sign into an account You want to make sure it's the exact same account signed in here That you have signed in here so Given that you have them both signed in you'll click add to assets and then Afterwards, you'll click open in unity a pop-up might show up here saying open in unity You can click either or all it will do is open the package in unity And we can see it here now Right here. It may not always start highlighted. So it could just show up in here look for Oculus integration and When you click on it, you will likely see a button prompting you to download it You'll download it and then you will import so Now I'm gonna click import And when the loading finishes you will see this Screen all you want to do is click import again. It's basically it's pulled together all the different files that you're about to import So you want to choose to import them all You're going to notice a theme today That there will be a lot of waiting for bars And that's just given that's how it's going to go there may even be if your computer is fairly slow this Waiting could total to around 45 minutes of total waiting through the whole thing. We're doing in this video today I will be skipping over all of my weights So you will not have to wait through the six minute or more wait times on my end But you will have to deal with however long it takes on yours So sit back. I'll pull up a video on your phone Have some fun Play a little game listen to some music Let's chill out while The loading bars do their thing All right, so when the import completes This will appear what you have to do is click yes Use open XR Okay, and then it will ask that you need to restart when you click this restart However, it will instead pull up this where you have to choose upgrade and Then it will ask you to restart again when you click this restart. It will actually restart unity Don't worry Everything will be as it was when you reopen All right, so we have finished importing the Oculus integration package what this means is We just finished the first step There's a few more steps to go and a few more weights So on to the next one The next thing we have to do is we have to change which platform We are building to So By default it's gonna build to PC. We don't want that we're gonna build to Android, which is what the quest runs on So I'm gonna click build settings under file file build settings In fact, I have just turned on a new feature so that when I click I'll show you this right now when I click normally it will be yellow when I right-click it'll be blue so I Hope that will help you throughout this next part of the process and go to file build settings and While we're at it, let's click add open scenes and now You see right now it's set to build for PC Mac Linux, but that's not what we want We want it to build for Android. So we're gonna click on Android here You will not be able to do this Unless you have installed that package that we we checked on earlier in the Unity Hub But given that you have installed all those Android tools You'll see all this stuff here. You can leave it What matters is we need to switch the platform to Android. So I'm gonna do that now This will take a while It can take Around six minutes or so on an average laptop Just depends how fast your processor is Who knows by the time you're viewing this maybe maybe processors have had some massive jump in performance. So But this this will take some time once again go get a drink go pull up a game on your phone and Chill out until it's done. Okay. So once that weight is done It's time to move on to yet another way To get to that we're going to click on a player settings this little button down here and We are going to go to the bottom where it says XR plug-in management We're gonna click install XR plug-in management. It's gonna take a little bit Generally doesn't take as long as the other things so far though. All right, so once we have Installed XR plug-in management. The next thing we have to do is Make sure that your tab is set to Android over here And you're gonna click this Oculus integration So let's click it and this one could take quite a while So I know maybe go get some snacks now All right, so once that completes you've got an option here Under the PC tab Is it the one that looks like kind of a screen desktop tab? You could also enable Oculus in here if you do it means that you would be able to preview Your app through the debug view over under under here up here in game However with pass-through that's not very helpful generally since The quest will not pass its video feed Into unity So the only real upside of doing it is I guess to test out Your tracking quickly and easily However, it really won't take that long to just build the app to your device over and over again But you will miss out on bugs if you were trying to write code So it's a trade-off you can decide on today. I'm not going to check the box But you could if you were interested in debugging so The next step will be to go up here to player project settings and down to Other settings it may not already be open. It may just look like this You don't want to open other settings and the very first option rendering color space We're going to set this from gamma to linear You'll see when you click on it that it warns you this could take some time and it will this may be the longest wait But I'm going to click it And it's important that we switch this to linear Because the quest can only run applications in a linear color space It's part of why it runs so efficiently even though it doesn't have Some crazy computing things in it. So anyway this Again can take an average of like six six and a half minutes, but for some people it may take more Maybe even take 15. Who knows All I do know is Go do something fun And I'll be right here When you're done All right That's your last wait We finished all of the really long wait times of this process And it's on to Finally putting this stuff together. There's only one thing left though In these menus, but you're going to scroll down in other settings until you find Keep going down Until you find configuration In configuration. We want to change two things real quick You also want to double check something so Let's let's double check it real quick right here. Just make sure that this is set to api level 23 Android 6.0 marshmallow. It should do that automatically when you install the oculus integration, but Just pointing that out All right, we want to change the scripting back end under configuration to il2 cpp and then We want to go down To where it says target architectures and we need to change it from arm v7 to arm 64 And that's it. That's all the weird settings. We got to deal with at least for now in this menu So I guess in in general for the menus. That's that's it So closing those two windows we're back to here And the next step is we want to get rid of this main camera We're going to be using a vr camera for this Or an oculus uh ovr rig And so we don't need another camera. So let's get rid of that. I'm just going to right click on it and click delete Okay Time to Add the Sample camera rig that we're going to be using today This is something that's pre-built by oculus In order to kind of give us an easy way to get started in making All these interactions work so All we got to do is type in Oculus into your project into your project section you want to type in oculus Into Her And by this point you can already it's already here It's this one that we're looking for one that kind of looks like Hands fused together with controllers. That's what we're going to pick And it's called oculus interaction sample rig dot prefab. So we're going to take that Grab it drag it up and just drop it over here on the left under the hierarchy And boom there it is we've got our new vr camera sets everything's here Though We're going to be making some changes to this and because we're going to be making changes to a prefab I always recommend whenever you're changing a prefab in unity to right click it Go to prefab And unpack completely what this means is when we change settings about this camera It's not going to affect the Base like the original version that we got so let's say we wanted let's say we messed something up and we wanted to go back And make some changes again in the future Uh, like we messed something up so bad. We just kind of want to start from scratch If we don't unpack it's going to change This object in here the one that we just got this prefab It will permanently change that one if we edit the prefab up here But if we unpack it it kind of Unwraps all of the parts from being a prefab So now they're all just objects and unity so you can see In the oculus interaction sample rig object we have Two children an ovr camera rig and an input ovr ovr camera rig is what you may expect It is the camera, but it also includes the tracking space and the hands or at least the Things to show where they are you'll notice when we expand we have a hand and a controller There's a prefab for the controller to show that And this is a prefab thing for the hands so that it can keep track of the finger positions and other things All you really need to know here is that that's what this is is the camera rig the hands are in here And they will move around automatically all the scripts are already handling everything you need We will be making a change in here though. I'm going to go to center eye anchor And we're going to change this background right here Change this background from blue to black And basically what that is is when the camera renders If it doesn't see anything it will color the background whatever this color is that we've given and in Pass through mode It always considers it as not seeing anything so it will color Everything you see through the cameras a little bit blue if we had it set that way now It's black which means it won't add any color if you wanted to somehow color it for some reason you could set that here All right, so we've done that again. It's under center eye anchor So now that we've set that up We should Mess with the settings a little bit There's a couple of things we have to change So under ovr camera rig object, you'll see there's a whole lot of Components here and the one we really care about is ovr manager the very first one In ovr manager We want to first change the color gamut to quest because that's what we're going to be working on Next under quest features There's a whole lot of different options here Where we will soon turn on pass through To do the pass through parts, but first i'm going to start us off in just a virtual reality app. Let's say you We're watching this hoping to build An application that is just you know a vr app for the quest Then I want to be able I want you to see how to do that as well So we're going to start with that and then we will build the pass through onto it All right We're going to start with hand tracking. So we're going to click here under hand tracking support change that to hands only And you can leave the hand tracking frequency to low If you have your quest running on version 39 and above Low hand tracking low frequency hand tracking will Provide the highest accuracy at the moment At the moment of writing of making this video that is the case that may change in the future But right now they've really optimized their low frequency hand tracking model and their high frequency model has gotten a little bit more finicky So that's just to know right now It could change and it likely will change Anyway, that's what we will leave that on leave the rest of it as it is And uh, we're not going to deal with mixed reality yet We just want to make sure that uh that the Tracking is set to hands only and we also want to make sure double check here. Sometimes it won't say this Make sure the tracking origin type is set to floor level. This means that whatever we have at 000 is the floor and When we stand up or when we put on our headset it knows to put anything we put at 00 At the floor and not at wherever your eyes were when you started So very critical very important. Make sure you have that on Okay, so let's start building our little kind of play space our little world to chill in Uh, first thing I'm going to do is make a cube. So I'm just going to click over here and uh Let's add A cube there's several ways to add objects as you've probably seen from my other video I'm going to do it from up here for this one. Just go 3d object cube And it places it in who knows where I'm just going to reset Again, I click the three dots reset to bring it back to somewhere usable Move it. I would like it to be in front of our camera So if our camera is pointing this way we can see that it is inside of the view and if we go to game We can see that it's there Okay, but this cube is pretty massive. Okay, this cube Is currently like a meter tall We're going to be playing with this cube almost like it's a building block So I worry if we make it that big It's not really going to be usable You don't typically pick up meter by meter cubes on a day-to-day basis and you definitely don't stack them So let's make this much smaller. Maybe like a tenth as large. I'm going to type 0.1 into each of these If you're doing this in unity 2021, you'll notice there'll be like a Link thing here and you can link them together and then just type it type it once but Today I'm in 2020 Okay, so you can see our small little cube here. I think We should probably position this better Again, I'm flying around by right clicking here and then w ast to fly around also e And q can fly up and down Okay, I'm gonna move this closer each of these lines is about a meter So how about, you know around a meter in front of us? I'd like this cube and I'd like it to be, you know, maybe 0.8 meters above the ground that seems fair um Measuring it right now looking in my room. In fact, let's go all the way up to a meter I think I think one meter height one meter of height sounds pretty good to me Now when we have this cube here Uh, it could be a pain in the butt to gotta lean over and pick it up off the ground over and over again Let's make a shelf to Kind of support this cube. So I'm gonna Right click on cube and go to duplicate. I'm gonna rename this for our sanity's sake. I'm gonna call it shelf Okay, I'm gonna move shelf down And let's scale it a bit. We can just do this by hand however you want You'll see up top. There's different tools when she's the scale tool And I'm gonna expand Our shelf this way a little bit this way. I think it's a little bit too Thick, so let's make it a little bit thinner more like a shelf That looks good to me Cool. Now we've got our shelf. There's still something we're missing though right now There's no floor and what I mean by that is When we go into pass the room The the quest doesn't know where the floor is It it doesn't keep track Of it and like make an object It doesn't look at all of the things in your room and build hard surfaces so that we can you know Place the cube on a chair in your room or place the cube on whatever else in the room At least not at the time that I am recording this so We kind of have to just tell it that there should be an object there and the most important object to exist is the floor so i'm gonna Make a new object. I'm just going to right click over here create 3d object plane And it looks like it's in the middle of nowhere once more. I'm just going to reset it to zero zero And it's kind of centered on our camera. That's perfect. Um, but We're going to be in pass through. We don't want to see that we put an object on the floor. We just want to know That the thing is keeping track of the floor that the floor has some collision surface So let's remove The renderer from this plane So now it will be see-through We won't see it But if the cube was to fall it will land on the floor All right a few more things to do before we're ready to test our app. I'm going to go to the Assets folder And in assets. I gotta click the x to get there Okay in assets. I'd like us to make a new folder. I'm going to call this materials So I'm just going to go to create folder Materials I like to keep things organized. I would highly recommend it From one dev to another Please keep your stuff organized. So here's materials In the materials folder There's nothing here But you may suspect from the name that we're going to fill this with a couple of custom materials again From the basics class, you'd know that materials are something similar to a texture for a 3d object They're not exactly a texture. They're more than a texture. They have all sorts of different properties We're going to make three really simple ones. All right, I'm going to right click in here create Down to material I'm going to name this one Blue You can name these whatever you want. All right, you don't have to be blue I've run this workshop several times and when I do Last time I had like a dark blue and I called it blug Uh, and I made a dark green and I named it gregg You can just have fun make up some colors. You'll see here. Uh, we have our blue texture It's clearly not blue right now, right? We want to make it blue to make it blue We're going to click up here on the right where there's this little color splotch and boom We see this whole menu. I'm going to make it blue and be a little bit dark Ish I kind of like darker colors on my objects. That looks good to me Here's our blue. So yeah, you feel free when you're making these objects. You don't need to make it blue You can make whatever you want. You can Choose all sorts of colors Make whatever you you'd like. All right, there is one exception though Humor me in this We all need to have a red material of some sort some kind of red because You'll you'll see you'll see so I'm going to make a I'm not going to make a red right now. I'm going to make a uh blue We already did let's just let's do green green sounds good to me Uh, wait for people who may be red green colorblind actually how about blue and Yellow yellow is good. Yeah, that's a good color Mm-hmm Or perhaps like a maybe a gold like an orangey yellow that sounds nice So I'll go with gold. I'm gonna Click over here make it a little bit more orangey That seems good goldish vaguely all right And finally as I said, we want to make sure we make one red Red I'm gonna make this one A nice red not too bright Just something good Feel free to do whatever you want, but I'd recommend just making make a red It'll I think it works best with what we're going to be doing today Just have at least one red now for the color of this cube. That's what we're going to do with these Uh color it anything but red. It's up to you. I think I'm going to start with a blue cube I like the look of that seems good to me And there's only one thing left to do for our cube our cube right now If we were to run this We would see that it doesn't Fall because it doesn't have physics at all. It's just gonna sit there in the air. There's no way to collide with it. Nothing We need to give it physics to give any object physics in unity. We just add a rigid body Rigid body just type that in add component rigid body Don't click rigid body 2d And we can leave all this as is So at this moment if we were to run the application We will see That there is a cube sitting on top of our shelf We'll look around and we'll be able to see our hands. So Let's build to our headset right now. Like let's let's check it out. Let's try it out Okay, so All that that all we have to do for that is first Save Save frequently and often you never know when you're going to lose your work And let's go to file build settings I saved with control s by the way Be command s on mac Same shortcut as every single software you've ever used Okay We're in our build settings now We want to Build and run this on our headset However, it's important to note We need to get our headset connected So assuming you went through all of the steps to set up the debug And not the debug to set up the developer mode what we have to do is Get our headset plugged in so i'm going to take my quest I'm going to take my Oculus link cable I'm going to plug it in And here let me give you Quick view let's go back to The camera view there's a mic in the way, but I'm going to quickly put this on I'm going to plug it in You can see There's a port over here You will need a usb-c cord it can be usb-c to usba Or it can be just usb-c to usb-c The headset comes with usb-c to usb-c cable so you could use that if your computer has usb-c ports What's going to happen though When you Plug it in is you're going to see something pop up that says allow usb debugging question mark On this you can click allow or you can click always allow from this computer I you can't see this menu at this moment, but i'm telling you right now It's going to say allow and always allow from this computer Choose whichever you'd like i'm going to click always allow because it will minimize Our number of clicks in the future But then after you do that it's going to ask you to allow it again If you chose to allow and not always allow for this computer You would have one more allow statement popping up right now Three allows in total that you have to click all that we're doing Is giving the computer access to our headset so that we can push our apps to it Anyway, take this off and we will get back To unity all righty So we're going to build to our quest One thing that's critical when building to your quest from unity Is you need to make sure that unity even knows that your quest is connected So over here in run device I'm going to click refresh I'm going to take a look and there's my oculus quest. I'm going to click on it and set it here You're going to notice every time I every time I go to build to my headset I'm clicking refresh and I'm checking to make sure that it's connected because if I mess up and I don't connect it Then it could get annoying and it's going to be really hard to follow these instructions If you don't make sure that your quest is connected every time. So again refresh that Then we're going to go to build and run here And you can choose anywhere to put it I have a folder on my computer for builds So I will just put it here My builds folder and you can name it whatever you'd like. It's not really going to impact too much So I'm going to call this one Work shop demo pass through Call it anything you'd like Make sure it's set to apk and click save and when you do that You're going to see that it's going to start building And welcome to another wait This wait may not take that long, but it can take a while sometimes depending on your computer However after we've done this wait one time Every sequential time afterwards is going to be so much faster It has to do a lot of compiling Right now, but once it's already done it then it's just Simply just going to make a small change and then compile that so the First time is the longest after that it will not take that long So catch you on the other side of this loading bar All right, so the app has been built Time for me to show you what it looks like. So I'm going to switch over to This and I'm going to cast it so that you can watch What it looks like through my headset Let me get back to that Here we go All right, so I'm going to unplug my headset from the computer doesn't need to be plugged in you've already built it You can leave it plugged in if you'd like And made with unity showing up Time for me to cast so I can show you what it looks like All right So you can see now let me uh full screen this for you make it nice and clear Okay So here's what it'll look like at this moment if you were to compile it it should look almost exactly like this There is a glitch that can happen occasionally That your hands get tracked as controllers all if the dude fix this is tap the sleep button on the right side of the Or the power button on the right side of the headset And then tap it again to turn Back out of sleep mode and you will see that your hands come back into place Another really important shortcut is to If you point your hands towards you you'll see that these interface components show up I'm gonna pinch here and hold And I'm gonna move up to recenter view And if you stand for a second It will recenter the view where you need Which is really helpful when working on this project. So cool We can see if I walk over and my I might get a little bit away from the microphone So pardon if I'm a little quiet, but when we walk over I can't actually touch this cube, right? There's no way for me to interact with it So Let's fix that So Let's fix that there's also something else you might see right now Which is we're in like a virtual space And where we want to be is in A pass through Mode that's kind of the idea of this workshop. So to switch it into pass through We're we're gonna be changing some settings All right, so for now I'm going to close out of this I close out of apps by tapping and clicking quit And for now, we'll just take off my headset And let's go back to unity So back in unity Let's make those changes. We just talked about Right first things first, let's add in our Let's add in Pass through It's all I have to do to add in pass through Is we need to go into this rig into over your camera rig right here. We're going to change Let me just double check that everything's capturing it is We're gonna change our Thing down here from pass through capability enabled being off to pass through capability enabled being on And you'll see that that turns on this section All we need to click down here is enable pass through And there's one more step In here at least We need to go to add component and type in Pass through It's called ovr pass through layer. Sorry. You could also just type ovr pass through layer Pass through layer. Here it is ovr pass through layer We're going to add that in And we need to change one thing about it right now. It's set to overlay Which means it will show us the pass through over everything else we are doing Which means we won't see anything. We won't see the table. We won't see the cube Nothing will just see the pass through instead. We're going to switch it to underlay, which means it's behind everything However There's a notable point with that which is it's behind everything. It's even behind the skybox So when you look out here, you see how there's a horizon This is a skybox around us We need to get rid of it if we want to see pass through to get rid of the skybox. You're going to go to window rendering Lighting and you should see something like this pop up Then you're going to go to environment And you're going to go to skybox material You're going to click the little circle next to it and we're just going to type none And then double click on that and boom no more skybox What at this point right now? We will have working video pass through going on in the oculus quest However You will not be able to see it for the duration of this demo because it's only visible inside of the headset That's just kind of a quirk with how this works There's no way to show other people the pass through The only way to do it is to download a bunch of software to see the exact screen view It's not worth it. Just trust me when you're going to see darkness. It's going to just look pure pitch black That in fact I can see my room around me Anyway On to the next step, right? There were two steps. I wanted to hit here. We wanted to Uh turn on pass through and we wanted to make it so that we can interact with the cube So the way that interactions work in ovr tool kit kind of thing is you need to add interactors to Your hands or your controllers and then You need to add interactable components to the objects that you want to interact with So to do that We're going to first start by adding these interactor components And these are all done underneath this object input ovr. So in here we have Our controllers and our hands today. We're really going to focus on our hands. So let's start with our hands here We're going to go to our left hand and our right hand and we're going to open up hand interactors and hand interactors on both of them So we can see here that these currently have a poke interactor in them But we want to grab so as you might assume We're going to type in something very similar. We're going to type in hand grab Interactor I believe here it is right here. So you hand grab interactor dot prefab This is the first half of being able to grab things. We're going to take it Drag it and drop it onto the hand interactors object. That's inside Of both the left and the right hand And We are going to now have to set it up. What I mean by that is if we take a look over here There's this script called hand ref that's on it's on both of them And it's looking for which hand we have placed this in So this one's on the right hand. I'm going to drag the right hand object into that slot Same over here on hand grab interactor in the left hand. I'm going to drag the left hand into that slot Okay, there's still another step In the hand interactors object There's actually a list of interactor drivers if we click here. We can see the list only has one element in it The poke interactor We need to add the grab interactor to it. So I'm going to drag the grab interactor over I'm going to drop it right on the word interactor drivers and you'll see it'll add it straight into our list Do not drop it In here drop it right on the word trust me You will cause a bug if you drop it on here because we're going to need this poke interactor later Okay, we do the same You can do the same thing over here Just grab the hand grab interactor Drop it on the word interactor drivers and boom. There it is at this point. Our hands are able to grab things However Our cube isn't able to be grabbed We need to make it grab a bowl. So we'll move on to that. We're clicking on the cube You know, I'm going to change these right now. It's scaling. I just I just went up here and change it to movement because I'm always afraid when the scaling ones are up. Okay In our cube, we want to add the grabable component just type grabable And uh, one thing I like to have on in here is transfer hand on second grab Just makes things a lot nicer It's kind of it's exactly what it says if you had the object in your hand and you grabbed It while you're holding it in the other hand, it will switch it to the hand that you're grabbing it with from the hand it was in It's like passing an object between both of your hands. It's like how it works in the real world, right? so, um What we next need to add though is Another component we need to Give it the option to grab it with our hand It's really weird how it works like this You have to add a grabable and then you have to add a separate Script depending on if you want to be able to pick it up with your hand or with your controller today We're really going to focus on hands. So I'm Going to go to add component. I'm going to type hand grab Interactable Boom here's hand grab interactable. You can see it's got a lot of different stuff in it It should automatically take in The transform of the cube the rigid body of the cube the grabable of the cube If any of these fields here are empty you got to make sure you fill them in with the object that is attached to Okay Now there's also this field here called physics grabable And you want if your object has physics, which our cube does You need to have a physics grabable as well as a normal grabable as well as a hand grab Interactable script in order for it to work. I know it's complicated Just follow and it'll work for you. So We're going to add a physics grabable Just go to add component Physics grabable. There it is click it. It's in it's good. It's good how it is Maybe you don't want it to scale mass with size. I like scale mass with size That seems to make the most sense to me. It works realistically for me But what we do want to change is Hand grabbing this is the whole thing about like recording Points on it and gestures and poses. We're not really concerned about that today I want us to be able to pick up these blocks like they are real objects And you can naturally kind of pinch on them and pick them up from any part along the object So to do that, we got to turn off snap right here. Turn it to none What this means is when we grab the cube, it's going to grab it from exactly where we grab it It's not going to snap the cube To have us grabbing onto its special grab point, which by default is the middle of the cube So with snap type none, we can grab the corner and it will let us just kind of grab it from the corner. Super nice Okay Now we have passed through and we have Hand grabbing abilities So Let's try it out All right, same thing as last time plug in your quest And whenever you plug in a quest you always have to go through that same allow dialogue over and over again So maybe you leave your quest plugged in in order to avoid that I just like to unplug mine personally so All right, it's been it's it's now enabled. It's connected I'm going to go to file build settings. Do not click build and run. Don't do it Just click build settings And we're going to refresh again as I said, we got to always make sure that our quest is connected before we build and run going to build and run gonna Save exactly the same thing we're going to replace it because we have a new version And boom it's going to build you're going to see now it's going to go so much faster It's going to go so much faster It still could take a little bit. So I will jump to the end So yeah, there we go. That took almost one minute exactly and uh, it took 51 seconds. You can see at the bottom Okay, I'm going to test it out now I'm gonna unplug my quest I'm gonna Go over so I can cast it so you can see what I'm seeing And let me show you what's up. So once again, it'll always show you the made with unity thing at the start and Mine currently has glitched out a little bit. This this tends to happen when you first install the app um with hand tracking It's currently not showing my hands as tracked. So all I'm going to do as I've said before is press The power button once and then press it again and uh Now everything's loaded. It's time for me to cast real quick so that you all can see what I am doing So I'm going to cast to my computer and Here we go Fantastic You all can see nice and clear Full screen it for you. Okay one thing you may notice right now Remember when I mentioned when we're do when we're using pass through you're not going to see That I can see the pass through so right now I can see my room through The headset and I can see this shelf and these cubes Overlaid inside of my room So Here's the hands And what do you know if we go over to grab the cube we can pick it up drop it We can't really throw it because uh currently There's no velocity tracking systems built into the grabbing mechanisms that Meta provides but that may change in the future All right, so that that's what we have right now I think it's time to add in distance Grab or distance selection so that we can try out different ways of interacting So let's get to it back to unity we go All right So we're back in here again And The final so that the net Alrighty, we're back in unity again And what we want to do now is We're going to build some means of interacting at a distance So that uses something called a ray cast mechanism To get our ray for our hands We're going to go Right over here open these all back up like we had earlier, you know back to the interactors and all of that We need to add a new interactor to our hands this one I'm gonna be able to guess it it's ray interactor Actually, I believe it's called hand ray interactor. There it is If you were using controllers, you would be doing the same thing but you would be dropping on The controller ray interactor. They're both very different I'm gonna add the hand ray interactor to our hand interactors of both hands And I'm going to Do the same exact thing we did before so I'm gonna do it kind of fast now because we've done it once Again, I drag the respective hand into the hand ref for each of them And I go into the interactors list and I drag It onto the words so that it gets added to the list I'm gonna do the same thing again over here Okay, cool hand ray interactors They're set up. We're good to go We don't have anything to interact with with our rays so We need to make some objects that are ray interactable And now I found through testing that Currently at least Making something ray interactable and grabbable Can cause some issues, but if you're doing them two distinct objects, then it works pretty well So what we're gonna do, I think let's let's use some spheres To select to do that. I'm just gonna go I'm gonna go to game object 3d object sphere And wow that is very far in the distance Let's reset it Now it is right here. Let's move this sphere over here You know, it's a nice big Spheres will make it easy for us to select them. So I definitely place them a bit of a distance behind your Table or your shelf Okay, I'm gonna make We'll start with one sphere that's what I'm putting on the left and then we're gonna duplicate it into a second sphere Okay, so Let's do that Our first sphere we got to set up The functionality we're going for here is we want our spheres to change the cube colors To whatever the color of the sphere is that we're using and we're gonna do this without any scripts Which means it's gonna be a little bit clunky so Let's get to it. All we got to do Our cube is currently blue. So if we were to change it to another color, perhaps that would be gold Okay, that seems fine But our sphere isn't doing anything right now, right? It's you can look at the components. It's just a sphere We need to make it so we can interact with this sphere with our ray So you may have guessed how to interact with a ray is we type ray Interactable Add that in And you'll see it has a spot for a collider here. Don't forget to add this I have forgotten so many times We need a collider or a surface so that it knows When we are pointing at it or not So I'm going to grab the sphere collider here and drag it and drop it into collider down here Okay Now we need to make it so that we can actually Do things when we interact and that means adding this other script called unity event Or sorry interactable. It's called interactable unity event wrapper Yeah, I know it's it's it's a long it's a long Name if you start typing it, it'll show up Okay, so here's our interactable unity event wrapper, but it needs to know what interactable it's viewing See here's as interactable view. We're going to drag ray interactable into this box So that it knows it's looking for our ray interactable to see what to do Now all we really care about here is when you select the the sphere. We don't care about when you Release the sphere. We have things for if you're hovering over it unhovering blah blah blah blah All we're concerned about today is when you click on the sphere. We want the cubes to change color to match the sphere Now you may have heard me slip a second right there. I said cubes, right? I think we should make more than one cube so that we can have some fun Using these so I'm gonna take the cube And I'm gonna duplicate it You can make as many cubes as you'd like, but be aware This step that we are doing right now Will become harder the more cubes you choose to build So I'm gonna go with three If you stick with one it's gonna be really easy, but with three it's only three times the work But it's not that much work. So anyway, I'm gonna duplicate this cube again Get our cube two. I'm keeping them organized over here so I can keep them together and drag cube two over here, this looks good like a good position for our cube two and Yeah, there are our cubes Our cubes are good to go but our sphere is not so Each of these cubes they're an exact copy, right? So they all are grabbable. They're all everything What they need is to be changed color when we click the sphere. So if we go to the sphere When we select the sphere, which is when we click on the sphere It will do whatever happens in when select. So I'm gonna click the plus button I'm gonna click the plus button three times in fact so we have Three of these boxes and I'm gonna draw drag cube Into each of them I'm gonna drag each of the cubes into each of them. Sorry Cube cube one cube two Right and now under each of these we want to change its material That's the thing we're trying to do. So you go to mesh renderer material material Don't choose material shared material. I know it can be easy to mix them up material material And I'm gonna do the same thing over and over again mesh renderer material material mesh renderer material material And now we want to pick the material we want to change it to well This is the gold sphere So we're gonna put gold as a material that we want to change these cubes to Okay, so we've done it all that we need to do now is uh Just a little a little something I think You may notice that if we right now if we change everything to gold there's no way to get back to blue So I'm gonna duplicate this sphere Now that we've made all of our stuff on it. I'm gonna duplicate this sphere boom second sphere And this new sphere I'm gonna color blue And importantly I'm gonna go into its script and I'm gonna replace all of these colors with blue So that now when we click on it it sets each of them to blue Okay Now I don't want us to have to keep building this over and over again I think there's one more fun interaction we can add in here to just you know Finish this off kind of put put the the ribbon on it so to speak and that is A big red button Like Alex, where did your what do you you expect me to build a big red button? Just like out of nowhere No Just type it in big red Button it comes built in In the oculus integration And we want to pick this one You'll see one of them is a prefab and one of them is an fbx or just a 3d model We want the prefab Take that prefab and just drop it Somewhere on your shelf preferably somewhere you can reach So I'll put it here There's so much garbage in the way. I can't tell where it is. I'm gonna toggle off the gizmos by clicking up here So that I can see past it. Let's see How is this hovering above it's hovering above the the surface. Let's drag it down a little bit That looks good to me Cool. Cool. Cool. All right. So now we have our big red button What does it do? Why is it here? Uh, well We have a big red button. Remember when I mentioned Uh that we were saving red for something Well, that's now so first thing We don't want to overwrite what a big red button is. So I'm just gonna Prefab unpack completely on the big red button to unwrap it And now I'm going to go into here. I'm going to go to button I know there's a whole bunch of different button things get the one that's just button And you'll see look it has an interactable unity event wrapper just like what we did with our spheres And I'm gonna add A couple of when selects so three more when selected things here And we're gonna do exactly what we did earlier this time though Yeah, let's grab a our cube one or cube normal cube one cube two And we want to mesh render material mesh render material mesh render material And the material we choose is red. That's why I made you make red Because when we press our big red button, I want everything to turn red. I think it's fun It makes more sense when you press the red button and uh, you'll notice that the Button All right. Yeah, it has a poke interactable on it already And you'll notice that our hands have poke interactors on them already Which means It's gonna naturally automatically let us poke it. We didn't have to build the interactors and stuff in That's why we don't have to do it. It already has it there for us But you know what time it is it's time to test it out. We've built the entire thing Let's play with our new creation. So I'm gonna save I'm gonna go to file build settings I'm gonna plug my headset back in I'm gonna take a look in the headset and I'm gonna allow Connections to it so that you can write things to the headset. I'm gonna Go over here and refresh to make sure that it recognizes that it's connected and I'm gonna build and run Send it and here we go. And I'll catch you all on the other side of this Uh setup Okay, so here we are back in the app that we just built and let's take it for a spin. So Let's see my hands you once again, you can't see my room when you're doing this on your own one You'll be able to see your room through it I'm gonna go and uh, first let's pick up some cubes. You can stack our cubes You do whatever we want. We don't have to just pinch script them either. Oh, there goes one on the floor Let's grab it. You can grab it With your palm you can grab it with your fingertips. You can grab it with like Whatever fingertips you want, whatever fingertips you choose Okay, so now remember the new things we added About selecting see how as I move my hand around The ray gets cast onto these spheres Okay, well if I choose if I click by pinching In fact, you'll see that like it turns into almost like a Turns from a tear drop shape into kind of like a line So when we when we pinch when we're pointing at something and select it, I'm gonna pinch now on the yellow sphere Or the gold sphere and look Our cubes are now gold You make a fun stack out of them so we can see it cleanly And if we want to go back to blue Click on blue Now But you're all waiting to hit the big red button. Let's give it a go It even has sounds that come built in I do love hitting the big red button So you'll notice that it uses poke interaction not like a touch thing and since it uses poke interaction It only works with our fingertip It only works with our index fingertip because that's what you poke with apparently It's what they decided at least that you poke with But yeah Here's our creation And that is the simple app that we're making in this in this workshop. Yeah So let's Jump back out so I can give some closing Notes to help you all out Hey, so That was the contents of the workshop Once again, we covered how to put Apps onto your quest How to give apps pass through capabilities How to interact with things using your hands How to make it so that you can interact with objects in general how you can do distance interaction near interaction poke grab and ray interactions and the quest And yeah at this point you should have an application on your quest That is The app that you have just made And yeah Hope you have some fun with it. Hope it helps to be a baseline for maybe future work that you're going to do So, yeah, thank you for attending my workshop. I hope you have a great day. See you