Nov 24 • Ashray Pai

Setting up Oculus Hand Interaction

The latest Oculus SDK comes with a hand tracking feature that enables the use of hands as an input method on Oculus Quest devices. By using simple hand gestures, such as pinch, poke, as well as pinch and hold, we can integrate hand tracking such that users can select, click, scroll, drag and drop, return, or exit in our app. We can also use these simple gestures to interact with 3D models.

Using the physical hand to interact with VR objects opens the door for many possibilities. You can create apps that are aimed at teaching communication with sign language or create a game like chess or rock paper scissors that feels more life-like.
YouTube

Want to watch the video instead?

1. Prerequisites

You should also have your Unity project set up for Oculus Quest. If you are unsure how that’s done, you can read through the instruction provided in sections 4 and 5 of the Passthrough blog. In addition, you can download the base repository from here.

2. Adding Hand Interactor

Once your Unity project is set up with all the required project settings and packages downloaded. We can start developing for hand interaction.

So let's begin by adding prefabs to the scene which will collectively enable us to use our hands for interaction.

  • In the Hierarchy window, delete the Main Camera.
  • Search for OculusInteractionSampleRig in the Project window → drag and drop the prefab into the Hierarchy window.
  • Open the OculusInteractionSampleRig prefab to view its children → open the InputOVR prefab → open the LeftHand and RightHand GameObjects.
  • Search for HandGrabInteractor in the Project window → drag and drop the prefab into the HandInteractor GameObject of both LeftHand and RightHand GameObjects → rename them as HandGrabInteractorLeft and HandGrabInteractorRight respectively.
  • Select the HandGrabInteractorLeft prefab, drag and drop the LeftHand GameObject into the Hand parameter of the Hand Ref component. Similarly, drag and drop the RightHand GameObject into the Hand parameter of the Hand Ref component of the HandGrabInteractorRight prefab.
  • Select the HandInteractor of the LeftHand GameObject and drag and drop the HandGrabInteractorLeft prefab into the Interactor Drivers parameter of the Interactor Driver Group component. Similarly, select the HandInteractor of the RightHand GameObject and drag and drop the HandGrabInteractorRight prefab into the Interactor Drivers parameter of the Interactor Driver Group component.
💡Note: The Support Grab Types parameter of the Hand Grab Interactor component attached to the HandGrabInteractorLeft prefab and HandGrabInteractorRight prefab determines how the GameObject can be interacted with. Basically, there are of two types, one is Pinch and the other is Palm. When Pinch is selected you can grab the object with the pinching action of your thumb with any of the other fingers. When the palm is selected you can grab the object by making a fist.
With that, we’ll now be able to use hands in our scene. Hit the play button and test the hand interaction. 
Yes, you will not be able to do anything apart from seeing your hands in VR, so in the next section, we’ll add an interactable GameObejct.

3. Adding Interactable

Let’s begin with one hand interaction. As the name suggests, this is a type of interaction where we can use only one hand at a time to interact with an object. This can be used in cases where we want to move an object from one place to another i.e simply drag and drop an object from one place to another. To add such a feature:

  • Right-click on the Hierarchy window → 3D Objects → Cube.
  • Adjust the scale and transform such as you have a handy-sized cube in front of the camera.
  • Select the 3D Cable GameObject and the following components to i 
    • The Rigidbody component → uncheck the Gravity parameter.
    • The Grabbable component
    • The Hand Grab Interactable component
  • Select the Cube GameObject → under the Grabbable component check the box ✅ for the parameter Transfer Hand On Second Grab, so that the object gets transferred from the current hand to the new one.
  • Also, set the Snap Type parameter as None of the Hand Grab Interactable component. This will ensure that the cube does not snap into the hand, instead, it will remain at the point of grab.
Now you can hit play and test the scene. That was simple right?
While testing it you might have observed that you can interact with the object with just one hand. That’s because by default One Grab Free Transformer component gets added. So how do we add two hand interaction? That’s what we will show you in the next part! 

Conclusion 

With that, we have learnt to add hand interaction to our scene and interact with a 3D game object. You can further this project and create amazing experience. Hand interaction provides a new sense of presence and can help deliver more natural interactions. In the next part, we’ll learn about two hand interaction, enabling physics and rotational transforms. 

Thanks for reading this blog post 🧡

If you've enjoyed the insights shared here, why not spread the word? Share the post with your friends and colleagues who might also find it valuable.
Your support means the world to us and helps us create more content you'll love.