Oculus Passthrough: Layer Customization

Previously we saw how to set up a Unity project to work with Oculus Passthrough API and how we can create an augmented interactable object. If you haven’t checked out the first two parts of this series, we highly recommend checking it out as the API will not work without the right settings and the knowledge from previous parts are required here as well.

In this blog post, we’ll learn how to customize the passthrough layer. There are various ways to customize passthrough. It could be through styling or composite layering or by surface-projected passthrough implementation. Our focus, for now, is going to be on styling, where we will write a script that will allow us to change the brightness, contrast, alpha and posterize values of the passthrough.

Let's start by creating a canvas with four sliders. The values from these sliders will later be used to change the brightness, contrast, alpha and posterize values of the passthrough.

• Right-click on the Hierarchy window and select UI → Slider.
• Select the Canvas GameObject and rename it as PassthroughStyler → change the Render Mode to World Space → adjust its position and scale, I have considered the position as (0,0,0) and the scale as (0.01, 0.01, 0.01) → for the EventCamera, drag and drop the CenterEyeAnchor GameObject from the OVRCameraRig GameObject into that field.

• Select the Slider GameObject → adjust its position and scale.
• Duplicate the Slider GameObject thrice and name them as BrightnessSlider, ContrastSlider, AlphaSlider and PosterizeSlider → adjust its position such that there is a sufficient gap in between them.

• Right-click on PassthroughStyler GameObject and select UI → select Text - TextMeshPro → click on Import-TextMeshPro Essentials, that popup will be seen if you don't have it imported already, as seen in the gif→ reduce the Font Size and Center align the text.
• Duplicate the Text(TMP) GameObject thrice → adjust their position from top to bottom and name them as Brightness, Contrast, Alpha and Posterize respectively.
• Copy and paste the same names into the respective Text Input field.

• Select the PassthroughStyler GameObject and move it in front of the camera.
• Disable the Cube GameObject

At this current state, we’ll be just able to view the UI canvas but not interact with it. To make it interactable we need to add a few components and helper prefabs to our scene.

• From the Hierarchy window delete the EventSystem GameObject.
• Select the OVRCameraRig GameObject and add the OVR Physics Raycaster component.

• In the Project window, search for UIHelpers → drag and drop the UIHelpersprefab into the Hierarchy window.
• Open the UIHelpersprefab’s children→ select the LaserPointer GameObject → enable the LineRenderer component.

• Select the PassthroughStyler GameObject → remove the Graphic Raycaster component → add the OVR Raycaster component.

• With this, we should be able to use the UI with the controller. But when you test it, you’ll not be able to interact with the UI. The reason is that the Canvas of the individual text is big and it's placed in front of the sliders. So to rectify that by:
  • Right-click on PassthroughStyler GameObject and select UI → select Canvas.
  • Name it as TextCanvas.
  • Select the four GameObject Brightness, Contrast, Alpha and Posterize, and move them into the TextCanvas GameObject.

The UI sliders have an event called On Value Changed which gets called when the value of the slider changes. The event can send the current value as a float type dynamic argument. So by making use of that feature we can change the Passthrough Layer’s fields dynamically using public methods.

When the OnBrightnessChanged method is invoked by the On Value Changed event of the slider, it passes the slider values as float parameters to this method. Which then sets the Passthrough filter’s brightness to that value.

Other methods OnContrastChanged, OnAlphaChangedwork and OnPosterizeChanged work in the same way as well. The only then we need to be care full about is to assign the right method to the right slider

In this section, we’ll see how to set up the OVRPassthroughLayer component. We’ll also see how to assign the StylingPassthrough component’s methods to the slider's events.

• In order to customize the Passthrough Layer, we need to enable the Layer’s control map. To do that, click on OVRCameraRig GameObject → under the OVRPassthroughLayer component, select the Color Map as Controls.

• Add the StylingPassthrough component to the PassthroughStyler GameObject by dragging and dropping the script on the GameObject.
• To add the parameter for Passthrough Layer , drag and drop the OVRCamraRig GameObject into the field. As the OVRPassthroughLayer is attached to the OVRCamraRig, it will automatically assign that to the field.
• Select all the four sliders and click on the plus ➕ button to add to an action to the event list → drag and drop the PassthroughStyler GameObject into the event field.
• Click on BrightnessSlider GameObject→ choose the function StylingPassthrough → under Dynamic float, select OnBrightChanged method*.* Similarly*, s*elect one slider at a time and assign the respective function under Dynamic float for the rest of them.