Oculus Passthrough: Part 3

Previously we saw how to set up a Unity project to work with Oculus Passthrough API and how we can create an augmented interactable object. If you haven’t checked out the first two parts of this series, we highly recommend checking it out as the API will not work without the right settings and the knowledge from previous parts is required here as well.

In this blog post, we’ll learn how to customize the passthrough layer. There are various ways to customize passthrough. It could be through styling or composite layering or by surface-projected passthrough implementation. Our focus, for now, is going to be on styling, where we will write a script that will allow us to change the brightness, contrast, alpha and posterize values of the passthrough.

1. Creating the UI Canvas

Let's start by creating a canvas with four sliders. The values from these sliders will later be used to change the brightness, contrast, alpha and posterize values of the passthrough.

• Right-click on the Hierarchy window and select UI → Slider.
• Select the Canvas GameObject and rename it as PassthroughStyler → change the Render Mode to World Space → adjust its position and scale, I have considered the position as (0,0,0) and the scale as (0.01, 0.01, 0.01) → for the EventCamera, drag and drop the CenterEyeAnchor GameObject from the OVRCameraRig GameObject into that field.

• Select the Slider GameObject → adjust its position and scale.
• Duplicate the Slider GameObject thrice and name them as BrightnessSlider, ContrastSlider, AlphaSlider and PosterizeSlider → adjust their position such that there is a sufficient gap in between them.

• Right-click on PassthroughStyler GameObject and select UI → select Text - TextMeshPro → click on Import-TextMeshPro Essentials, that popup will be seen if you don't have it imported already, as seen in the gif→ reduce the Font Size and Center align the text.
• Duplicate the Text(TMP) GameObject thrice → adjust their position from top to bottom and name them as Brightness, Contrast, Alpha and Posterize respectively.
• Copy and paste the same names into the respective Text Input field.

• Select the PassthroughStyler GameObject and move it in front of the camera.
• Disable the Cube GameObject.

The result of what we have created is the UI Canvas. Now, you hit play and view the canvas.

Note: While in play mode make any position or size adjustment if needed → copy the values → exit the play mode and paste the values back into the component.

2. Making the UI interactable

At this current state, we’ll be just able to view the UI canvas but not interact with it. To make it interactable we need to add a few components and helper prefabs to our scene.

• From the Hierarchy window delete the EventSystem GameObject.
• Select the OVRCameraRig GameObject and add the OVR Physics Raycaster component.

• In the Project window, search for UIHelpers → drag and drop the UIHelpersprefab into the Hierarchy window.
• Open the UIHelpersprefab’s children→ select the LaserPointer GameObject → enable the LineRenderer component.

• Select the PassthroughStyler GameObject → remove the Graphic Raycaster component → add the OVR Raycaster component.

• With this, we should be able to use the UI with the controller. But when you test it, you’ll not be able to interact with the UI. The reason is that the Canvas of the individual text is big and it's placed in front of the sliders. So to rectify that by:
• Right-click on PassthroughStyler GameObject and select UI → select Canvas.
• Name it as TextCanvas.
• Select the four GameObject Brightness, Contrast, Alpha and Posterize, and move them into the TextCanvas GameObject.

You can now hit play and test the UI interaction.

3. Scripting

Now that we have the slider UI ready, we can write a script that will take the slider values to adjust different parameters of the passthrough layer. Thus, we will be able to customize it.

3.1 Code

In the next section we have the breakdown of the code as well, so don't worry if you are not able to understand the code immediately.

Create a new C# script, name it as StylingPassthrough and copy the following code. The code will use the events from the slider to update the parameters of the passthrough layer.

using UnityEngine;

public class StylingPassthrough : MonoBehaviour
{
    [SerializeField] private OVRPassthroughLayer passthroughLayer;

    private float setBrightness = 0.0f;
    private float setContrast = 0.0f;
    private float setPosterize = 0.0f;
    
    private Color setEdgeColor = Color.white;

    private void Start()
    {
        setEdgeColor.a = 0.0f;

        passthroughLayer.colorMapEditorBrightness = setBrightness;
        passthroughLayer.colorMapEditorContrast = setContrast;
        passthroughLayer.edgeColor = setEdgeColor;
        passthroughLayer.colorMapEditorPosterize = setPosterize;
    }

    public void OnBrightnessChanged(float newValue)
    {
        setBrightness = newValue;
        passthroughLayer.colorMapEditorBrightness = setBrightness;
    }

    public void OnContrastChanged(float newValue)
    {
        setContrast = newValue;
        passthroughLayer.colorMapEditorContrast = setContrast;
    }

    public void OnAlphaChanged(float newValue)
    {
        setEdgeColor.a = newValue;
        passthroughLayer.edgeColor = setEdgeColor;
    }

    public void OnPosterizeChanged(float newValue)
    {
        setPosterize = newValue;
        passthroughLayer.colorMapEditorPosterize = setPosterize;
    }
    
}

3.2 Code Breakdown

Declaration

using UnityEngine;

public class StylingPassthrough : MonoBehaviour
{
    [SerializeField] private OVRPassthroughLayer passthroughLayer;

    private float setBrightness = 0.0f;
    private float setContrast = 0.0f;
    private float setPosterize = 0.0f;
    
    private Color setEdgeColor = Color.white;

Variable NameTypeUsepassthroughLayerfloatTo store the value between 0 and 1 from the slider. The value will determine the brightness of the passthrough layer.setBrightnessfloatTo store the value between 0 and 1 from the slider. The value will determine the brightness of the passthrough layer.setContrastfloatTo store the value between 0 and 1 from the slider. The value will determine the contrast of the passthrough layer.setPosterizefloatTo store the value between 0 and 1 from the slider. The value will determine the posterize of the passthrough layer.setEdgeColorColorTo store the alpha Color of the rendered edges.

Initialization

private void Start()
    {
        setEdgeColor.a = 0.0f;

        passthroughLayer.colorMapEditorBrightness = setBrightness;
        passthroughLayer.colorMapEditorContrast = setContrast;
        passthroughLayer.edgeColor = setEdgeColor;
        passthroughLayer.colorMapEditorPosterize = setPosterize;
    }

At the start when this script is enabled for the first time, we want the Passthrough Layer’s fields to be set to zero. In particular, we want to set the brightness, contrast, alpha value of edge color and posterize to zero. So using the properties of the Passthrough Layer the respective field is initialized to zero.

Methods

public void OnBrightnessChanged(float newValue)
    {
        setBrightness = newValue;
        passthroughLayer.colorMapEditorBrightness = setBrightness;
    }

    public void OnContrastChanged(float newValue)
    {
        setContrast = newValue;
        passthroughLayer.colorMapEditorContrast = setContrast;
    }

    public void OnAlphaChanged(float newValue)
    {
        setEdgeColor.a = newValue;
        passthroughLayer.edgeColor = setEdgeColor;
    }

    public void OnPosterizeChanged(float newValue)
    {
        setPosterize = newValue;
        passthroughLayer.colorMapEditorPosterize = setPosterize;
    }

The UI sliders have an event called On Value Changed which gets called when the value of the slider changes. The event can send the current value as a float type dynamic argument. So by making use of that feature we can change the Passthrough Layer’s fields dynamically using public methods.

When the OnBrightnessChanged method is invoked by the On Value Changed event of the slider, it passes the slider values as float parameters to this method. Which then sets the Passthrough filter’s brightness to that value.

Other methods OnContrastChanged, OnAlphaChangedwork and OnPosterizeChanged work in the same way as well. The only then we need to be care full about is to assign the right method to the right slider

Customizing the Passthrough Layer

In this section, we’ll see how to set up the OVRPassthroughLayer component. We’ll also see how to assign the StylingPassthrough component’s methods to the slider's events.

• To customize the Passthrough Layer, we need to enable the Layer’s control map. To do that, click on OVRCameraRig GameObject → under the OVRPassthroughLayer component, select the Color Map as Controls.

• Add the StylingPassthrough component to the PassthroughStyler GameObject by dragging and dropping the script on the GameObject.
• To add the parameter for Passthrough Layer, drag and drop the OVRCamraRig GameObject into the field. As the OVRPassthroughLayer is attached to the OVRCamraRig, it will automatically assign that to the field.
• Select all the four sliders and click on the plus ➕ button to add an action to the event list → drag and drop the PassthroughStyler GameObject into the event field.
• Click on BrightnessSlider GameObject→ choose the function StylingPassthrough → under Dynamic float, select the OnBrightChanged method.
  Similarly, select one slider at a time and assign the respective function under Dynamic float for the rest of them.
  

With that, we have finished setting up our scene for customizing the Passthrough during run time. You can hit play and test it.

Conclusion

The whole intent of this blog was to understand and learn about the passthrough filter.  When you play around with this scene you will be able to have a better understanding of the passthrough filter. What we saw above was just one part of this. There are other ways of using a passthrough filter as well as we mentioned earlier. You can find different types of example scenes showing the various implementation of passthrough in the Oculus Integration package. In the project folder navigate to Assets → Oculus → SampleFramework → Usage → Passthrough, to find the example scenes. To learn more you can check out this documentation.

‍