Dec 28 • Ashray Pai

Creating a Local AR Multi-User Experience with Photon Fusion and Immersal SDK

In today’s tutorial, we’re diving into the creation of a local AR multi-user experience using Photon Fusion and the Immersal SDK.

If you’re not familiar with Immersal, they’re a company focused on technologies like spatial mapping and visual positioning systems (VPS), which help developers like us create incredible AR experiences.
YouTube

Want to watch the video instead?

Prerequisites

Before we get started, there are a few prerequisites you need to meet. Don’t worry, I’ll give you an overview of each one, and if you need more detailed instructions, you can check out our previous blogs here.

  1. Create a Developer Account

    Start by creating a developer account with Immersal. 

  2. Download the Immersal Mapper App from Playstore/AppStore

    Log in using your developer credentials and download the Immers Mapper app. This will be essential for mapping your environment.

  3. Map Your Environment

    Once you’ve downloaded the app, you’ll need to map your environment. For this tutorial, I used the manual mode to carefully map the room where we’ll be setting up our AR experience.

  4. Set Up a Unity Project with Immersal SDK

    Finally, you’ll need to set up your Unity project with the Immersal SDK. Here’s a quick rundown on how to do that:

    • Go to Windows > Package Manager.
    • Click the plus symbol (+) and select Add package from Git URL.
    • Paste the following Git URL and click Add: https://github.com/immersal/imdk-unity

    Once that’s done, navigate to File > Build Settings, switch the platform to Android, and use the Project Validation Tool to fix any issues.

Setting Up the Scene

With the prerequisites out of the way, let’s start setting up our scene in Unity.

  1. Delete the Default Main Camera

    In your Unity hierarchy, select the main camera and delete it. We’ll be using a different setup for the AR experience.

  2. Add XR Session and Origin

    Right-click in the hierarchy, navigate to XR, and select AR Session and XR Origin. Set the position and rotation of both to zero.

  3. Add Immersal SDK Prefab

    In the project folder, search for Immersal SDK and add the appropriate prefab to your scene.

  4. Log in to Immersal SDK

    To retrieve the developer token, log in within the Immersal SDK using your developer credentials. Once logged in, the token will be visible in the settings.

  5. Create XR Space and Data Processors

    Create an empty GameObject named XR Space, and add the XR Space component. Then, add two Data Processors and reference them accordingly.

  6. Add Post Filter and Smoother

    As children of XR Space, create two empty GameObjects named Post Filter and Post Smoother. Add their respective components.

  7. Download the Spatial Map

    Under XR Space, create an empty GameObject named XR Map and add the XR Map component.To download the map:

    • Visit the developer portal and copy the map ID.
    • Paste the map ID into Unity and check the boxes for Map File and Visualization. Click Download to get the point cloud of the mapped environment.

Downloading Required Assets and Setting Up Photon Fusion

Next, we’ll need some additional assets and packages to complete our setup.

  1. Download and Import Assets

    You’ll need the Dragon Boss Monster, Joystick Pack, and Photon Fusion. You can get all of these from  the Unity's Asset store.

  2. Photon Fusion Setup

    Log into your Photon account, create a new app, and select Photon Fusion as the SDK. Copy the App ID and paste it into Unity.

  3. Download Immersive Multi-User Package

    Download this Immersive Multi-User Package . This package contains prefabs like Debug UI, Joystick Canvas, and Multiuser UI. Add these to your XR Space.


Configuring the Multiplayer UI and Scripts

Now that the assets are imported, we need to configure the multiplayer UI and scripts.

  1. Debug UI

    Although optional, the Debug UI is helpful for debugging, as it can show errors or confirm multiplayer functionality. Open the Debug UI prefab and disable the Debug Scroll View for now.

  2. Joystick Canvas

    In the Joystick Canvas, you’ll find the horizontal and vertical fixed joysticks, which allow for player control. These are part of the Joystick Pack we downloaded earlier.

  3. Multi-User UI

    The Multiuser UI prefab allows users to create or join a room using an input field for the room code and two buttons—Create Room and Join Room.

  4. Scripts Overview

    Let’s go over the key scripts involved:

    • Network Manager: Handles room creation, player connection, and player ID logging. It also invokes the OnPlayerJoin event when a player joins.
    • Connection Manager: Simple script to create or join rooms based on input from the UI.
    • Network Dragon Spawner: Responsible for spawning dragons for each player in the session.
    • Dragon Controller: Manages the movement of dragons using joystick inputs.
    • Game Manager: Coordinates the downloading of the map and spawning of dragons when players join.
    • Spawn Behavior: Ensures networked objects are placed correctly under the XR Space when spawned.

Setting Up Dragons

Now, let’s prepare the dragons for multiplayer.


Prefab Setup

  1. Navigate to the Dragon Assets Folder: Select three dragons from the assets.
  2. Fix Materials: Convert the materials to URP by selecting all materials and using Edit > Rendering > Materials > Convert Selected Built-in to URP.