If you've ever wondered how to integrate AR content into your environment, by the end of this blog, you'll be equipped to create spatial maps, capturing and mapping your environment; develop AR experiences, implementing features like AR multi-user interactions, indoor navigation, and overlaying BIM models to visualize hidden structures; and explore WebXR, extending your AR projects to web-based platforms.
Want to watch the video instead?
1. Map Your Room
First, head to the Immersal Developer Portal and create a developers account. (Later, you will be using the same credentials to login inside the Immersal App and also while developing inside Unity.)
Now to create a spatial map of your environment, download the Immersive Mapper App from the Google Play Store or App Store.
Once the app is installed, log in and create a new map by pressing the + button. On the top right corner you will be able to see two modes: the manual mode and the real time mode. So what’s the difference?
- In the Manual Mode, you will have to take the pictures of the target area by yourself from different positions and angles. Then, upload the images to the server for map construction, which will then create the point clouds for you.
- In the Real Time Mode, you can just press the record button and move around inside the target area. It will capture the images automatically and generate the point clouds in real time.
So now the question is which mode you choose?
Well, if you are looking for a higher quality map, or if you are looking to map complex areas, then you can choose manual mode as it provides better result. However, it requires more careful mapping
as it's easy to miss certain angles or transitions.
On the other hand, you can use real time mode for testing or for mapping small areas as it generates the point clouds on the fly, but it does not save the images or generate a mesh of your environment.
In this tutorial we’ll be using Manual Mode to create a spatial map.
How to Validate Map Quality
After the map is processed, you get an option to download it and test localization in the real world. A high-quality map localizes quickly and consistently. If localization fails, adjust your mapping process to include more angles and features.
- Click on View Your Maps and it will show you a list of all the maps that you have captured.
- And then if you click on the three dots, it gives you options to view the sparse poly, the dense polygon, or the GLB.
- Now, if it gives you the option just to view the sparse poly, then it means that your map quality is not so good and it does not have enough number of feature points to generate a dense poly or a GLB.
- There could also be cases where you are able to view the sparse poly and the dense poly, but when you try to view the GLB, it says “Mesh failed”. Once again, it means that the quality of your map is not so good that it can generate a mesh.
💡 If you use real time mode to capture your environment, then make sure to upload the map to the cloud. Also, realtime mode does not generate a mesh so you’ll never get a .glb file.
4. Integrating the Immersal SDK and Setting Up VPS Features
4.1 Add the Immersal SDK Prefab
With our Unity project and scene set up fro AR, it's time to integrate the Immersal SDK and configure the VPS (Visual Positioning System) features to place virtual content.
- In your project window,set the search filter to
"All"and search for"Immersal". - Select the Immersal SDK prefab and add it to your scene. This prefab includes AR Foundation support, which configures the device and retrieves necessary camera data.
4.2 Understand the Components of Immersal SDK Prefab
- Localization Methods: The prefab includes two localization methods that define how localization should occur.
- Localizer Component: Manages and configures the localization methods.
- Scene Updater Component: Adjusts the position from the localizer and updates the scene accordingly.
- Spatializer Component: Interprets data such as device status and localization results, constructing a tracking status to describe positioning quality.
- Immersal Session Component: Connects all components, managing a continuous loop to process data based on internal logic and configuration.
4.3 Set Up XR Space
- Create an empty GameObject named "XRSpace" and add the XR Space component.
- Add Data Processors:
- Create a child GameObject named "Pose Filter" and add the Pose Filter component.
- Create another child GameObject named "Pose Smoother" and add the Pose Smoother component.
- Add both components to the XR space's data processor list.
4.4 Download the Spatial Map
- Create an empty GameObject as a child of XR space, name it “XR Map” and add the XR Map component.
- From the toolbar, select ImmersalSDK and click on Login. Use your Immersal credential and upon successful login, you'll see a token.
- Now to download the spatial map, go to the Immersal Developer Portal and copy the map ID from the Immersal developer portal.
- Back in Unity, paste it in the map ID parameter of the XR Map component → check the box for Mapfile and Visualization and click
Download
4.5 Configuring Spatial Map
- Configure Localization: Decide if you want the localization method to be device-localized or server-localized. Similarly, choose if the map data should be embedded or downloaded.
- Render Settings: Select the XR map's child object with the XR map visualizer, choose the render mode (Editor or Runtime), and adjust point color and size as needed.
4.5 Testing Localization
Before we start placing AR Content, let’ first build and test this on to our mobile device to make sure it’s able to localize.
- Make sure to have enabled developer mode on your device and connect it to your computer.
- Navigate to
File→Build Settings, add the open scene, select your device from the dropdown, and click"Build and Run"
After launching the app, ensure the spatial map localizes correctly in your environment. The point cloud should align seamlessly with physical surroundings.
5. Adding AR Content
Now that our app can localize the spatial map, it's time to add virtual content to your AR scene.
5.1 Import Core Samples
- Navigate to
Window→Package Manager. - Find the
Immersal SDKin the list. - Go to the
Samplestab and importthe core samples.
5.2 Place Virtual Objects:
- Create an empty GameObject under XR Space and name it Content.
- Inside the imported samples, navigate to
Core Samples→Prefabs→Content Placement. - Drag and drop the AR Diamond prefab into the "Content" GameObject you created.
- Move and align the diamond with your spatial map. For example, we have place one diamond above a camera and another is placed another near a laptop.
- If you don't want to see the point clouds in your application, navigate to the XR map, select its child object, and set the render mode to
Editor Only.
5.3 Testing Localization
- Ensure all changes in your scene are saved.
- Navigate to
File→Build Settings, add the open scene, and build the application to your device.
After launching the app, it takes a couple of seconds to localize but once that's done, the virtual content should align seamlessly with physical surroundings.
7.3 Set Up in Unity
- Navigate to XR space and delete the previously added diamonds.
- Create UI Button:
- In the Canvas, create a UI Button, adjust its size and position, and rename the text to "Spawn Anchor."
- Attach the
InstantiateSpatialAnchorscript to the button and configure references for the anchor prefab, camera transform, and XR space parameters. - Add an item to the On-click event, select the script, and choose
Instantiate Anchorfrom the dropdown.
7.4 Testing
- Ensure all changes to the scene are saved.
- Navigate to
File→Build Settings, add the open scene, and build the application to your device.
Test the application by spawning anchors → quitting the application → and when reopened, after successful localization, the virtual anchors should appear in the previously set positions, demonstrating persistence across sessions.
