How to create cross device compatible WebXR experiences
Demonstrating how the WebXR API can be used to create cross device compatible experiences.
Intro
I’ve recently posted about what the buzzword ‘Metaverse’ means, as well as examples of what the underlying technology ‘WebXR’ is and what it is capable of.
What I didn’t address, was how to start developing these experiences from a practical development perspective.
I’m currently in the process of writing documentation for an open source library I’ve created called Wrapper.js, which aims to easily allow developers to spin up Full Stack Web3 based applications.
In the meantime, I wanted to share some key code snippets — that may be helpful to someone out there looking to develop these kinds of experiences.
Overview
In order to make the most of the information in this post, its important that you understand these two key points:
The Concept
The big difference between tradition XR and WebXR, is that WebXR based applications run in the browser.
Using the WebXR API, you can query what capability is of the device you’re using to view the experience and load the appropriate content so that it is able to load without issue.
This enables you to open the web application on an XR headset, desktop or mobile device and be able to interact with the content.
The Tech Stack
The libraries we will be using to implement this concept are:
- Three.js (implemented with React-Three-Fiber) for the creation of 3D assets.
- React-XR for leveraging the WebXR API (e.g VR Headset + controllers) & React-Three-Drei which has useful functions for creating WebGL content.
- React to componentise & standardise the logic — in this case we will use NextJS.
WebXR Code Examples
There are two examples to talk through in order to implement cross device compatible WebGL content, these are outlined below.
These are extracts from the WebXR template of the Wrapper.js repository.
The index file
In Next.js, this file would be located in pages/index.js — if you’re using another implementation of react, you can extract this into your index file.
Important lines to note here are:
- Line 5: where we import the XRScene Higher Order Component we will write to do the logic to load assets correctly.
- Line 16 and line 39: which indicate how you wrap your 3D assets (GLTF models / meshes / lights etc) with the XRScene component, demonstrating how to neatly contain and separate ‘web compatibility’ logic — from ‘WebGL content’.
import Head from 'next/head'
import dynamic from 'next/dynamic';
import React, { useRef, useState, Suspense, lazy, useEffect } from 'react'
const XRScene = dynamic(() => import("../components/XRScene"), { ssr: false });
const Shiba = lazy(() => import("../components/3dAssets/Shiba.js"), {ssr: false});
const Slide = lazy(() => import("../components/3dAssets/Slide.js"), {ssr: false});
const Dome = lazy(() => import("../components/3dAssets/Dome.js"), {ssr: false});
export default function Home() {
return (
<>
<Head>
<title>Wrapper.js Web XR Example</title>
</Head>
<XRScene>
<Shiba
name={'shiba'}
position={[1, -1.1, -3]}
rotation={[0,1,0]}
/>
<Dome
name={'breakdown'}
image={'space.jpg'}
admin={true}
/>
<Slide
name={'smile'}
image={'smile.jpeg'}
position={[-2, 1, 0]}
rotation={[0,-.5,0]}
width={10}
height={10}
/>
<ambientLight intensity={10} />
<spotLight position={[10, 10, 10]} angle={0.15} penumbra={1} />
<pointLight position={[-10, -10, -10]} />
<spotLight position={[10, 10, 10]} angle={15} penumbra={1} />
</XRScene>
</>
)
}
XRScene Higher Order Component
The XRScene component is written in a manner that you can pass other child components into it and render them based on specific logic (essentially acting like a parent wrapper that renders children componets, this is known as a Higher Order Component).
Important lines to note are between 38–45, where it is calling the webxr api through navigator.xr:
- If the WebXR api is not accessible, then the device is considered a normal website rendering device
- If ‘immersive-ar’ is possible on that device, then the scene will be rendered for Augmented Reality
- If ‘immersive-vr’ is possible on that device, then the scene will be rendered for Virtual Reality
Another important set of lines is between 9–13, which sets the device’s capability (‘normal’ Web vs WebAR vs WebVR) so that you can reference it in other components throughout your application.
import React, { useRef, useState, useEffect, Suspense, lazy } from 'react'
import { Canvas } from '@react-three/fiber'
import { VRCanvas, ARCanvas, useXR, DefaultXRControllers, Hands } from '@react-three/xr'
import deviceStore from '../../stores/device';
export default function XRScene(props) {
const { children } = props;
const { device, setDevice } = deviceStore();
useEffect(() => {
const fetchData = async() => setDevice(await checkDevice())
fetchData();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [])
return (
<>
{device != undefined && device == 'webAR' &&
<ARCanvas style={{
height: '100vh',
width: '100vw'
}}>
<Suspense fallback={null}>
<perspectiveCamera
fov={65}
aspect={window.innerWidth / window.innerHeight}
radius={1000}
/>
<DefaultXRControllers />
{children}
</Suspense>
</ARCanvas>
}
{device != undefined && device == 'webVR' &&
<VRCanvas style={{
height: '100vh',
width: '100vw'
}}>
<Suspense fallback={null}>
<perspectiveCamera
fov={65}
aspect={window.innerWidth / window.innerHeight}
radius={1000}
/>
<DefaultXRControllers />
<Hands />
{children}
</Suspense>
</VRCanvas>
}
{device != undefined && device == 'web' &&
<Canvas style={{
height: '100vh',
width: '100vw'
}}>
<Suspense fallback={null}>
<perspectiveCamera
fov={65}
aspect={window.innerWidth / window.innerHeight}
radius={1000}
/>
{children}
</Suspense>
</Canvas>
}
</>
)
}
const checkDevice = async() => {
if(navigator.xr == undefined) return 'web'
let isAR = await navigator.xr.isSessionSupported( 'immersive-ar');
if(isAR) return 'webAR';
let isVR = await navigator.xr.isSessionSupported( 'immersive-vr');
if(isVR) return 'webVR';
return 'web'
}
Conclusion
This post has shown you how to render a scene based on the device you’re using’s compatibility with the WebXR API.
I hope this is helpful for you developing you WebXR experiences, have fun :D