Qualcomm XR2 5G reference design adds Ultraleap hand tracking tech

Ultraleap’s fifth generation hand tracking platform, called Gemini, will be integrated into Qualcomm Technologies Inc.’s standalone Snapdragon XR2 5G reference design for head-mounted displays (HMDs). This is the first design in for Ultraleap’s Gemini technology, which tracks hand and finger positioning using artificial intelligence (AI). The two companies signed a multi-year co-operation agreement that brings Ultraleap’s hand-tracking capabilities to Qualcomm’s 5G platform for untethered virtual reality, mixed reality, and augmented reality (VR, MR and AR), also known as extended reality or XR.

Designers of XR HMDs face big challenges in bringing together the right processing power, tracking systems that use complex sensor fusion gathering data from sensors and cameras, and other components in the right form factor. Qualcomm’s reference design was developed to simplify this complex system and help OEMs accelerate their HMD designs.

With the new Ultraleap technology, the companies believe it will further simplify and improve the process of adding hand tracking technology, while providing more immersive experiences for users.

Delivering a fast and accurate hand tracking technology to the Snapdragon XR2 5G platform, it allows for always on experience and the most natural interaction in untethered XR, according to Ultraleap. Steve Cliffe, Ultraleap’s CEO noted in a release that it “will make the process of designing hand tracking within a very wide variety of products as simple as pick and place.”

(Image: Qualcomm)

Introduced earlier this year, Qualcomm’s reference platform combines 5G and AI with 360° spherical view and 3D audio for high-fidelity sound in all directions. The 5G-enabled XR reference design was developed to simplify complex technologies around the delivery of AR/VR/MR devices for both consumers and enterprises.

The reference design comes in a form factor that enables designers to jump-start their next-generation 5G-enabled HMD design and software development. Qualcomm said it delivers “boundless XR” that is more immersive over a 5G solution for the first time with the Qualcomm Snapdragon X55 5G Modem-RF System. It provides native support for 5G mmWave and sub-6 GHz, and has been tested over Ericsson’s 5G infrastructure.

One example of how 5G and XR come together is through distributed compute between the edge of the cloud and on device, said Hugo Swart, Qualcomm’s VP and GM, product management, XR, in a video. “We call it boundless XR. It provides a mobile untethered experience and unlimited processing capacity at the edge of the cloud to provide photo-realistic images.”

The reference design supports multiple cameras for precise motion tracking and gesture recognition. It also enables developers to build different configurations for controller tracking.