Disney’s Magic Bench creates group virtual reality experience without a headset

By Heather Hamilton, contributing writer

Disney has again expanded its empire with the Disney Research Magic Bench, a group virtual reality experience designed to let users cozy up to their favorite characters. By combining augmented and mixed reality experiences, users can enjoy the experience in unison. Unlike other virtual reality projects, this one doesn’t involve a head-mount display or even a handheld device, instead instrumenting the surroundings instead of the individual, which Moshe Mahler, a principal digital artist at Disney Research, says makes the experience much less isolating.

According to a press release, people on the Magic Bench can see themselves in a mirrored image on a large screen in front of them, which creates a third person point of view. Using a depth sensor, participants occupy the same space as the characters or objects, which are computer-generated. CNET likens the experience to being in a Roger Rabbit movie.

“The platform creates a multi-sensory immersive experience in which a group can interact directly with an animated character,” said Mahler. “Our mantra for this project was: Hear a character coming, see them enter the space, and feel them sit next to you.” At SIGGRAPH 2017, a computer graphics and interactive techniques conference that begins on July 30 in Los Angeles, the research team will present the Magic Bench.

How does it work?
By using a color camera and depth sensor, the team created an HD-video-textured reconstruction of the surroundings, participants, and bench. The algorithm aligns the information from the RGB camera and the information from the depth sensor to recreate the scene. In areas where the depth sensor lacks a line of sight with the color camera, a modified algorithm creates a 2D backdrop to eliminate shadow. Both 2D and 3D reconstructions live in virtual space and contain characters and effects that create a seamless composite in real time, capable of interacting with physics, light, and shadows.

Mahler reports that the bench is critical to the experience. “Not only does it contain haptic actuators, but it constrains several issues for us in an elegant way,” he said. “We know the location and the number of participants and can infer their gaze. It creates a stage with a foreground and a background, with the seated participants in the middle ground. It even serves as a controller; the mixed reality experience doesn’t begin until someone sits down and different formations of people seated create different types of experiences.”

A team of Disney Research scientists, including Kyna McIntosh, John Mars, James Krahe, Jim McCann, Alexander Rivera, Jake Marsico, Ali Israr, and Shawn Lawson worked alongside Mahler to create the Magic Bench.

In a paper describing the project, the team writes that it can be run as a traditional VR Village installation or as an autonomous piece in an unsuspecting area, allowing participants the fun of a surprise VR experience.

Source: Press Release, Disney Research, CNET, Magic Bench — A Multi-User & Multi-Sensory AR/MR Platform
Image Source:  Disney Research