Figure 1

Substitutional Reality System.

In the recording module (left), the panoramic view was recorded in advance by a panoramic camera and stored in the data storage connected to the control computer. In the experience module (right), either a live scene captured by a head-mounted camera or recorded scenes cropped from a pre-recorded movie were shown on a head-mounted display (HMD). The cropped area presented in the recorded scenes was determined in real-time using head orientation information calculated from the HMD orientation sensor. Scene examples are shown here. In the recorded scene a person with a lab coat waved his hand, who was not present in the live scene. A participant believed the person with the lab coat was physically present there, when the covert switch from the live to the recorded scene was successfully performed.