Time-offset Conversations on an Automultiscopic Projector Array

Andrew Jones, Jonas Unger, Koki Nagano, Jay Busch, Xueming Yu, Hsuan-Yueh Peng, Joseph Barreto, Oleg Alexander, Paul Debevec, Mark Bolas
CVPR workshop on Computational Cameras and Displays 2016 - 2016
We present a system for creating and displaying interactive life-sized 3D digital humans based on pre-recorded interviews. We use 30 cameras and an extensive list of questions to record a large set of video responses. Users access videos through a natural conversation interface that mimics face-to-face interaction. Recordings of answers, listening and idle behaviors are linked together to create a persistent visual image of the person throughout the interaction. The interview subjects are rendered using flowed light fields and shown life-size on a special rear-projection screen with an array of 216 video projectors. The display allows multiple users to see different 3D perspectives of the subject in proper relation to their viewpoints, without the need for stereo glasses. The display is effective for interactive conversations since it provides 3D cues such as eye gaze and spatial hand gestures.

Images and movies

 

BibTex references

@InProceedings\{JUNBYPBADB16,
  author       = "Jones, Andrew and Unger, Jonas and Nagano, Koki and Busch, Jay and Yu, Xueming and Peng, Hsuan-Yueh and Barreto, Joseph and Alexander, Oleg and Debevec, Paul and Bolas, Mark",
  title        = "Time-offset Conversations on an Automultiscopic Projector Array",
  booktitle    = "CVPR workshop on Computational Cameras and Displays 2016",
  year         = "2016"
}

Author publication list