Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination

Andreas Wenger, Andrew Gardner, Chris Tchou, Jonas Unger, Tim Hawkins, Paul Debevec
ACM Transactions on Graphics (Proceedings of SIGGRAPH 2005), Volume 24, Number 3, page 756--764 - 2005
We present a technique for capturing an actor's live-action performance in such a way that the lighting and reflectance of the actor can be designed and modified in postproduction. Our approach is to illuminate the subject with a sequence of time-multiplexed basis lighting conditions, and to record these conditions with a high-speed video camera so that many conditions are recorded in the span of the desired output frame interval. We investigate several lighting bases for representing the sphere of incident illumination using a set of discrete LED light sources, and we estimate and compensate for subject motion using optical flow and image warping based on a set of tracking frames inserted into the lighting basis. To composite the illuminated performance into a new background, we include a time-multiplexed matte within the basis. We also show that the acquired data enables time-varying surface normals, albedo, and ambient occlusion to be estimated, which can be used to transform the actor's reflectance to produce both subtle and stylistic effects.

Images and movies


See also

BibTex references

  author       = "Wenger, Andreas and Gardner, Andrew and Tchou, Chris and Unger, Jonas and Hawkins, Tim and Debevec, Paul",
  title        = "Performance Relighting and Reflectance Transformation with  Time-Multiplexed Illumination",
  journal      = "ACM Transactions on Graphics (Proceedings of SIGGRAPH 2005)",
  number       = "3",
  volume       = "24",
  pages        = "756--764",
  year         = "2005"

Author publication list