Transformed Human Presence for Puppetry

Abstract:
We propose a system for transmitting a human performer’s body and facial movements to a puppet with audiovisual feedback to the performer. The system consists of a head-mounted display (HMD) that shows the performer the video recording of the puppet’s view, a microphone for voice capture, and photoreflectors for detecting the mouth movements of the human performer. In conventional puppetry, there is also the need for practice in the manipulation of the puppets to achieve good performance. The proposed telepresence system addresses these issues by enabling the human performer to manipulate the puppet through their own body and facial movements. The proposed system is expected to contribute to the development of new applications of puppetry and expand the interactivity of puppetry and the scope of entertainment.

Keisuke Kawahara, Mose Sakashita,  Amy Koike, Ippei Suzuki, Kenta Suzuki, Tatsuya Minagawa, Yoichi Ochiai
University of Tsukuba

UIST 2017 Technical Paper (Author Preprint) : PDF (3.7MB)