3D model silhouette-based tracking in depth images for puppet suit dynamic video-mapping
Video-mapping is the process of coherent video-projection of images, animations or movies on static objects or buildings for shows. This paper focuses on the dynamic video-mapping of the suit of a puppet being moved by its puppeteer on the theater stage. This may allow changing the costume dynamically and simulate light interaction and more. Contrary to common video-mapping, the image warping cannot be done once, offline, before the show. It must be done in real-time, and considering a non-flat projection surface, so that the video-projected suit always maps perfectly the puppet, automatically. Hence, we propose a new visual tracking method of articulated object, for the puppet tracking, exploiting the silhouette of a 3D model of it, in the depth images of a Kinect v2. Then, considering the precise calibration between the latter and the video-projector, that we propose, coherent dynamic video-mapping is made possible as the presented results show.
Code (0)
등록된 구현이 없습니다.
Tasks
Visual TrackingSimilar Papers 제목 키워드 기반
HaSPeR: An Image Repository for Hand Shadow Puppet Recognition
Hand shadow puppetry, also known as shadowgraphy or ombromanie, is a form of theatrical art and storytelling where hand shadows are projected onto flat surfaces to create illusions of living creatures. The skilled perfor…
image-classificationImage ClassificationReal-time Pupil Tracking from Monocular Video for Digital Puppetry
We present a simple, real-time approach for pupil tracking from live video on mobile devices. Our method extends a state-of-the-art face mesh detector with two new components: a tiny neural network that predicts position…
Pupil TrackingDirect Handheld Burst Imaging to Simulated Defocus
A shallow depth-of-field image keeps the subject in focus, and the foreground and background contexts blurred. This effect requires much larger lens apertures than those of smartphone cameras. Conventional methods acquir…
Depth EstimationPuppet-CNN: Input-Adaptive Convolutional Neural Networks with Model Compression using Ordinary Differential Equation
Convolutional Neural Network (CNN) has been applied to more and more scenarios due to its excellent performance in many machine learning tasks, especially with deep and complex structures. However, as the network goes de…
Model CompressionJoint Multiview Segmentation and Localization of RGB-D Images Using Depth-Induced Silhouette Consistency
In this paper, we propose an RGB-D camera localization approach which takes an effective geometry constraint, i.e. silhouette consistency, into consideration. Unlike existing approaches which usually assume the silhouett…
Camera LocalizationImage SegmentationSegmentationSemantic Segmentation