Real-time deep hair matting on mobile devices
Augmented reality is an emerging technology in many application domains. Among them is the beauty industry, where live virtual try-on of beauty products is of great importance. In this paper, we address the problem of live hair color augmentation. To achieve this goal, hair needs to be segmented quickly and accurately. We show how a modified MobileNet CNN architecture can be used to segment the hair in real-time. Instead of training this network using large amounts of accurate segmentation data, which is difficult to obtain, we use crowd sourced hair segmentation data. While such data is much simpler to obtain, the segmentations there are noisy and coarse. Despite this, we show how our system can produce accurate and fine-detailed hair mattes, while running at over 30 fps on an iPad Pro tablet.
Code (2)
Tasks
Image MattingSegmentationVirtual Try-onSimilar Papers 제목 키워드 기반
Fast Deep Matting for Portrait Animation on Mobile Phone
Image matting plays an important role in image and video editing. However, the formulation of image matting is inherently ill-posed. Traditional methods usually employ interaction to deal with the image matting problem w…
Image MattingPortrait AnimationVideo EditingInductive Guided Filter: Real-time Deep Image Matting with Weakly Annotated Masks on Mobile Devices
Recently, significant progress has been achieved in deep image matting. Most of the classical image matting methods are time-consuming and require an ideal trimap which is difficult to attain in practice. A high efficien…
Image MattingTowards Real-Time Automatic Portrait Matting on Mobile Devices
We tackle the problem of automatic portrait matting on mobile devices. The proposed model is aimed at attaining real-time inference on mobile devices with minimal degradation of model performance. Our model MMNet, based …
Image MattingReal-time Hair Segmentation and Recoloring on Mobile GPUs
We present a novel approach for neural network-based hair segmentation from a single camera input specifically designed for real-time, mobile application. Our relatively small neural network produces a high-quality hair …
SegmentationReal-Time High-Resolution Background Matting
We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our technique is based on background matting, where an additional fra…
4kGPUImage MattingVocal Bursts Intensity Prediction