Development of an Face Chroma-key Application for Mixed Reality based Interactive Contents

Abstract

In this paper, to develop mixed reality (MR) based interactive contents, we introduce face chroma-key application using face detection and face landmark detection techniques. For face detection, we propose deep learning based face detection algorithm, For face landmark detection, we use Constrained Local Neural Fields (CLNF) model as patch model of Constrained Local model (CLM). For interactive contents, we send the result of face chroma-key to main server and put them on the virtual character’s face. We implement face chroma-key application on android mobile.