Webcam Mouse Using Face and Eye Tracking in Various Illumination Environemts

Yuan-Pin Lin, Chung-Chih Lin, Jyh-Horng Chen

Institue of Electrical Engineering,

National Taiwan University

Yi-Ping Chao,

Department of Computer Science and Information Engineering,

Chang Gung University

Comments:

Manoj’s Blog

Summary:

Introduction:

This article presents a way of using a screen mounted webcam to aquire human gestures as a substitute input for the mouse. K-Nearest neighbor classifier in combination with an adaptive skin model is used to provide real time tracking. The system implemented on a standard laptop is able to perform at 15fps.

This system is proposed as an alternative interface for the disabled. Unlike traditional solutions it is minimally intrusive, not requiring the user to wear additional devices. The system requires a dedicated machine to perform the visual tracking whose data is forwarded to the computer that is being operated by the user.

Face tracking is performed using a non linear skin color transform to overcome variations in lighting. Iris tracking is used to identify the eyes and their position. The cursor is then calculated as a product of the eye position relative to the head position. Turning the head has the effect of moving the cursor from side to side.

Feature Recognition:

After defining the KNN features for recognizing illumination conditions. An elliptical model was used with 10 images under different illumination conditions. An average of 92% accuracy for skin detection was achieved. The face was distinguishable due to its clustered appearance.

Eye tracking was then performed with a frame capture comparing the centers of the face and eyes.

Results:

The system runs on a standard 2.4-GHz laptop dedicated to tracking the position of the head and eyes at 15fps occupying 45% of the windows resources. It successfully tracks the head and eyes under a variety of different lighting conditions with complex real world backgrounds.

Discussion:

Calculating the vectors of the face and eyes separately and then determining the point of gaze as a product of the eye vectors relative to the face is a good approach which should lead to improved accuracy. Although it is mentioned that the system is under development. It would have been beneficial to have provided some details as to how the system was tested.