Wiizards: 3D Gesture Recognition for Game Play Input

Louis Kratz,

Frank J. Lee,

Dept. of Computer Science

Matthew Smith,

Digital Media Labs

Frank J. Lee,

Drexel Universoty

3141 Chesnut Street

Philadelphia, PA 19104

Comments:

Manoj’s Blog

Franck’s Blog

Summary:

Introduction:

This paper explores the use of gestures for game play input using a 3D accelerometer as an input device. Two types of gestures are defined: Static, and dynamic. Bayesian methods will be used to classify the accelerometer data directly in place of path tracking. This is seen as more efficient than path shape matching. The Hidden Markov Models approach used in this paper is additionally well suited to the noisy sensor data provided by the accelerometer.

Implementation:

Wiizards in a two player zero sum game where opponents cast spells at each other. As each player performs a gesture it is placed on a queue so that a series of gestures leading to a cumulation is possible. Several forms of visual feedback are provided. A visual representation of the spell queue which serves as a reminder. A indication of how long until a spell becomes available.

A nintendo Wii controller and a gesture recognition system are used in the implementation. Gestures are represented in the form a collection of vectors. A model is created for each gesture and then a probabilistic matching approach is used as calculated by the Viterby Algorithm.

Results:

7 Users were tested.

Each gesture was performed over 40 times.

Multiple states were used.

90% correct recognition was obtained with only 10 states.

93% with 15 states.

250 gestures per second were possible with a 2.66Ghz PC.

The time of training significantly increases with the number of states.

Conclusions:

The implementation and structure of the game allowed it to adapt with the users varying levels. As the individual’s skill increased so could the complication of the gestures.

Discussion:

The system can run in real time but is encumbered by a 10 second machine training session, which is seen as a limiting factor.

The adaptive nature of the system is very appealing as it should maintain interest as the users skill improves.