Office Activity Recognition using Hand Posture Cues

Brandon Paulson, Tracy Hammond Sketch Recognition Lab 3112 TAMU College Station, TX 77843

Comments:

Paul’s Blog

Manoj’s Blog

Summary:

Activity recognition is critical in context aware applications. The goal of the paper is to determine if it is possible to use hand postures as a cue to identifying the types of user object interactions in the office environment. Furthermore, are hand postures user specific during natural object interactions. The study has determined that it is possible to predict object interactions from hand postures with an accuracy of above 94%, and that the hand postures are user specific.

With mobile computing, context awareness is increasingly of interest. Location, identity, activity and time are viewed as the most significant contextual elements. Of these activity is the hardest to measure and the least used. Activity theory ties objectives to objects and tool, but it is still necessary to determine when the activity is taking place. The data glove was chosen for this particular study as it was portable and did not suffer from the occlusion problems of video based systems. Additionally, the glove enabled the variation in natural hand postures to be explored.

Activity is related not only to objects, but to the user’s own body. Examples of which are walking, climbing stairs etc. Previous studies have been limited to predetermined objects and spaces. The goal of this work has been to identify the objects which are being interacted with and not the hand posture or grip.

A right handed glove was used with a data acquisition rate of 10Hz, 8 subjects were tested with 12 interactions which were repeated 5 times. A 1-nearest neighbor classifier was used.

User independent results varied between 41.7 - 81.7% with an average of 62.5%. There was a high degree of individual variation in technique.

User dependent results were much tighter with an average of 94.2%. However, there was still some ambiguity in like actions.

Future improvements with the use of additional sensory modalities, improved segmentation, and a more advanced classifier were proposed.

Discussion:

This paper proposes a simple and effective approach to identifying activity in a particular environment with the use of a data glove by itself. It is impressive that the use of a simple classifier was able to accurately discriminate between the objects in a restricted but still relatively complicated environment. With additional sensors and a more complex classifier similar results should be possible for a user independent study.