Exploring the time course of facial expressions with a fuzzy system

Abstract

Recognizing facial expressions is an important challenge in current human-computer interaction (HCI) research . Previous research shows the limits of recognition based on a single static image, and analyzing video sequences seems a more promising approach. We explore three fuzzy systems for the classification of basic facial expressions and compare their performances with a template-correlation approach. We then use these to examine the dynamics of facial expressions. The system's inputs are the relative variations of distances defined by salient facial points from one frame to the next. For maximum compatibility, these facial points (eyebrows, eyes, mouth) were chosen from the set of points defined in the standard MPEG-4 specifications, so that their automatic extraction is tractable. The first results suggest that some expressions can be recognized early. For other expressions, it is in general possible to reduce significantly the number of possibilities. Forming early hypotheses for expressions could be necessary for a system to work in real-time, since other steps may have to follow: such as predicting user actions, the choice of computer response, etc. This also has implications for the recognition of milder expressions