Audio-Visual Speaker Identification Based on the Use of Dynamic Audio and Visual Features

Abstract

This paper presents a speaker identification system based on dynamical features of both the audio and visual modes. Speakers are modeled using a text dependent HMM methodology. Early and late audiovisual integration are investigated. Experiments are carried out for 252 speakers from the XM2VTS database. From our experimental results, it has been shown that the addition of the dynamical visual information improves the speaker identification accuracies for both clean and noisy audio conditions compared to the audio only case. The best audio, visual and audiovisual identification accuracies achieved were 86.91%, 57.14% and 94.05% respectively.