TY - GEN
T1 - Two-level bimodal association for audio-visual speech recognition
AU - Lee, Jong Seok
AU - Ebrahimi, Touradj
PY - 2009
Y1 - 2009
N2 - This paper proposes a new method for bimodal information fusion in audio-visual speech recognition, where cross-modal association is considered in two levels. First, the acoustic and the visual data streams are combined at the feature level by using the canonical correlation analysis, which deals with the problems of audio-visual synchronization and utilizing the cross-modal correlation. Second, information streams are integrated at the decision level for adaptive fusion of the streams according to the noise condition of the given speech datum. Experimental results demonstrate that the proposed method is effective for producing noise-robust recognition performance without a priori knowledge about the noise conditions of the speech data.
AB - This paper proposes a new method for bimodal information fusion in audio-visual speech recognition, where cross-modal association is considered in two levels. First, the acoustic and the visual data streams are combined at the feature level by using the canonical correlation analysis, which deals with the problems of audio-visual synchronization and utilizing the cross-modal correlation. Second, information streams are integrated at the decision level for adaptive fusion of the streams according to the noise condition of the given speech datum. Experimental results demonstrate that the proposed method is effective for producing noise-robust recognition performance without a priori knowledge about the noise conditions of the speech data.
UR - http://www.scopus.com/inward/record.url?scp=70549102475&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=70549102475&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-04697-1_13
DO - 10.1007/978-3-642-04697-1_13
M3 - Conference contribution
AN - SCOPUS:70549102475
SN - 3642046967
SN - 9783642046964
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 133
EP - 144
BT - Advanced Concepts for Intelligent Vision Systems - 11th International Conference, ACIVS 2009, Proceedings
T2 - 11th International Conference on Advanced Concepts for Intelligent Vision Systems, ACIVS 2009
Y2 - 28 September 2009 through 2 October 2009
ER -