Visual-speech-pass filtering for robust automatic lip-reading

Jong Seok Lee

doi:10.1007/s10044-013-0350-x

Visual-speech-pass filtering for robust automatic lip-reading

Jong Seok Lee

Research output: Contribution to journal › Article › peer-review

5 Citations (Scopus)

Abstract

This paper proposes a temporal filtering technique used in extraction of visual features for improved robustness of automatic lip-reading, called visual-speech-pass filtering. A band-pass filter is applied to the pixel value sequence of the images containing the speaker's lip region to remove unwanted variations that are not relevant to the speech information. The filter is carefully designed based on psychological, spectral, and experimental analyses. Experimental results on two speaker-independent and one speaker-dependent recognition tasks demonstrate that the proposed technique significantly improves recognition performance in both clean and visually noisy conditions.

Original language	English
Pages (from-to)	611-621
Number of pages	11
Journal	Pattern Analysis and Applications
Volume	17
Issue number	3
DOIs	https://doi.org/10.1007/s10044-013-0350-x
Publication status	Published - 2014 Aug

Bibliographical note

Funding Information:
This research was supported by the Ministry of Science, ICT & Future Planning (MSIP), Korea, in the ICT R&D Program 2013, and by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (No. 2013R1A1A1007822).

All Science Journal Classification (ASJC) codes

Computer Vision and Pattern Recognition
Artificial Intelligence

Access to Document

10.1007/s10044-013-0350-x

Cite this

@article{c585ec3b21904421ab8479c3c919e2de,

title = "Visual-speech-pass filtering for robust automatic lip-reading",

abstract = "This paper proposes a temporal filtering technique used in extraction of visual features for improved robustness of automatic lip-reading, called visual-speech-pass filtering. A band-pass filter is applied to the pixel value sequence of the images containing the speaker's lip region to remove unwanted variations that are not relevant to the speech information. The filter is carefully designed based on psychological, spectral, and experimental analyses. Experimental results on two speaker-independent and one speaker-dependent recognition tasks demonstrate that the proposed technique significantly improves recognition performance in both clean and visually noisy conditions.",

keywords = "Automatic lip-reading, Feature extraction, Noise-robustness, Temporal filtering, Visual-speech-pass filtering (VSPF)",

author = "Lee, {Jong Seok}",

note = "Funding Information: This research was supported by the Ministry of Science, ICT & Future Planning (MSIP), Korea, in the ICT R&D Program 2013, and by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (No. 2013R1A1A1007822). ",

year = "2014",

month = aug,

doi = "10.1007/s10044-013-0350-x",

language = "English",

volume = "17",

pages = "611--621",

journal = "Pattern Analysis and Applications",

issn = "1433-7541",

publisher = "Springer London",

number = "3",

}

TY - JOUR

T1 - Visual-speech-pass filtering for robust automatic lip-reading

AU - Lee, Jong Seok

N1 - Funding Information: This research was supported by the Ministry of Science, ICT & Future Planning (MSIP), Korea, in the ICT R&D Program 2013, and by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP (No. 2013R1A1A1007822).

PY - 2014/8

Y1 - 2014/8

N2 - This paper proposes a temporal filtering technique used in extraction of visual features for improved robustness of automatic lip-reading, called visual-speech-pass filtering. A band-pass filter is applied to the pixel value sequence of the images containing the speaker's lip region to remove unwanted variations that are not relevant to the speech information. The filter is carefully designed based on psychological, spectral, and experimental analyses. Experimental results on two speaker-independent and one speaker-dependent recognition tasks demonstrate that the proposed technique significantly improves recognition performance in both clean and visually noisy conditions.

AB - This paper proposes a temporal filtering technique used in extraction of visual features for improved robustness of automatic lip-reading, called visual-speech-pass filtering. A band-pass filter is applied to the pixel value sequence of the images containing the speaker's lip region to remove unwanted variations that are not relevant to the speech information. The filter is carefully designed based on psychological, spectral, and experimental analyses. Experimental results on two speaker-independent and one speaker-dependent recognition tasks demonstrate that the proposed technique significantly improves recognition performance in both clean and visually noisy conditions.

KW - Automatic lip-reading

KW - Feature extraction

KW - Noise-robustness

KW - Temporal filtering

KW - Visual-speech-pass filtering (VSPF)

UR - http://www.scopus.com/inward/record.url?scp=84903955792&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84903955792&partnerID=8YFLogxK

U2 - 10.1007/s10044-013-0350-x

DO - 10.1007/s10044-013-0350-x

M3 - Article

AN - SCOPUS:84903955792

SN - 1433-7541

VL - 17

SP - 611

EP - 621

JO - Pattern Analysis and Applications

JF - Pattern Analysis and Applications

IS - 3

ER -

Visual-speech-pass filtering for robust automatic lip-reading

Abstract

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this