Dominant edge direction based fast intra mode decision in the H.264/AVC encoder

Byeongdu La, Minyoung Eom, Yoonsik Choe

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)


The H.264/AVC video coding standard uses an intra prediction mode with 4×4 and 16×16 blocks for luma and 8×8 blocks for chroma. This standard uses the rate distortion optimization (RDO) method to determine the best coding mode based on the compression performance and video quality. This method offers a large improvement in coding efficiency compared to other compression standards, but the computational complexity is greater due to the various intra prediction modes. This paper proposes a fast intra mode decision algorithm for real-time encoding of H.264/AVC based on the dominant edge direction (DED). The DED is extracted using pixel value summation and subtraction in the horizontal and vertical directions. By using the DED, three modes instead of nine are chosen for RDO calculation to decide on the best mode in the 4×4 luma block. For the 16×16 luma and the 8×8 chroma, only two modes are chosen instead of four. Experimental results show that the entire encoding time saving of the proposed algorithm is about 67% compared to the full intra search method with negligible loss of quality.

Original languageEnglish
Pages (from-to)767-777
Number of pages11
JournalJournal of Zhejiang University: Science A
Issue number6
Publication statusPublished - 2009 Jun

Bibliographical note

Funding Information:
‡ Corresponding author * Project (No. IITA-2009-(C1090-0902-0011)) supported by the Ministry of Knowledge Economy of Korea under the ITRC Support Program supervised by the IITA # Expanded based on “Fast Mode Decision for Intra Prediction in H.264/AVC Encoder” by La et al., which appeared in the Proceedings of the 2007 IEEE International Conference on Image Processing

All Science Journal Classification (ASJC) codes

  • Engineering(all)


Dive into the research topics of 'Dominant edge direction based fast intra mode decision in the H.264/AVC encoder'. Together they form a unique fingerprint.

Cite this