On designing paired comparison experiments for subjective multimedia quality assessment

Research output: Contribution to journalArticlepeer-review

26 Citations (Scopus)


This paper investigates the issue of designing paired comparison-based subjective quality assessment experiments for reliable results. In particular, the convergence behavior of the quality scores estimated from paired comparison results is considered. Via an extensive computer simulation experiment, the estimation performance in terms of the root mean squared error, the rank order correlation coefficient, and the change of the estimated scores with respect to the number of subjects are mathematically modeled. Furthermore, it is confirmed that the models coincide with the theoretical convergence behavior. Issues such as the effect of human errors and the underlying distribution of the true quality scores are also examined.

Original languageEnglish
Article number6675876
Pages (from-to)564-571
Number of pages8
JournalIEEE Transactions on Multimedia
Issue number2
Publication statusPublished - 2014 Feb

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'On designing paired comparison experiments for subjective multimedia quality assessment'. Together they form a unique fingerprint.

Cite this