Reducing the duration of paired-comparison experiments for visual quality assessment

Claire Mantel; Soren Forchhammer; Jari Korhonen

doi:10.1109/QoMEX.2013.6603209

Reducing the duration of paired-comparison experiments for visual quality assessment

Claire Mantel, Soren Forchhammer, Jari Korhonen

Computing Science

Technical University of Denmark

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

Abstract

The subjective methodology of paired comparison is currently recognized as the most precise methodology. However, it is not widely used because it implies a larger number of assessments than other methodologies and therefore a longer test duration. In this paper we investigate how sorting algorithms can be used to decrease the duration of such experiments by selecting only a fraction of all the possible comparisons. Three methods based on the insertion sort and the binary tree algorithms are applied to paired-comparison data. Performance analysis shows interesting trade-offs between the number of comparisons and the accuracy of the results.

Original language	English
Title of host publication	2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX)
Publisher	IEEE Press
Pages	50-51
Number of pages	2
ISBN (Electronic)	978-1-4799-0738-0
DOIs	https://doi.org/10.1109/QoMEX.2013.6603209
Publication status	Published - 2013
Event	2013 5th International Workshop on Quality of Multimedia Experience, QoMEX 2013 - Klagenfurt am Worthersee, Austria Duration: 5 Jul 2013 → 7 Jul 2013

Conference

Conference	2013 5th International Workshop on Quality of Multimedia Experience, QoMEX 2013
Country/Territory	Austria
City	Klagenfurt am Worthersee
Period	5/07/13 → 7/07/13

Keywords

Paired comparison
Statistical model
Subjective methodology

Access to Document

10.1109/QoMEX.2013.6603209

Cite this

Mantel, C, Forchhammer, S & Korhonen, J 2013, Reducing the duration of paired-comparison experiments for visual quality assessment. in 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX). IEEE Press, pp. 50-51, 2013 5th International Workshop on Quality of Multimedia Experience, QoMEX 2013, Klagenfurt am Worthersee, Austria, 5/07/13. https://doi.org/10.1109/QoMEX.2013.6603209

@inproceedings{6ccbb8db582b48dab9480bf5429649ba,

title = "Reducing the duration of paired-comparison experiments for visual quality assessment",

abstract = "The subjective methodology of paired comparison is currently recognized as the most precise methodology. However, it is not widely used because it implies a larger number of assessments than other methodologies and therefore a longer test duration. In this paper we investigate how sorting algorithms can be used to decrease the duration of such experiments by selecting only a fraction of all the possible comparisons. Three methods based on the insertion sort and the binary tree algorithms are applied to paired-comparison data. Performance analysis shows interesting trade-offs between the number of comparisons and the accuracy of the results.",

keywords = "Paired comparison, Statistical model, Subjective methodology",

author = "Claire Mantel and Soren Forchhammer and Jari Korhonen",

year = "2013",

doi = "10.1109/QoMEX.2013.6603209",

language = "English",

pages = "50--51",

booktitle = "2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX)",

publisher = "IEEE Press",

note = "2013 5th International Workshop on Quality of Multimedia Experience, QoMEX 2013 ; Conference date: 05-07-2013 Through 07-07-2013",

}

TY - GEN

T1 - Reducing the duration of paired-comparison experiments for visual quality assessment

AU - Mantel, Claire

AU - Forchhammer, Soren

AU - Korhonen, Jari

PY - 2013

Y1 - 2013

N2 - The subjective methodology of paired comparison is currently recognized as the most precise methodology. However, it is not widely used because it implies a larger number of assessments than other methodologies and therefore a longer test duration. In this paper we investigate how sorting algorithms can be used to decrease the duration of such experiments by selecting only a fraction of all the possible comparisons. Three methods based on the insertion sort and the binary tree algorithms are applied to paired-comparison data. Performance analysis shows interesting trade-offs between the number of comparisons and the accuracy of the results.

AB - The subjective methodology of paired comparison is currently recognized as the most precise methodology. However, it is not widely used because it implies a larger number of assessments than other methodologies and therefore a longer test duration. In this paper we investigate how sorting algorithms can be used to decrease the duration of such experiments by selecting only a fraction of all the possible comparisons. Three methods based on the insertion sort and the binary tree algorithms are applied to paired-comparison data. Performance analysis shows interesting trade-offs between the number of comparisons and the accuracy of the results.

KW - Paired comparison

KW - Statistical model

KW - Subjective methodology

UR - http://www.scopus.com/inward/record.url?scp=84885403860&partnerID=8YFLogxK

U2 - 10.1109/QoMEX.2013.6603209

DO - 10.1109/QoMEX.2013.6603209

M3 - Published conference contribution

AN - SCOPUS:84885403860

SP - 50

EP - 51

BT - 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX)

PB - IEEE Press

T2 - 2013 5th International Workshop on Quality of Multimedia Experience, QoMEX 2013

Y2 - 5 July 2013 through 7 July 2013

ER -

Reducing the duration of paired-comparison experiments for visual quality assessment

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this