2.Survey: Measurement and Scaling

2.2 Comparative Scales

Classification of Scaling Techniques


Relative Advantages of Comparative Scales


+ small differences between stimulus objects can be detected

+ same known reference points for all respondents

+ easy to understand and to use

+ involve fewer theoretical assumptions

+ tend to reduce halo or carryover effects from one judgement to another


– have only ordinal and rank-order properties ⟶ limited set of statistical methods available for analysis

– data must be interpreted in relative terms

– Inability to generalize beyond the set of compared objects


Comparative Scales: Paired Comparison

Paired Comparison – Respondent is presented with two objects and asked to select one according to some criterion

We are going to present you with ten pairs of beer brands. For each pair, please indicate which one of the two brands of beer you would prefer to purchase.


Paired Comparison Scales: Examples

Paired Comparison: Pros-and-Cons


+ direct comparison and overt choice

+ good for blind tests, physical products, and MDS

+ allows for calculation of percentage of respondents who prefer one stimulus to another

+ can assess rank-orders of stimuli (under assumption of transitivity)

+ possible extensions: “no difference” alternative; graded comparison


– # of comparisons grows quicker than # of stimuli (for n objects n(n-1)/2 comparisons)

– presentation order bias possible

– preference of A over B does not imply subject’s liking of A

– little similarity to real choice situation with multiple alternatives

– violations of transitivity may occur


Ordinal Data: violations of transitivity in paired comparison


Ordinal data: violations of transitivity when aggregating preferences

Apple is both the best and the worst alternative. Aggregated preferences of the group are inconsistent!


Comparative Scales: Rank Order Scaling

Rank Order Scaling – Respondents are presented with several objects simultaneously and are asked to order or rank them according to some criterion



Rank Oder Scales: Example

Rank Oder Scales: Pros-and-Cons


+ direct comparison

+ more realistic than paired comparison

  •  + # of comparisons is only (n-1)
  •  + easier to understand
  •  + takes less time
  •  + no intransitive responses
  •  + can be converted to paired comparison data

+ good for measuring preferences of brands or attributes; conjoint analysis


– preference of A over B does not imply subject’s liking of A

– no zero point / separation between liking and disliking

– only ordinal data

– violations of transitivity may occur


Comparative Scales: Constant Sum Scaling

Constant Sum – Respondents allocate a constant sum of units (points, dollars, chips, %) among a set of stimulus objects with respect to some criterion


Constant Sum Scaling: Example of Analysis


Constant Sum Scaling: Example


Constant Sum Scaling: Pros-and-Cons


+ allows for fine discrimination among stimulus objects without requiring too much time

+ ratio scaled ⟶ flexible choice of data analysis methods


– results are limited to the context of stimuli scaled, i.e., not generalizable to other stimuli not included in the study

– relatively high cognitive burden for respondents, esp. when # of items is large

– prone to calc. errors (e.g., allocation of 108 or 94 points)


Comparative Scales: Q-Sort Scaling

Q-Sort – A rank order procedure in which objects are sorted into piles based on similarity with respect to some criterion. Usually used to discriminate among a relatively large number (60-140) of objects quickly. The number of objects in each pile is limited, usually so that all piles imitate normal distribution.

To prevent epidemics, the Ministry of Health has developed the following 25 measures recommended for implementation in hospitals. Please distribute these measures for preventing the spread of infections according to their importance using the scheme below. Please allocate only one measure per box. 

To the table of contents