Random order of answer options, questions, pages and blocks

Why random order?

Randomizing questions or answer options (often called randomization in technical terms) in online questionnaires helps to avoid various biases and improve data quality.

For one, it reduces position or order effects: respondents tend to choose answers at the beginning or end of a list disproportionately often — regardless of the content.

For another, it lessens what are known as spillover effects: here, the answer to an earlier question unconsciously influences the subsequent answers, especially when they're closely related in content. Randomization minimizes this influence.

QUESTIONSTAR offers flexible settings for randomizing answer options, questions, pages and page groups.

Random order of answer options

Randomizing answer options is especially useful when there's no natural or logical order to the options. This reduces answer tendencies (e.g. always choosing the first option).

Here's how to enable random order:

1. Select the question in question.
2. In the settings panel on the right, open the Question Settings.
3. Enable the “Randomized options enabled” option.

4. Once enabled, two indicators appear:

   • In the top bar of the question
   • To the left of the answer options

   These indicators show at a glance that randomization is enabled.

Random order of items and scale points in matrix questions

In matrix questions, both the order of the items (rows) and the scale points (columns) can be randomized — especially helpful when all answer options are equivalent and don't require a fixed order.

Here's how to enable randomization:

1. Select the matrix question.
2. Open the settings panel on the right.

3. Enable:

   • “Randomize Question Order” for rows (items)
   • “Randomize Question Order” for columns (scale points)

4. Once enabled, visual cues appear:

   • Icon in the top bar of the question
   • Indicator to the left of the rows
   • Indicator above the column labels

   These elements show whether and which type of randomization has been enabled.

Pinning answer options

In certain cases, individual answer options should not be randomized but instead displayed in a fixed position. This applies, for example, to options like “Other”, “Miscellaneous” or “None of the above”, which usually appear at the end of a list.

Here's how to pin an answer option:

1. Click into the text field of the relevant answer option. This opens the helper menu.
2. Select the “Pin item” option.
3. A pin icon then appears to the left of the answer option. This icon indicates that the item is pinned and is not affected by the randomization.

Random order of questions

Randomizing the question order serves to avoid influence effects between questions — particularly when the answer to one question could be biased by the previous one. This method is often used in knowledge checks or test situations to encourage unbiased and independent answers.

Here's how to enable randomization of questions within a questionnaire page:

1. Open the Page Settings.
2. Enable the “Randomize Question Order” toggle.
3. An indicator in the sidebar shows that the questions on this page are displayed in random order.

Pinning questions

Some questions — such as introductions, instructions or notes about the test procedure — should not be randomized but instead remain in a fixed position in the questionnaire.

Here's how to pin a question:

1. Open the Question Settings.
2. Enable the “Pinned Question” toggle.
3. A pin icon in the question bar indicates that the question is pinned and is not included in the random order.

Random order of pages

Randomizing the page order is especially useful when order effects are to be avoided — for example in product tests, knowledge checks or experimental designs. By arranging the pages randomly, each respondent sees the content in a different order, which reduces bias caused by position effects.

Here's how to enable random order of pages:

1. Open the three-dot menu in the top menu bar.
2. Select the “Randomize Blocks” option.

3. Combine two adjacent pages that you want to randomize into a block.

   

4. Extend the block if needed upward or downward until all the desired pages are included.

   

5. Enable the “Randomize pages in this block” checkbox.

   

6. If a page within the block is to be pinned, click the pin icon to the left of the page.
7. A block created by mistake can be removed again via the “Ungroup” button.

Random order of page groups

If your questionnaire consists of several thematically related pages, you can group them into blocks and randomly vary their order. This lets you avoid order effects between topic areas while preserving the content structure within each block — ideal for personality, attitude or topic-module questions.

Here's how to enable randomization of blocks:

1. First create the desired blocks by following steps 1 to 4, as described in the “Random order of pages” section.
2. Enable the “Randomize groups” toggle to display the blocks in random order.
3. By clicking the pin icon to the left of a block, you can fix its position and exclude it from the randomization.
4. A block created incorrectly can be dissolved again via the “Ungroup” button.