Randomizing Questions/Pages/Blocks

There are times where the researcher wants to randomize a series of pages, questions on a page, or blocks of questions. This is most often done to reduce response bias due to question order. Question order bias occurs when respondents answer differently to later questions because of the influence of previously answered questions. If questions are displayed in random order, question order bias still exists at the individual level. But, because individuals each receive a different question ordering, the bias is nearly eliminated when viewing aggregate results.

You can randomize the order that questions are asked within the same page. You can also randomize the order of pages. You cannot randomize the order of questions that span from one page to another, when there are multiple questions per page.

---

Randomizing Questions on a Page

To randomize the order that questions are asked on a page:

1) Click Compose | Write Questionnaire and click the Randomize... | Questions on a Page button. From the Randomize Questions on a Page dialog, click the Add... button.

2) Select the beginning and ending question. All questions from the beginning to the ending question (including the beginning and ending questions) are displayed in random order when the questionnaire runs on a Web server.

---

Randomizing the Order of Pages

1) Click Compose | Write Questionnaire and click the Randomize... | Pages button. From the Randomize Pages dialog, click the Add... button.

2) Select any question from the first and last pages in the randomized sequence. All pages including and between the anchor questions are displayed in random order when the questionnaire runs on a Web server.

---

Randomizing Blocks of Questions

Example: imagine a questionnaire with three blocks of questions Q1-Q3 (block 1), Q4-Q10 (block 2), and Q11-Q20 (block 3). We may want to randomize the order that those three blocks of questions occur. For example, one possible ordering is Q4-Q10, Q1-Q3, Q11-Q20.

1) Click Compose | Write Questionnaire and click the Randomize... | Blocks button. From the Randomize Blocks dialog, click the Add Set button.

2) Add each block of questions by clicking Add Block... and then choosing a beginning and ending question for each block.

3) Select to either randomize the order that the blocks are asked, or to ask the blocks in the order as indicated in a constructed list. For example, if the blocks were to be asked in the order as represented within a constructed list, and that list contained elements ordered 3, 2, 1 then the blocks would be asked in order: block 3, block 2, block1.

Notes: Each block must be separated by at least one page break. Questions can be additionally randomized within blocks.