On the Effect of None Position for CBC Studies

In December 2018 we fielded a CBC study using Lighthouse Studio, collecting 201 respondent records via Amazon’s Mechanical Turk panel.  The subject matter was restaurant choice and we used a screener question to qualify respondents who ate at restaurants at least once per month.  We were interested in whether placing the None concept as the right-hand concept vs. placing it as a concept along the bottom of the task would affect the frequency of None usage.  We found that the placement of the None concept did not lead to a statistically significant difference in the None usage.

More Details Regarding the Experiment

We displayed 12 choice tasks, each with three concepts (alternatives) plus a None, for a total of four alternatives per choice task.

We tested two approaches for the None, as displayed further below¹.  The treatments, sample sizes, and resulting None percentages were:

1. Traditional (in-line) None as bottom concept (n = 101) None% = 11.5%
2. Traditional (in-line) None as right-most concept (n = 100) None% = 13.7%

The small difference in None usage is not statistically significant (t = 0.9).  

Here are these formats, visually:

Appendix

Attribute List

1. Theme:
   • Chinese food
   • Japanese food
   • Mexican food
   • Thai food
   • American food
   • Italian food
   • Indian food
2. Distance:
   • 5 minutes travel from you
   • 15 minutes travel from you
   • 45 minutes travel from you
3. Yelp Review:
   • 5 star Yelp review
   • 4 star Yelp review
   • 2 star Yelp review
4. Price:
   • $
   • $$
   • $$$
   • $$$$

¹ Sawtooth Software’s Lighthouse Studio’s CBC tasks can be made responsive for mobile layout (default) or can be made to show a fixed CBC layout whether respondents use mobile or large-display devices. If mobile responsive, the None concept may change positions between mobile and large-display devices. Because we did not have control whether respondents used mobile or large-display devices and we wanted to test the effect of None position on choice, we turned off the responsive functionality and fixed the display of CBC tasks for both mobile and large-display devices.