Sawtooth Software: The Survey Software of Choice

Clients don’t seem to be able to get enough of a good thing and this seems to apply more to MaxDiff than to some of the other methods we use: clients frequently ask for MaxDiff experiments that include more items than would allow us to expose each item to each respondent the recommended three or four times.   In 2012, Wirth and Wolfrath introduced “Express” MaxDiff and “Sparse” MaxDiff as ways to handle more than the usual number of items.  Express MaxDiff draws a subset of items for each respondent and shows each item multiple times.  Sparse MaxDiff typically shows each item perhaps just once to each respondent, trying to obtain as much coverage as possible for each respondent over the items.

The author, Keith Chrzan, investigates whether Express or Sparse MaxDiff does a better job of recovering “true” utilities using simulated robotic respondents.  For two data sets that were based on real patterns of live humans’ preferences, he finds a modest edge in performance for Sparse MaxDiff.

Express MaxDiff has the benefit of repeated measures at the individual level.  Sparse has the benefit of avoiding so many missing items at the individual level.  Both methods may be analyzed via counts, aggregate logit, latent class, or HB.  Even though it is possible to estimate individual-level scores via HB, it is relatively imprecise under Express or Sparse MaxDiff compared to the recommended MaxDiff study that shows each item to each respondent about 3 or 4 times.

  Download PDF   View in new window