Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

CBC # of respondents per version

If I have 10 versions of my survey and 90 respondents, do I have to have the same exact number of respondents per version (9 responses per version in my example) or is it okay to have 12 for one version 8 for another, etc...?
asked Nov 17, 2011 by anonymous
retagged Sep 13, 2012 by Walter Williams

2 Answers

+1 vote
Each version by itself (if using Shortcut or especially Balanced Overlap/Complete Enumeration) is level balanced and near-orthogonal.  That means each version by itself (from a statistical standpoint) provides unbiased, quite precise estimates of at least main-effects.  Estimation of interactions are improved with increased number of versions fielded.

One of the main reasons to do multiple versions is to reduce psychological/order effects.  Once you have quite a bit of variation across respondents in terms of tasks shown, you will significantly reduce context/order effects.  

So, if your versions aren't distributed quite evenly, this should have very little effect on the quality of utilities estimates via methods that employ the logit rule (logit, latent class, and HB).  

So, don't fret if the completes per version aren't balanced.
answered Nov 17, 2011 by Bryan Orme Platinum Sawtooth Software, Inc. (131,990 points)
0 votes
Although your respondent number seems to be a bit low at first glance, in general your versions don't have to be equally distributed. As far as I remember, in cases where some versions aren't shown at all, your study is still OK.
answered Nov 17, 2011 by Bahadir Ozkurt Gold (16,830 points)