Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Magic number of choice-task in CBC?


In ACA there is a recommended algorithm for determining the number of pairs questions to ask based on the size of your design --

3 (N - n - 1) - N, where:
N = total number of levels taken into the Pairs section
n = total number of attributes taken into the Pairs section

Is there a similar algorithm for CBC? Is there an ideal number of choice-tasks based on design-size?

asked Mar 3, 2015 by WTW_cal Bronze (1,555 points)
retagged Mar 3, 2015 by Walter Williams

1 Answer

0 votes
Sorry, but there are no such simple guidelines for CBC.

So much depends on what you want to do with the data, if you need to have excellent accuracy at the individual level (or if group-level prediction accuracy is enough), your sample size, your attribute and level structure.

If you are just concerned with group-level accuracy of predictions (share of preference accuracy), then one simple rule of thumb to start with is:

1.  Using our CBC software, select "Advanced Test" from the "Design" tab.
2.  Simulate random responder data (the software does this automatically for you) for your expected sample size, with an expected % responding None (if you have a None in your questionnaire).
3.  Examine the part-worth utility report that the test gives you (it's an aggregate logit run).

For reasonable stability of estimates, the rule of thumb is from this aggregate logit report the main effects utilities should be about .05 or smaller.  If including interaction effects (or alt-spec attribute effects), the standard errors for those should be about 0.10 or less.

Again, this is a general rule of thumb which may or may not be appropriate for your specific study aims.

As another guideline, for typical CBC studies (with about 7 attributes or fewer where each attribute has about 3 to 5 levels at most), most researchers feel comfortable with the precision of their utility estimates at the individual level if each respondent answers somewhere around 12 tasks.

Feel free to talk to our tech support staff at +1 801 477 4700 who can also ask you more questions about your design and your aims and give you their thoughts.
answered Mar 3, 2015 by Bryan Orme Platinum Sawtooth Software, Inc. (163,515 points)
CBC - partial profile