Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

The "Number of Levels" effect on partial profile CBC


I have a few questions regarding the "Number of Levels" effect...

In the Sawtooth research paper "Formulating Attributes and Levelsin Conjoint Analysis", it is noted that "the number of levels you choose to define an attribute can have a significant bearing on the results." And the suggestion is to at least approximately balance the number of levels across attributes.  

I looked into another Sawtooth research paper, "The Number of levels Effect in Conjoint: where does it come from and can it be eliminated?" It seems in the literature, the "NoL" effect was mostly found in full profile designs, and compared with ACA.  So, does the "NoL" effect have a similar bearing on the results of partial profile CBC?

In my project using partial profile CBC design, we often have 6 levels in some attributes, 2 levels in a few, and 3-5 levels in others.  Let's say I couldn't find a way to approximately balance the number of levels across attributes.   What would be the consequence?  Does it mean the 2-level attributes' importance might be underestimated, and the 6-level attributes' importance might be overestimated due to the "NoL" effect?

If I can't balance the number of levels across attributes, is it possible for me to specifiy somethng in Sawtooth CBC system so that it could display the 2-level attributes less frequently and the 6-level ones more frequently in the choice tasks in order to potentially ease the NoL effect by balancing the percentage of shown vs. selected per attribute?

Thanks very much for your help!
asked May 30, 2014 by anonymous

1 Answer

+1 vote
You could create a partial profile design, export it into a CSV file and then make adjustments to the design by hand in order to balance the number of times each level (as opposed to each attribute) occurs, but this could be a pretty painful bit of work (and one which you would want to make sure to test  when you imported the design back into SSI Web.

Even easier (and this is how I've done it before) would be to build 2 partial profile experiments, one with all the attributes and a second that just has the attributes with the larger number of attributes.  Have each respondent do a few tasks from each of the two experiments and you will have gone a good way toward balancing the number of times each level appears.
answered May 30, 2014 by Keith Chrzan Platinum Sawtooth Software, Inc. (62,700 points)
follow up on NoL