Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Do I need fixed-tasks if I don't do market simulation?


one question: are fixed tasks only needed for testing the models predicting power when doing market simulation?

As I most probably won't do market simulation, only logit and latent class analysis, I wonder whether I need fixed tasks?

Do they serve for any other important issue?
Just want to make sure that I don't need them for any other thing before I decide not to have any fixed tasks in my CBC.

asked Nov 13, 2013 by anonymous

1 Answer

+1 vote
Fixed tasks are useful for double-checking that your estimated models can work well to predict held out data (could be especially of value for testing different model specifications and different utility estimation techniques).  They can be useful for directly asking about specific product scenarios the client is interested in.  For certain situations, particularly when budget is limited and when both the client and the researcher already have supreme confidence in a conjoint analysis technique, model specification, and utility estimation approach, holdouts aren't always necessary in CBC.

That said, if you are going to use holdouts for validation purposes and testing the effectiveness of different conjoint models, just 1 or 2 holdout scenarios typically won't cut it.  Just one or two scenarios typically don't give you enough data points to validate against.

You'll see in methodological research papers that are meant to stand up to academic scrutiny and to provide good indications of comparative predictive validity that often six or more holdout tasks are used.  And, preferably, an entire group of a few hundred respondents is treated as an out-of-sample holdout group where all their tasks are fixed (all respondents in the held-out group receive a single version of the questionnaire, typically with 8 to 15 choice tasks).
answered Nov 14, 2013 by Bryan Orme Platinum Sawtooth Software, Inc. (140,065 points)
edited Nov 14, 2013 by Bryan Orme