Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to see whether covariates have significance in terms of choice prediciton?

As Walter (thank you) told us, someone should add 'wisely chosen' covariates to estimate utilities within CBC/HB.  These covariates "should have significance in terms of choice prediction".  But how can I prove/check if they really improve the choice prediction/the quality of model?
related to an answer for: Covariates in CBC/HB
asked Sep 18, 2014 by Hansson (270 points)
edited Sep 18, 2014 by Walter Williams

1 Answer

+1 vote
A recent (2010 paper at the Sawtooth Software Conference) featured an  investigation of numerous types of covariates, different sorts (demographics, benefit segments, or past behavior/purchase) for 10 different commercial CBC datasets.  The authors (Kurz/Binner) found very few opportunities for significant improvement in predictive validity for any of the covariates they investigated, for any of the data sets.

The authors used held-out tasks to judge predictive validity.

So, if your desire is to improve predictive validity with covariates in HB, I think you will be facing a difficult time to find this.

Covariates, however, can improve the characterization of heterogeneity across the sample (involve less Bayesian smoothing toward the population means).  Covariates provide a theoretically more sound (according to the Bayesians) for comparing groups of respondents on the parameters.  Covariates also provide a way to better estimate individual-level models when some groups of respondents have been significantly oversampled in the sampling plan.
answered Sep 18, 2014 by Bryan Orme Platinum Sawtooth Software, Inc. (174,215 points)