Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Does it mean CBC/HB automatically remedies any version imbalance or that we can ignore it as negligible?

Hi Keith

Thanks a lot for prompt and extensive reply - much appreciated! I am a bit unclear however about answers to the questions I asked.

Kindly confirm if I got you right that  CBC/HB to some extent remedies version imbalance (owing to multinomial logit modelling which provides control for slight imbalance in designs). If so, could you elaborate more on what you mean by "slight imbalnce"? Woudl you say the imbalance I observed for n=393 is still slight? How to discern when to accept it and when to say 'no' to a request for subgroup analysis?

Or maybe it's the case that CBC/HB is not addressing the imbalance issue, but only because it is simply irrelevant (as per your research published in 2013)? In that light, I guess that if we are extremely unlucky and all respondents in a subgroup see only 1 of 10 versions, the maximum impact on item utilities would be always below 5% and probably around 2%, which is surely negligible.

Kindly share your thoughts.
related to an answer for: Maxdiff design imbalance - how to remedy?
closed with the note: Issue resolved, thanks a lot!
asked Nov 21, 2016 by Piotr (185 points)
closed Nov 22, 2016 by Piotr

1 Answer

+1 vote

In a multivariate model like regression or logit, the model gives us coefficients (or utilities) that isolate the effect of each variable controlling for the effects of all the other variables.  The imbalance you describe is just the kind of effect that the multivariate technique will control.  We know that collinearity can be a problem for multivariate models, but that happens when the correlations between variables are very high, and the imbalances you describe will result in correlations low enough not to have to worry about collinearity, right?  That's the sense in which I consider them slight.  If we were computing the utilities directly via some transformation of counts we might be in trouble, but with the statistical model controlling for the slight imbalances we eliminate that source of trouble.  

CBC/HB is NOT ignoring the imbalance at all.
answered Nov 21, 2016 by Keith Chrzan Platinum Sawtooth Software, Inc. (92,075 points)