Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Interpretation of a attribute if Count is not significant

Good Morning,

I have a question regarding the interpretation of my analysis. I have conducted a alternative specific CBC. One alternative specific attribute is not significant I my Counts output.

My question is: do I still use it in my HB-output or do I simply not interpret it if it is not significant?

Thanks for the help.

asked Apr 9, 2018 by anonymous

1 Answer

0 votes
Counts give you a good hint about what you'll see when you start building formal statistical models.  But counts are NOT those formal statistical models, so we don't use the results use the results of their stat tests to make big decisions.  I would include that attribute in your HB analysis and use those models to make decisions about the final structure of your model.

It sometimes happens that an attribute doesn't look like a significant predictor because of respondent heterogeneity.  For example, say an attribute is color, orange or blue.  If half your respondents love orange and hate blue while the other half hate orange and love blue, then in the aggregate analyses (counts, logit) it will look like color doesn't matter.  When you run HB you might see that orange and blue still have (average) utilities close to zero but that color had a significant level of importance.  This is because you have a lot of blue-lovers and a lot of orange-lovers and all that love cancels out in the aggregate.  If you don't include color in your HB model you'll never catch this.

Of course it could also happen that color isn't important in the aggregate and then it isn't important at the respondent level either.  Then I suppose you could drop it from the HB analysis, but most folks would probably still keep it in because if it's that unimportant it is unlikely to change things by being left in, while it might raise questions if you leave it out.
answered Apr 9, 2018 by Keith Chrzan Platinum Sawtooth Software, Inc. (85,325 points)
Thank you very much for the answer. It helped!