Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Profiling using HB

I have a question about profiling on demographics in HB.

Sawtooth software can be used to get an impression of the differences in utilties on profiling questions for example education levels. In college is said, if you want to report these differences, you have to check their significance in SPSS. Sawtooth can't give you this.

My question is: how should I do this? I want to do an ANOVA or F-test for the significance of the total attribute. I only have utilities on levels but I need scores per attribute. I do have importance scores per attribute, should I use these?

Is it correct that the next step is to check the significance of the differences (per two categories) in an independent sample t-test?

It is  perhaps more a question about SPSS but I hope someone here can help me
asked Jan 4, 2018 by anonymous

1 Answer

0 votes
I think you don't want to use the importance scores for this analysis - they would be appropriate for answering some questions, but probably not this one.  

When comparing utilities in this way make sure to use the zero-centered diffs (ZCD) transformed utilities and not the raw HB utilities, as the former include our best effort to make utilities from different respondents, who have different levels of response error, comparable.  

In academic papers you'll see authors run models with and without a given demographic variable (or with and without the interactions of design variables with the demographic variable) to see if including the demographic improves model fit.  That's a lot of complicated analysis, so folks outside of academia rarely do it.  Much more common is the approach you're considering, which is to do ANOVAs, one per attribute level, which, you're right, doesn't exactly answer the question of whether the entire attribute differs significantly.  

What I do in this case is run all the ANOVAs of potential interest, then do an appropriate correction for multiple tests (I like the Benjamini-Hochberg procedure for preventing false discovery) and then if any level from an attribute is significant we can say that the attribute is significantly different by groups.
answered Jan 4, 2018 by Keith Chrzan Platinum Sawtooth Software, Inc. (66,225 points)
Thanks for the answer.
I understand but if I do it this way, I have many attributes who have 1 or 2 singificant levels and not all (4 levels in total).
Is the total attribute only significant if all levels are significant?

Furthermore in case I first conclude the total attribute is not significant, may I no longer compare the differences between groups on the significant levels of that specific attribute?
I think the attribute is significant if any of its levels are IF you've taken into account multiple comparison error.