Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Comparing HB maxdiff results: same tasks & model, different samples

I have a question about comparing importance scores from HB analysis of 2 separate samples (A, B) completing same tasks, but models estimated separately.

Take the example of mean scores for items X & Z, with the HB scores scaled to sum to 100 in both cases:

•    For group A, the mean importance scores are X[12] and Z[3]
•    For group B, the mean importance scores are X[12] and Z[6]

My understanding is that I can argue that
1.    for A: X is 4 times more important than Z
2.    for B: X is 2 times more important than Z

What about comparisons between groups A and B?
Can I infer that:
3.    Z is more important for B than for A?
4.    Z is twice as important for B than for A?

Although in both samples/models the scores are scaled to sum to 100, the scale factor could be different in the 2 samples which suggests I can’t make comparisons 3 or 4 - I can compare ratios between models, but not absolute values. Or is that overly cautious?

Can anyone advise on (in)appropriate inference?

thanks
Dan
asked Aug 29, 2013 by Dan

1 Answer

+1 vote
By scaling your utilities to sum to 100% you've already (sort of) taken scale differences into account.  I think you could argue this version of #3 above:  "Z is more important, relative to X, for B respondents than for A respondents."  

I'd be a little more cautious about claiming that Z is more important in the absolute since we're not measuring the absolute importance of either X or Z.
answered Aug 29, 2013 by Keith Chrzan Platinum Sawtooth Software, Inc. (58,850 points)
...