Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Analyze Segments in HB estimations (importances/part-worths)

Dear community,
for my analysis of CBC-HB data I want to check whether demographics (age, gender, income, work status, ...)  lead to different results regarding avarage importance of attributes and  part worths.

Besides gender these demographics consist of several segments.
E.g. age: 16-25; 26-35; 36-45; 46-55; 56-65; >65 (=6segments)
Do I compare a segment with another segment or compare a segement with the  overall sample (without segmentation)?
Which statistical test can I use for that?
Are t-tests the means to identify significant differences?
Does this testing also work for part-worths? Can you measure differences in amplitude and value? Or are simple observations enough for scientific papers?
Thank you in advance for the answers. Fast answers are highly appreciated.
asked Jul 20, 2018 by botmar (395 points)

1 Answer

+1 vote
You have many questions, so it's probably clearest to answer them in turn:

Q:  Do I compare a segment with another segment or compare a segment with the  overall sample (without segmentation)?  Which statistical test can I use for that?
A:  What would you do if you were comparing means of rating scales here?  If your hypothesis is that Segment A differs from Segment B, then that's a two group test and you run it as an independent groups t-test.  If your hypothesis is that Segment A is different from the total sample then it's A vs total sample, but then you have to use a t-test that corrects for the partial overlap in your samples.  If it's that Segment A is different from the non-A parts of the sample, then it's a two-group independent t-test again.  If you want to know if there are any differences among segments A, B, C, D then you could run an ANOVA for the omnibus test and follow up with post hoc tests appropriate for ANOVA.

Your remaining questions to be answered in comments below for clarity.
answered Jul 20, 2018 by Keith Chrzan Platinum Sawtooth Software, Inc. (75,450 points)
Q:  Are t-tests the means to identify significant differences?
A:  Yes, they are a quick and dirty way to do significance testing.  There are perhaps more appropriate Bayesian tests, but folks are familiar with t-tests and they'll work in a pinch.
Q:  Does this testing also work for part-worths? Can you measure differences in amplitude and value?
A:  Yes, with some caveats; for part worths you probably want to use zero-centered diffs and not raw utilities (because of a possible logit scale factor confound)
Q:  Or are simple observations enough for scientific papers?
A:  It depends on the field you are publishing in and the standards and traditions of its journals.  I would take a look at the articles in the journal you want to publish in and see what norms the journal exhibits.  I would also check the guidelines for authors as these vary from one field to the next and often from one journal in a field to the next.
@Q:  Are t-tests the means to identify significant differences?
A:  Yes, they are a quick and dirty way to do significance testing.  There are perhaps more appropriate Bayesian tests, but folks are familiar with t-tests and they'll work in a pinch.
What bayesian tests are there for that matter?
...