Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Best method for MaxDiff HB analysis for various sub-groups in sample


I conducted a MaxDiff and now want to look deeper into various sub-groups within my sample. By comparing various methods I came down to 4 options:

1. Running HB analysis on the whole sample, split the data file afterwards into the sub-groups and use the averages for each sub-group. However, I realized that this method would calculate the results "borrowing" information from the whole population, so larger sub-groups would dominate and influence the results for smaller groups.
2. Running HB analysis on whole sample but adapt the prior variance in the advanced settings in order to shift more weight to the individual responses and away from the whole population. --> Which variance would be suitable in that case, as the value can be chosen from 0.1 to 100?
3. Enter co-variates for the various sub-groups before running the HB analysis. However, I learned that this slows down the process and doesn't necessarily enhance the results.
4. Split the file manually and upload the sub-group data separately to run separate HB analyses on each file. I think that worked in Sawtooth version 6, but I didn't find a way to do it in version 8, as this version seems to pull the data for HB analysis directly from the online database.

Does anyone have an idea with which option the data quality would be best or whether there are other possibilities?

Thanks a lot in advance!
Best regards,
asked Mar 24, 2017 by Sophie
retagged Mar 24, 2017 by Walter Williams

1 Answer

0 votes
Option 3: I like taking all the data together and running MaxDiff/HB with covariates.  And, if the run becomes too long (often does when coded covariates as independent variable columns exceeds 8 or 10), then I would try running 3 to 6 MaxDiff runs with fewer covariates in each run, then average the scores across the multiple runs.  (That's the ensemble approach).  Good covariates that are predictive of preferences and also the covariates describing the cuts of the data you are most interested in making would be the ones I'd target.

If you have time, it's also a good idea to use the CBC/HB Model Explorer to search for the prior variance and D.F. setting that improves the fit to the data without overfitting.  This can take 3 to 18 hours for typical MaxDiff datasets in practice.
answered Mar 24, 2017 by Bryan Orme Platinum Sawtooth Software, Inc. (169,815 points)