Sawtooth Software: The Survey Software of Choice

HB Improves ACA Part Worth Estimation

According to a 1997 industry survey conducted by Wittink, Vriens and Huber, ACA is the most widely used methodology in the world for conjoint analysis. Given its popularity, it is not surprising that ACA has been widely scrutinized and been the subject of a great deal of debate. Most notably, in a 1991 Journal of Marketing Research article by Green, Krieger and Agarwal, ACA was criticized because of potential scale incompatibilities between the self-explicated priors and conjoint trade-off sections of the interview.

ACA version 4 was released shortly after the 1991 JMR article. It used a slightly different technique from earlier ACA versions for combining self-explicated priors and conjoint pairs information. Although Version 4 improved the way ACA combined the two sections, it was still necessary to assume that the self-explicated priors were properly scaled.

In 1995, Allenby, Arora and Ginter published an article also in the Journal of Marketing Research reporting improvements for ACA through Hierarchical Bayes estimation. Allenby and a number of co-authors' collective work on HB methods has been ground breaking and important. Now four years later, we have developed the ACA Hierarchical Bayes Estimation module and have documented its benefits.

ACA/HB is a post-data collection module that reads data from studyname.ACD files and computes utilities, saving them in a new ASCII file that has the same format as the familiar .UTL files that you use with the ACA simulator. The process is so automatic, even a data processor with no analytic experience can adequately run ACA/HB by just using the defaults and pressing a few keystrokes.

ACA/HB provides two major benefits:

  1. The ACA/HB module improves the quality of each individual's utility estimates by "borrowing" information from other individuals. This translates to more accurate predictions of both individual choices and share estimations. We have tested the results on dozens of real and synthetic data sets. HB at least matches and usually beats traditional ACA utility estimation.
  2. ACA/HB provides a more theoretically sound way of combining data from the self-explicated and paired comparison sections of the interview. Because the priors information can be applied in a purely ordinal way as constraints, it entirely avoids the issue of combining two separate sets of metric dependent variables with potentially different variances.

Not only is the technique more defensible, but the results are generally better. Notably:

  • ACA/HB utilities are less biased toward equal utility increments spacing between levels as compared to ACA v4.
  • ACA/HB importances reflect slightly more discrimination than under ACA v4.
  • ACA/HB does a better job of estimating utilities for the levels not taken forward into pairs when using "Most Likelies" and "Unacceptables."

In addition to those benefits, ACA surveys can now be shorter. ACA/HB does not require the calibration concept data (unless you want to calibrate the data for purchase likelihood simulations). Therefore, you can cut this sometimes confusing section from your ACA surveys. Rather than reducing the length of the interview, you might decide instead to add a few more pairs questions to further stabilize utilities.

We think that the ACA/HB module will prove a valuable tool for ACA users. For a typical data set, with a few hours of extra computational time one can significantly improve the quality of the ACA data set. This new development brings ACA up-to-date with the most cutting edge estimation techniques being applied today. It also provides a way of combining ACA's self-explicated data with answers from paired comparison questions without having to make any assumptions about the scale of the self-explicated data.

For more details on ACA/HB, please download the "ACA/HB Technical Paper" from the Technical Papers library at www.sawtoothsoftware.com.