Using Constraints in CVA/HB

Conjoint studies frequently include product attributes for which almost everyone would be expected to prefer one level to another. However, estimated part-worths sometimes turn out not to have those expected orders. This can be a problem, since part-worths with the wrong relationships (especially if observed at the summarized group level) are likely to yield nonsense results and can undermine users' confidence.

CVA/HB provides the capability of enforcing constraints on orders of part-worths within attributes. The same constraints are applied for all respondents, so constraints should only be used for attributes that have unambiguous a priori preference orders, such as quality, speed, price, etc.

Evidence to date suggests that constraints can be useful when the researcher is primarily interested in individual-level classification or the prediction of individual choices, as measured by hit rates for holdout choice tasks. However, constraints appear to be less useful, and indeed can be harmful, if the researcher is primarily interested in making aggregate predictions, such as predictions of shares of preference. Most research is principally concerned with the latter. Another concern is that constraints can bias the apparent importances of constrained attributes in market simulations, relative to unconstrained attributes.

If you check the Use Constraints box on the CVA/HB Settings dialog, CVA/HB automatically constrains the part-worths if you have specified a priori orders (e.g. Best to Worst or Worst to Best) when you specified your attributes, or if you specify additional utility constraints by clicking Edit... and supplying additional constraints using the Custom Constraints dialog.

CVA/HB employs a technique called Simultaneous Tying. In a paper available on the Sawtooth Software Web site (Johnson, 2000), the author explored different ways of enforcing constraints in the HB context. He found the method of simultaneous tying to perform best among the techniques investigated.

Simultaneous tying features a change of variables between the "upper" and "lower" parts of the HB model. For the upper model, we assume that each individual has a vector of (unconstrained) part-worths, with distribution:

ß_i ~ Normal(á, D)

where:

ßi = unconstrained part-worths for the ith individual

á = means of the distribution of unconstrained part-worths

D = variances and covariances of the distribution of unconstrained part-worths

For the lower model, we assume each individual has a set of constrained part-worths, b_i where b_i is obtained by recursively tying each pair of elements of ß_i that violate the specified order constraints.

With this model, we consider two sets of part-worths for each respondent: unconstrained and constrained. The unconstrained part-worths are assumed to be distributed normally in the population, and are used in the upper model. However, the constrained part-worths are used in the lower model to evaluate likelihoods.

We speak of "recursively tying" because, if there are several levels within an attribute, tying two values to satisfy one constraint may lead to the violation of another. The algorithm cycles through the constraints repeatedly until they are all satisfied.

When constraints are in force, the estimates of population means and covariances are based on the unconstrained part-worths. However, since the constrained part-worths are of primary interest, we plot the constrained part-worths to the screen. Only the constrained part-worths are saved to the utility run for use in the market simulator.

When constraints are in place, measures of fit (average r-squared) are decreased. Constraints always decrease the goodness-of-fit for the sample in which estimation is done. This is accepted in the hope that the constrained solution will work better for predictions in new choice situations. Measures of scale (Avg. Variance and Parameter RMS), which are based on unconstrained part-worths, will be increased.