Modeling Interaction Effects with CVA

This is an advanced area. We assume the reader understands experimental design principles, is familiar with dummy-coding for part-worth attributes, is familiar with tuning simulators via the Exponent to best fit holdout data, and is able to do some additional data processing.

Background

As a traditional full-profile conjoint approach, CVA is a "main effects only" model, and assumes there are no interactions among attributes. Many conjoint practitioners agree that one must remain alert for the possibility of interactions, but that it is usually possible to choose attributes so that interactions will not present severe problems. Like other conjoint methods, CVA can deal with interactions in a limited way by defining composite variables. For example, we could deal with an interaction between car color and body style by cross-classifying the levels:

Red Convertible

Black Convertible

Red Sedan

Black Sedan

However, if the attributes in question have many levels, or if an attribute (such as price, for example) is suspected of having interactions with many others, then composite attributes will not be enough. In that case too many parameters must be estimated to permit analysis with a purely individual-level model, and the most common solution is to evaluate interactions by pooling data from many respondents. For example, CBC (Choice-Based Conjoint) is considered a stronger approach for estimating interaction terms. It can automatically estimate interaction effects between all attributes, not necessarily just those that the analyst identifies prior to designing the study.

CVA can accommodate up to 15 levels for each attribute. Therefore, you can use the composite attribute approach to model interactions between two attributes having as many as 5 and 3 levels, respectively (such that the product of the levels between the attributes does not exceed 15).

---

Designing CVA Studies for Limited Interactions

In the previous section of this documentation, we introduced the advanced design strategy of conditional pricing tables. We warned that these should be built with near-proportional pricing across the price tiers. We also warned that specifying conditional pricing tables might increase the likelihood that interactions would be needed to properly model the data. The following instructions provide a framework for advanced analysts to develop designs that formally support estimation of limited interactions and also examine whether those interaction terms are useful for a given data set. Rather than use the Conditional Pricing facility in CVA (which assumes main effects), this example also assumes customization of price ranges for each brand. For this example, we will assume a conjoint study where two of the attributes are brand (4 levels) and price (3 levels). (Note that the steps below may be used for any two attributes, whether involving price or not.)

1. When designing the study, create a single composite attribute reflecting all combinations of brand and price (4 x 3 = 12 levels). This will increase the number of parameters that need to be estimated relative to the main effects plan. With main effects, there are (4-1) + (3-1) = 5 parameters to fit to account for the main effects of brand and price. With the composite factor accounting for all combinations of brand and price, there are 12 - 1 = 11 parameters to fit. Thus, the interaction design increases the number of parameters to estimate by 11 - 5 = 6. Given the standard rule-of-thumb for designing CVA questionnaires (field twice as many questions as parameters to estimate), this will increase the number of cards that each respondent should evaluate by 2 x 6 = 12 cards. Of course, with CVA/HB estimation, you may do just as well with fewer cards than this recommendation.

2. Add CBC-looking fixed holdout questions at the end of the questionnaire (as Select-type or Grid questions). Three or four scenarios, each having 4 concepts (one for each brand) should be sufficient. Don't include a "None" option. These holdouts are useful not only for tuning the CVA part-worths to have the appropriate scale factor for predicting choice probabilities, but for investigating whether interaction effects improve the accuracy of the model.

3. Estimate the part-worth utilities (preferably with CVA/HB).

4. Plot the mean part-worths for the brand x price composite factor (pseudo demand curves). If reversals are present, you may consider re-running the estimation with constraints in place. Examine the resulting curves for face validity.

5. Specify the holdout scenarios each as a separate simulation scenario within the market simulator. Tune the Exponent setting (not independently for each simulation scenario, but using the same scale factor across all simulation scenarios) so that the simulated shares most closely fit the actual observed choices for the holdouts (Mean Absolute Error, or Mean Squared Error).

6. Repeat the analysis using main effects only. To do this, save a copy of the study within SSI Web and work from this duplicate study. Export the original design to a .csv file. Modify the design file to separate the brand x price composite factor into two attributes. Recode the level numbers accordingly to reflect main effects. Modify the attribute list to separate brand and price as attributes, to match your modified design. Import the recoded design.

7. Import the respondent data as if the study were conducted via paper-and-pencil.

8. Repeat steps 3 through 5 with the main effects model. Compare the plot of utilities and also the fit to holdouts. If interaction effects have greater face validity (visual plot) and noticeably stronger fit to holdouts, then this is evidence that interaction effects are useful. If the two models appear equally useful, we suggest using the more parsimonious model (main effects).