Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

ACBC & Interaction effects

Dear all,

since I have a little trouble making sense of the role of interaction effects in my ACBC study, I´d be very grateful for your help.

1. First-order interaction effects
I can´t get my head around interaction effects between attributes, e.g. there is always "size" and "color" (attributes) involved with many different interactions based on the combination of their levels.  "Size XL" and "color pink" (levels) on the other hand might have a specific interaction, I can understand.
* How do you interpret first-order interaction effects between attributes?
* Are the interactions between levels called "second-order interaction effects"?

2. Identification of relevant interaction effects
Regarding first-order interactions I have e.g. one interaction that is highly significant (p-value: 0,000000171), but that only marginally improves the model over the main effects (0,12%). I would therefore argue that in order to avoid overfitting, the interaction effect is ignored.
* Does that make sense?
* How is the "Gain in Pct. Cert. over Main Effects" calculated?
* Is it possible to derive any statements about the relevance of interactions between levels from the first-order interaction effects?
I understand that with ACBC the best approach to assessing the model´s improvement through interaction effects is based on predictive hit rates. I therefore derived some potential interactions between levels from theory and would like to test, if their consideration in the model improves predictive hit rates.

3. Calculation of total utilities including interaction effects between levels
I understand that the formula to calculate total stimuli utilities is as follows:
Utility(Att1_level1 x Att2_level1) =
main_effect(Att1_level1)+main_effect(Att2_level2)+interaction_effect (Att1_level1 x Att2_level1)
The CBC/HB allows me to calculate the interactions between levels. In the file ACBCExport_utilities.csv respectively ACBCExport_alpha.csv there is only one sheet that contains raw utilities.
* How do I aggregate the main effects and interaction effects?
* How do get the Zero-Centered Diffs for the main effects and interaction effects, as well as the importances?

Thank you very much for your support!

Best
Christian
asked Jul 5, 2017 by Chris

1 Answer

+1 vote
Chris,  you have a bunch of questions, so I think the clearest way to answer them will be with inline CAPS:

1. First-order interaction effects
I can´t get my head around interaction effects between attributes, e.g. there is always "size" and "color" (attributes) involved with many different interactions based on the combination of their levels.  "Size XL" and "color pink" (levels) on the other hand might have a specific interaction, I can understand.
* How do you interpret first-order interaction effects between attributes?  INTERACTION MEANS THAT THE WHOLE "BLUE + XL," IS DIFFERENT THAN THE SUM OF ITS PARTS "BLUE" + "XL."  SO I LIKE CHEESE AND I LIKE CHOCOLATE, BUT CHEESY CHOCOLATE I DO NOT LIKE - MY LIKING FOR THE WHOLE IS DIFFERENT THAN THE SUM OF MY LIKING FOR THE PARTS.
* Are the interactions between levels called "second-order interaction effects"? NO.  SECOND ORDER INTERACTIONS ARE ALSO CALLED THREE-WAY INTERACTIONS AND INVOLVE 3 VARIABLES, NOT TWO.

2. Identification of relevant interaction effects
Regarding first-order interactions I have e.g. one interaction that is highly significant (p-value: 0,000000171), but that only marginally improves the model over the main effects (0,12%). I would therefore argue that in order to avoid overfitting, the interaction effect is ignored.  EVEN IN ANOVA MODELS, THE CLASSIC STATISTICAL MODEL FOR INTERACTIONS, MAIN EFFECTS USUALLY ACCOUNT FOR MORE THAN 97% OF VARIANCE.  SO FINDING THAT YOUR INTERACTIONS AREN'T IMPROVING FIT MUCH ISN'T A SURPRISE AND DOESN'T MEAN THAT THEY'RE NOT IMPORTANT.  
* Does that make sense?
* How is the "Gain in Pct. Cert. over Main Effects" calculated? IT'S THE DIFFERENCE IN PERCENT CERTAINTY (MCFADDEN'S RHO-SQUARED) CALCULATED WITH AND WITHOUT THE INTERACTIONS.
* Is it possible to derive any statements about the relevance of interactions between levels from the first-order interaction effects?  IF THEY SIGNIFICANTLY IMPROVE MODEL FIT THEN THAT WOULD BE A PRETTY IMPORTANT STATEMENT, I THINK.  
I understand that with ACBC the best approach to assessing the model´s improvement through interaction effects is based on predictive hit rates. I therefore derived some potential interactions between levels from theory and would like to test, if their consideration in the model improves predictive hit rates.  THIS MAKES GOOD SENSE.  HIT RATES ARE PROPORTIONS, SO YOU CAN USE A TEST OF DEPENDENT PROPORTIONS TO SEE IF ADDING THE INTERACTIONS IMPROVES YOUR HIT RATE.  

3. Calculation of total utilities including interaction effects between levels
I understand that the formula to calculate total stimuli utilities is as follows:
Utility(Att1_level1 x Att2_level1) =
main_effect(Att1_level1)+main_effect(Att2_level2)+interaction_effect (Att1_level1 x Att2_level1) THIS IS CORRECT
The CBC/HB allows me to calculate the interactions between levels. In the file ACBCExport_utilities.csv respectively ACBCExport_alpha.csv there is only one sheet that contains raw utilities.  I THINK THEY ARE IN THE ALPHA FILE, BUT IF YOU CANNOT FIND THEM, PLEASE CONTACT SUPPORT@SAWTOOTHSOFTWARE.COM FOR HELP.
* How do I aggregate the main effects and interaction effects? THROUGH ADDITION, AS ABOVE.
* How do get the Zero-Centered Diffs for the main effects and interaction effects, as well as the importances?  I AM NOT SURE I UNDERSTAND THE QUESTION.
answered Jul 5, 2017 by Keith Chrzan Platinum Sawtooth Software, Inc. (62,500 points)
...