Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Interaction effects in Logit (and also LCA?)

Hello,

According to the Interaction Search, some interactions are significant and therefore I can add them to the Logit Analysis.

My question: Should I also add those interactions to the LCA? If I do not add those interactions to the LCA, the choice for optimal amount of segments (looking at CAIC, BIC, LL, and Chi-Square) is very clear, whereas if I do add them in the LCA, the choice is extremely unclear.

Is it okay for me to include those interaction effects in the Logit, but to only use main effects for the LCA? Or does this invalidate the analysis in some way?

Thank you in advance for the help!

Floor
asked Oct 17, 2018 by Floor (310 points)
edited Oct 17, 2018 by Floor

1 Answer

+1 vote
It is very rare to use aggregate logit for your final market simulation model and for the final utility results.  Most of the time, aggregate logit is a tool for quickly assessing the basic preference structure for the market as a whole and for getting a quick look at the overall precision of the model.  Indeed, it sounds like you are moving from pure aggregation toward a segment-based model (latent class analysis).

Usually, interactions that are seen in aggregate logit models tend to go away once you conduct HB analysis (individual-level estimation), since most of the time those interaction effects seen in the aggregate model are due to unrecognized heterogeneity.

Now, latent class analysis is in the middle between aggregate models and individual-level models (in terms of modeling heterogeneity).  If you have a fairly low-dimension latent class solution (such as 2 to 4 groups), it's quite probable that the interaction effects (seen as significant in aggregate logit modeling) could provide significant fit to the latent class model that would be useful for out-of-sample predictions.  But, if we have high-dimension latent class solutions, such as 8-10 classes or more, then my guess is that the additional heterogeneity captured in the main effects across the multiple latent class segments may near fully explain the interaction effects in many if not most cases.

Fitting interactions is a tradeoff between fitting vs. overfitting the data.  Although an interaction may lead to p<0.05 in terms of internal fit to the data, the big question is whether adding the interaction effect to the model would improve predictions for new data situations (out of sample predictions, such as predicting what real customers would buy in the real marketplace).
answered Oct 17, 2018 by Bryan Orme Platinum Sawtooth Software, Inc. (162,715 points)
Thank you for your clear answer!
...