Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Total LL for split samples

So when running the HB on split samples, can the total LL (computed as the sum across respondents of the LN(RLH)*#tasks) be summed across the two sub-samples to be compared with the total LL across respondents from the pooled model? Or is this conceptually wrong?

Thanks in advance!
related to an answer for: Follow up on LL
asked May 17, 2015 by anonymous

1 Answer

0 votes
This is the sort of thinking one brings from Latent Class, where the goal is to maximize the LL.

HB does not have as a goal to maximize the LL of the fit to respondents' choices.  In fact, it purposefully gives up a lot of fit at the individual level to the individual's choices in favor of making sure the respondents seem to be drawn from higher density areas of a multivariate normal distribution, according to population means and covariances.  

If your two groups have quite different preferences, then the compromise toward population means and variances will yield higher individual-level fit to respondents' choices than when using a pooled model.  So, it's probably the case that you will obtain higher individual-level fit on average across respondents when using a segmented model.  However, segmented models also have lower sample size within each HB model to stabilize the means and especially the covariances.  So, the better individual-level fit isn't necessarily an indication of better quality.  There may be some overfitting, for example.
answered May 18, 2015 by Bryan Orme Platinum Sawtooth Software, Inc. (170,115 points)