Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Comparing HB validity to other models

Dear all,

I conducted a HB analysis and used it in the choice simulator to simulate first choices which I used to calculate hit rates for the hold-out , MAE and RMSE.
Now, to show that my model is well suited for prediction I compared it to the null model. I also tried a few different settings  like covariates for HB and compared them to the orignal model but they all had quite similar results.

Does it make sense to compare the mentioned figures to those obtained by using logit utilities instead of my HB estimates in the choice simulator to show the superiority of the HB model? Obviously, under the first choice rule, logit estimates would predict every respondent to choose the same stimulus.

Thank you and best regards
asked Jul 18, 2017 by Colin (305 points)
retagged Jul 18, 2017 by Walter Williams

1 Answer

0 votes
First choice rule is not appropriate for aggregate (pooled) logit models.  Rather, use the "share of preference" logit rule.
answered Jul 18, 2017 by Bryan Orme Platinum Sawtooth Software, Inc. (131,390 points)
thanks! Does it make sense though to compare SOP logit hit rates and errors to first choice HB hit rates? Or is the comparison to the null model sufficient?