Comparing HB validity to other models

Question

Dear all,

I conducted a HB analysis and used it in the choice simulator to simulate first choices which I used to calculate hit rates for the hold-out , MAE and RMSE.
Now, to show that my model is well suited for prediction I compared it to the null model. I also tried a few different settings  like covariates for HB and compared them to the orignal model but they all had quite similar results.

Does it make sense to compare the mentioned figures to those obtained by using logit utilities instead of my HB estimates in the choice simulator to show the superiority of the HB model? Obviously, under the first choice rule, logit estimates would predict every respondent to choose the same stimulus.

Thank you and best regards
Colin

Answer 1

First choice rule is not appropriate for aggregate (pooled) logit models.  Rather, use the "share of preference" logit rule.

Comments

thanks! Does it make sense though to compare SOP logit hit rates and errors to first choice HB hit rates? Or is the comparison to the null model sufficient?