Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Model pct. certainty from individual RLH versus software output

Dear all,

I computed the model pct. certainty by converting the individual RLHs into total LL and applying the formula illustrated in related posts.
I found that the value thus obtained differs from the final output provided by the software under the "Current" heading of the estimation monitoring window of the CBC/HB tool. Since I am using pct. certainty for model comparison and those values point to different conclusions, I was wondering which of the two (computed versus given) I should use.

Thank you!
related to an answer for: Converting to pct. Certainty
asked Aug 16, 2016 by lotika (455 points)
retagged Aug 16, 2016 by Walter Williams

1 Answer

+2 votes
One thing to remember is that HB's on-screen output regarding RLH and Pct.Cert. is based on the draws it is working from, not the average of the draws which are a post hoc procedure to summarize the "average" utilities (the point estimates) for each respondent.  Also, the RLH reported in the point estimates file is actually the average of the RLH fit across the draws for each respondent!  This has fooled me in the past, when I naturally thought the RLH reported in the point estimates file would be the RLH based on the point estimates.
answered Aug 16, 2016 by Bryan Orme Platinum Sawtooth Software, Inc. (138,915 points)
Reporting RLH and Percent Certainty