Let's assume we are done with a MaxDiff HB estimation and a final set of betas for each person for 10 MaxDiff items.

Could you please share how Root Likelihood is calculated for one person?

Thanks a lot!

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Let's assume we are done with a MaxDiff HB estimation and a final set of betas for each person for 10 MaxDiff items.

Could you please share how Root Likelihood is calculated for one person?

Thanks a lot!

+1 vote

RLH is computed as follows:

For each choice task, we divide the tasks into two separate tasks: choice of best, and choice of worst. So, if there are 10 total MaxDiff tasks, there will be 20 actual tasks used to compute the RLH.

It all hinges on the logit equation, or Pa= e^Ua/(e^Ua+e^Ub+...+e^Ud), where Pa is the probability of picking item a from the set of items shown to the respondent, Ua through Ud are the utilities of items a through d that are shown in the current choice set. Typically, only 4 or 5 items are shown per choice set in MaxDiff.

For the "best" tasks, use the logit equation to compute the likelihood (given the current utilities) that the person would pick the item that they actually picked as best. For example, the respondent actually picked item 2 and the logit equation calculated the likelihood of picking item 2 as 0.4.

For the "worst" tasks, first multiply all the utilities by -1, then use the logit equation to compute the likelihood (given the current utilities) that the person would pick the item that they actually picked as worst. For example, the respondent actually picked item 1 as worst and the logit equation calculated the likelihood of picking item 1 as 0.8.

Now, you have 20 probabilities ranging from 0 to 1.0. Take the geometric mean of them.

Note that these RLHs are computed for each used draw, so the final RLH is the average of the RLHs for the used draws.

For each choice task, we divide the tasks into two separate tasks: choice of best, and choice of worst. So, if there are 10 total MaxDiff tasks, there will be 20 actual tasks used to compute the RLH.

It all hinges on the logit equation, or Pa= e^Ua/(e^Ua+e^Ub+...+e^Ud), where Pa is the probability of picking item a from the set of items shown to the respondent, Ua through Ud are the utilities of items a through d that are shown in the current choice set. Typically, only 4 or 5 items are shown per choice set in MaxDiff.

For the "best" tasks, use the logit equation to compute the likelihood (given the current utilities) that the person would pick the item that they actually picked as best. For example, the respondent actually picked item 2 and the logit equation calculated the likelihood of picking item 2 as 0.4.

For the "worst" tasks, first multiply all the utilities by -1, then use the logit equation to compute the likelihood (given the current utilities) that the person would pick the item that they actually picked as worst. For example, the respondent actually picked item 1 as worst and the logit equation calculated the likelihood of picking item 1 as 0.8.

Now, you have 20 probabilities ranging from 0 to 1.0. Take the geometric mean of them.

Note that these RLHs are computed for each used draw, so the final RLH is the average of the RLHs for the used draws.

...

So, as you said, Sawtooth reports as the last RLH the average of the RLHs for the used draws. That would imply that if I use the last 1,000 draws and report point estimates that are averages of those, then I could take the point estimates I am reporting (using) and calculate the RLH for them. I wonder if they'll be the same as the averages of the 1000 last RLHs.