How is RLH for MaxDiff calculated

Question

Hello!

Let's assume we are done with a MaxDiff HB estimation and a final set of betas for each person for 10 MaxDiff items.
Could you please share how Root Likelihood is calculated for one person?

Thanks a lot!

Answer 1

RLH is computed as follows:

For each choice task, we divide the tasks into two separate tasks: choice of best, and choice of worst.  So, if there are 10 total MaxDiff tasks, there will be 20 actual tasks used to compute the RLH.

It all hinges on the logit equation, or Pa= e^Ua/(e^Ua+e^Ub+...+e^Ud), where Pa is the probability of picking item a from the set of items shown to the respondent, Ua through Ud are the utilities of items a through d that are shown in the current choice set.  Typically, only 4 or 5 items are shown per choice set in MaxDiff.

For the "best" tasks, use the logit equation to compute the likelihood (given the current utilities) that the person would pick the item that they actually picked as best.  For example, the respondent actually picked item 2 and the logit equation calculated the likelihood of picking item 2 as 0.4.

For the "worst" tasks, first multiply all the utilities by -1, then use the logit equation to compute the likelihood (given the current utilities) that the person would pick the item that they actually picked as worst.  For example, the respondent actually picked item 1 as worst and the logit equation calculated the likelihood of picking item 1 as 0.8.

Now, you have 20 probabilities ranging from 0 to 1.0.  Take the geometric mean of them.

Note that these RLHs are computed for each used draw, so the final RLH is the average of the RLHs for the used draws.

Comments

Thank you very much, Brian! Very clear!
So, as you said, Sawtooth reports as the last RLH the average of the RLHs for the used draws. That would imply that if I use the last 1,000 draws and report point estimates that are averages of those, then I could take the point estimates I am reporting (using) and calculate the RLH for them. I wonder if they'll be the same as the averages of the 1000 last RLHs.

---

Indeed, you could compute RLH based on the single summary vector (the point estimates, which are the average of the used draws) of utility for each respondent.  You will find that the RLH estimated for each respondent from the average of each respondent's used 1000 draws will be a bit higher than the average of the RLH values that were computed on each of the 1000 separate used draws.