Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Evaluation RLH for ACBC studies


I did an ACBC study and now I am wondering about how to interpret RLH.
In the CBC/HB paper it was mentioned that a comparison with the chance level is necessary.
Therefore I want to derive the average number of alternatives per choice task across all individuals and tasks.
Is it possible to derive this number or is it even already findable somewhere in the reports?
asked Mar 21, 2013 by anonymous

1 Answer

0 votes
This is tricky to do, unfortunately, and the software doesn't automatically give you the answer for null RLH per respondent.  The problem is that each person can have a different design, including numbers of tasks in the Consideration and Tournament phases.  So, the null likelihood is potentially different for each person.

With some effort and advanced data processing of the .cho file that you can export within ACBC software one could derive the null likelihood for each individual.  It would be the geometric mean of the null likelihoods across the tasks.  The BYO section contributes a single task per attribute seen in the BYO section, with as many alternatives per task as levels per attribute.  The Consideration phase contributes as many tasks as alternatives considered (must include any replacement concepts within the tally of tasks here!), where each task had two alternatives (near-neighbor concept vs. None concept).  The choice tournament typically is shown in triples, so each has a null likelihood of 1/3 for as many tournament tasks as the respondent received.  Of course, all this doesn't consider the aspect of possibly dropping levels as inferior.  When constructed levels of lists are used and certain levels are dropped, additional tasks are added to the data to show that the dropped levels were inferior to the included levels.  So, this gets a bit nasty indeed to compute the null RLH for each respondent in ACBC.  But, if you can write a script that can interpret a .cho file (can count how many tasks each respondent receives and how many alternatives are available per task) then you can certainly compute the geometric mean of the null likelihoods across those tasks.
answered Mar 21, 2013 by Bryan Orme Platinum Sawtooth Software, Inc. (136,165 points)