Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

ACA formula - how many pairs to ask?

Hi. When applying this formula to determine the number of pairs questions, I am curious to know how much can we deviate from that ideal number before we lose integrity of the data.

(3*(n-N-1))-n, where:
N = number of attributes
n = total number of levels across all attributes

In a current example, this formula yields a number which makes our survey very long. In order to shorten the survey we could reduce the number of pairs questions but don't want to jeopardize capturing high-quality data.

Are there rules of thumb here? Thanks.
asked May 26, 2017 by WTW_cal Bronze (1,225 points)

1 Answer

0 votes
Nice question.  For exactly this reason, the inventor of ACA (Rich Johnson) conducted a series of simulations to investigate how much resolution (accuracy) is lost by using fewer pairs than recommended by the formula.  The formula indeed can end up recommending too many pairs for humans to bear.  Luckily, ACA is quite robust when cutting that number down below the recommended...especially with the use of ACA/HB (which was not available when Rich Johnson conducted these simulations).  For Rich's simulation results, please see the paper:

answered May 26, 2017 by Bryan Orme Platinum Sawtooth Software, Inc. (133,765 points)