Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to calculate and interpret a D-efficiency

We are currently preparing a partial profile choice-based conjoint analysis questionnaire using Sawtooth Software. Our current questionnaire has 25 choice tasks, with 2 concepts per task, and the estimated strength of the design (based on 170 respondents) is 684.70669.
We have also generated an orthogonal design (with the same number of choice tasks and respondents), and this has an estimated strength of the design of 1498.56.
Thus, we calculated the D-efficiency to be: 684.70669/1498.56=0.4569 (45.7%).
We have two questions:
1. Is the calculation of D-efficiency correct?
2. How should we interpret these numbers?
Any response would be greatly appreciated!
Laura and Nadine
asked Jan 21, 2014 by anonymous

1 Answer

0 votes
Yes, this is the way to interpret the relative strength of the two designs.  

If respondents answered like computers, examining all attributes and levels on the screen, adding up their internal part-worth utilities for each concept (with some error, distributed Gumbel), and picking the concept that has the highest utility each time...then indeed you could get the same precision of utilities by using about 1/2 as many full-profile tasks as partial-profile tasks.  

However, we all know that respondents do not behave so rationally and consistently.  Plus, if you have more than about six or eight attributes (depending on the complexity of displaying those attributes), so much information can become overwhelming on the screen to view.  In some cases, the added response error for humans dealing with full-profile tasks rather than partial-profile tasks (which are more compact and easier to digest, since they only include a subset of the attributes within each task) counterweighs the statistical efficiency of the full-profile design over the partial-profile design.  It might be possible that your partial-profile design could lead to higher precision utilities than the full-profile design, if response error is much less for partial-profile.

Partial-profile is potentially good and potentially bad.  It often can underweight the importance of price by a modest amount if price is part of your experiment.  

If you had intended to do 10 full-profile CBC tasks.  You might want to do something like 12 to 18 partial-profile tasks (since they are much easier for respondents to do).

You say you are wanting to use pairs (2 concepts at a time) rather than the typical 3 to 5 concepts per task used in marketing applications.  Please consider how much gain you could get (without much added human cost) by using quads instead of pairs!  You could probably avoid doing 25 choice tasks (a lot!) if you were using quads instead of pairs.  I suspect you'd do just as well with 15 quads as about 25 pairs...but that's an empirical question that would need to be tested with real respondents.
answered Jan 21, 2014 by Bryan Orme Platinum Sawtooth Software, Inc. (132,290 points)
Partial profile tasks are easier for respondents, hence their ability to reduce response error and counterbalance (and usually overbalance) their lower statistical efficiency (=greater statistical error).  Because of this overbalancing you don't usually need to ask more of them, though of course you can if you want.  

Bryan mentions that partial profile models sometimes understate the importance of price .  Another thing to watch out for is that, unless you design them carefully up front with specific interactions in mind, you cannot use them to estimate interactions correctly.
We thank you for your help!
Laura and Nadine