Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

D efficiency of a design matrix

I have built a design in CBC HB. All my attributes are dichotomous. I have 10 attributes, 20 versions, 1 alternative per task + Traditional None.
I want to compare the D efficiency of my design with the efficiency of the design generated in SAS.

I took my design matrix - only the attributes columns (those with 1s and 0s).
I would like to calculate the true D efficiency of this design.
What is the formula for that? In my Sawtooth materials it said it's the determinant of the X'X matrix. However, I am not sure that's correct because I calculated it and it is = 1.57e40
So, I guess it's a bit more complicated than that.
Thank you for your help!
asked Mar 1, 2013 by Dimitri Bronze (645 points)

2 Answers

+1 vote
Easiest approach would be to import the SAS design into our CBC software (use the Import button from the Design tab) and use our Test Design procedure to compute the D-efficiency for the SAS design as well as the Sawtooth Software Design, holding the number of respondents constant.  The Sawtooth Software design might have 300 unique blocks/versions, and the SAS design might have a lower number of blocks.  You'll need to adjust the number of versions in your Sawtooth Software project specifications to match the specifications of the SAS plan.

(First, do an Export of the Sawtooth Software design, to get the right format in a .csv file.  Then, simply swap in the SAS design into the .csv file and re-import into your Sawtooth Software project.)

Please recognize that the D-efficiency of a design isn't a perfect predictor for how well an experimental design will do in practice to estimate utilities with high precision.  For example, the default design method in Sawtooth Software is now "Balanced Overlap" that on purpose gives up some d-efficiency for the benefit of achieving a modest degree of level overlap (level repeats within each task).  Level repeating makes it so that respondents who use non-compensatory decision making (such as always picking their favorite brand or always picking the lowest price) will face a tradeoff beyond just picking their one favorite level per task.  

Also, we take the approach of Balanced Overlap, because it often leads to the efficient estimation of all potential first order interaction effects.  That's another benefit of Sawtooth Software's designs: they typically support all main effects plus 1st order interaction effects.

BTW, you need to raise the determinant of the X'X matrix to the -1/n power where n is the number of rows or column in the X'X matrix (covariance matrix).
answered Mar 1, 2013 by Bryan Orme Platinum Sawtooth Software, Inc. (138,915 points)
I am afraid I cannot use the Test Design procedure because I have 33 attributes. When I started generating this design in Sawtooth I got a warning message - something about the fact that Test Design will not be conducted because of too many attributes. This is why I am trying to calculate the D efficiency myself and asked for a formula.
The Advanced Design Module in version 8 bumps it up to 250 attributes.  We could give you a trial license for version 8 (you can install it alongside a previous version) and you could run the design test.
0 votes
I also found the D-efficiency formulas below. Looks like they also divide the discussed product by n (number of design points) - I assume n is the number of rows in the design matrix...

In SAS: http://support.sas.com/documentation/cdl/en/qcug/63922/HTML/default/viewer.htm#qcug_optex_a0000000394.htm

In in R (
y is a vector of n observations, Z is an n x k matrix, and b is a vector of k parameters
D efficiency = det(M)^(1/k), where det(M) is the determinant of the normalized dispersion matrix M – i.e. M=Z'Z/n, where Z=X[rows,]
answered Mar 1, 2013 by Dimitri Bronze (645 points)