Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to deal with inefficient designs?

My design has turned out to be inefficient. Is there anything I can do about it if I need to maintain the prohibitions?

asked Aug 3, 2015 by anonymous

2 Answers

+3 votes
Often deficient designs result from setting up prohibitions that result in having two levels that always show up together. For example, if I have a product and one feature is having a fan attached to it, I could have attributes for it's size and speed:

Fan Size:
No fan

Fan Speed:
No Fan

If I were to set up prohibitions, I would back the designer into a corner and say, if Fan Size is "no fan" then Fan Speed must be "no fan" which makes sense logically, but in the behind-the-scenes math, it's trying to estimate two variables for one thing, which doesn't result in a stable solution.

For example, if my utility for not having a fan is 2, if I have one variable, then it's a simple x=2, if I use two x+y=2, then I can pick any x and y pair that equal 2, anywhere from x=0,y=2 to x=1002,y=-1000.

The workaround for this is to either collapse the attributes into one attribute, or create an alternative specific design.

The collapsed attribute would have levels like this:
No Fan
Large Fan, Slow Speed
Large Fan, Fast Speed
Small Fan, Slow Speed
Small Fan, Slow Speed

The alternative-specific specification would look like this:

Attribute: Has Fan

Size (only if Has Fan = Yes)

Speed (only if Has Fan=Yes)

If you want me to take a look at your design and comment specifically, please send the .ssi file  to support@sawtoothsoftware.com and I would be happy to offer more specific advise.
answered Aug 3, 2015 by Kenneth Fairchild Bronze Sawtooth Software, Inc. (3,495 points)
+1 vote
Well there are a few things you can try (perhaps you already have tried some of them?).

First, you might want to think about whether your prohibitions could be recast as alternative specific effects - when that's the case you have an elegant solution that can improve your efficiency very nicely.  

If that won't work, and perhaps you've already done this, you can try one of the less restrictive design strategies - balanced overlap or random:  sometimes they don't struggle as much with prohibitions.

Finally, you can make up for lost efficiency with larger sample size:  if you were planning to interview 300 respondents but your design has only 75% efficiency, you could interview 400 instead and get the same level of precision as if you had a 100% efficiency design and 300 respondents.  This last point uses logic that applies to an aggregate logit model, so it won't be completely accurate if you're running  and HB model.
answered Aug 3, 2015 by Keith Chrzan Platinum Sawtooth Software, Inc. (64,975 points)