Is there a guideline for the acceptable % of attribute/level effic utility that can be go ahead the survey? 95%, 90% or 85%...?

Thanks and appreicated replies!

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Is there a guideline for the acceptable % of attribute/level effic utility that can be go ahead the survey? 95%, 90% or 85%...?

Thanks and appreicated replies!

0 votes

Hi, lacking statistical background I can tell you that, what prohibitions basically do is to not showing some levels, thus decreasing number of observations for these levels. That causes those levels to be calculated with lesser degrees of accuracy due to relatively smaller base.

I don't know about any percentage to serve as a guideline though.

I don't know about any percentage to serve as a guideline though.

Thanks Bahadir.

0 votes

A prohibited pair leads to lower statistical precision for estimating the effects (utilities) of the levels of each attribute level on choice, due mainly to two factors: 1) it causes some correlation between the patterns of the two attributes shown (multicollinearity in the design matrix), 2) it often causes the levels to not be able to be shown an equal number of times.

Let's say you and your client were comfortable with the precision of the utility estimates that would result from a particular CBC study given 500 respondents and a certain number of choice tasks shown per each respondent. Then, let's say that the client orders up a certain series of prohibitions that leads to 50% relative efficiency compared to the original design specification. All that means is that you would need to double the sample size to get the same precision of the estimates that you were comfortable with when you had no prohibitions and 500 respondents. So, you could simply say to your client that what is needed to counteract the negative effects of that series of prohibitions is to double the sample size. The client pays the extra money to bring the precision of the design back to what you had both been comfortable with (by doubling the sample size), and you're both again satisfied.

This is an example where what appears to be a terrible hit in efficiency can be remedied by increasing the amount of data substantially. Most folks would see this as a terrible loss in efficiency and a huge hit in expense. But, it demonstrates that there isn't some magic relative efficiency hit that suddenly becomes unbearable. Money can solve the problem (unless the sample frame is so small that you cannot get 1000 of the type of respondents you need!...think doctors in a certain hard-to-reach specialty).

But, better idea is to use tricks that we discuss in our training clases to often get around the issues of prohibitions:

1) Avoid them if respondents can still view the combinations of levels as reasonable and possible. Respondents can often imagine possibilities that the client thinks are strange or not probable. Then, just don't simulate those possibilies in the market simulator when conducting the analysis.

2) Use alternative-specific designs to handle sometimes what seem to be really restrictive prohibitions, while still maintaining independence among the attributes (factors). These are provided through our advanced design module. Classic example is transportation problems where bus attributes only appear with buses, train attributes with trains, car attributes with cars, and walking attributes with walking.

3) Collapse two attributes (factors) into a single attribute, representing all combinations you need to show to respondents. For example, imagine two 3-level attributes that normally would result in 9 possible combinations of those two attributes to show respondents. If the client prohibits 5 of these combinations, you are just left with 4 combinations to show respondents. A single 4 level attribute in place of the two 3-level attributes could accomplish that. The respondent still is seeing two pieces of information per level of that collapsed attribute, but the design itself thinks it is completely free of prohibitions and has excellent design efficiency. Market simulations are fine; but you no longer automatically get separate attribute level utilities for those two collapsed factors and you don't get separate attribute importance scores for the two factors any more. However, since market simulations are where the real value is with conjoint, this hopefully won't be a deterrent to this excellent trick.

4) Use conditional pricing tables if the the prohibitions involve the Price attribute. This allows you to customize the range of prices shown for premium vs. dicsount brands (for example) while not using any prohibitions at all, so the design is still perfectly balanced and involves no collinearity in the design matrix. Each brand gets the same number of price levels; but the levels are customized per brand according to a lookup table of price values you supply. See our documentation in the CBC manual on that, including rules and caveats for interpretation of utility effects when that's done.

Let's say you and your client were comfortable with the precision of the utility estimates that would result from a particular CBC study given 500 respondents and a certain number of choice tasks shown per each respondent. Then, let's say that the client orders up a certain series of prohibitions that leads to 50% relative efficiency compared to the original design specification. All that means is that you would need to double the sample size to get the same precision of the estimates that you were comfortable with when you had no prohibitions and 500 respondents. So, you could simply say to your client that what is needed to counteract the negative effects of that series of prohibitions is to double the sample size. The client pays the extra money to bring the precision of the design back to what you had both been comfortable with (by doubling the sample size), and you're both again satisfied.

This is an example where what appears to be a terrible hit in efficiency can be remedied by increasing the amount of data substantially. Most folks would see this as a terrible loss in efficiency and a huge hit in expense. But, it demonstrates that there isn't some magic relative efficiency hit that suddenly becomes unbearable. Money can solve the problem (unless the sample frame is so small that you cannot get 1000 of the type of respondents you need!...think doctors in a certain hard-to-reach specialty).

But, better idea is to use tricks that we discuss in our training clases to often get around the issues of prohibitions:

1) Avoid them if respondents can still view the combinations of levels as reasonable and possible. Respondents can often imagine possibilities that the client thinks are strange or not probable. Then, just don't simulate those possibilies in the market simulator when conducting the analysis.

2) Use alternative-specific designs to handle sometimes what seem to be really restrictive prohibitions, while still maintaining independence among the attributes (factors). These are provided through our advanced design module. Classic example is transportation problems where bus attributes only appear with buses, train attributes with trains, car attributes with cars, and walking attributes with walking.

3) Collapse two attributes (factors) into a single attribute, representing all combinations you need to show to respondents. For example, imagine two 3-level attributes that normally would result in 9 possible combinations of those two attributes to show respondents. If the client prohibits 5 of these combinations, you are just left with 4 combinations to show respondents. A single 4 level attribute in place of the two 3-level attributes could accomplish that. The respondent still is seeing two pieces of information per level of that collapsed attribute, but the design itself thinks it is completely free of prohibitions and has excellent design efficiency. Market simulations are fine; but you no longer automatically get separate attribute level utilities for those two collapsed factors and you don't get separate attribute importance scores for the two factors any more. However, since market simulations are where the real value is with conjoint, this hopefully won't be a deterrent to this excellent trick.

4) Use conditional pricing tables if the the prohibitions involve the Price attribute. This allows you to customize the range of prices shown for premium vs. dicsount brands (for example) while not using any prohibitions at all, so the design is still perfectly balanced and involves no collinearity in the design matrix. Each brand gets the same number of price levels; but the levels are customized per brand according to a lookup table of price values you supply. See our documentation in the CBC manual on that, including rules and caveats for interpretation of utility effects when that's done.

Thanks so much for your detail explaination and suggested approach. During my test of different design w/ or w/o prohibitions, it seems like double (or even triple) the sample size not be able recovery the effect utility lose. I also learn conditional pricing table from the help file, that really a good practice to solve such pricing related question.

BTW: How can I get the CBC manual? I can't find it from my SMRT and SSI installation on my PC. Thanks.

...