Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to compare preferences across two choice-based surveys?

I want to conduct a CBC in order to find out, whether a pay-per-use pricing plan is better than a flatrate. This brings along two difficulties:
1) A per-unit-price is much lower than a flatrate price, therefore the levels of the price variable have a much higher range than all other variables.
2) Subjects might find it difficult to trade-off between these two options because they cannot be really compared due to their differing consumption interval (one month for the flatrate, one unit for pay per use) and are just too different.

My question: If I conduct two separate CBCs - one for PPU, one for flatrate - is there a way to merge the results and predict choice for either option altough it was not part of the other option's CBC design, respectively?

More concretely:
1) Is it possible to compute choice probabilities, compare them, and whichever option has the higher probability, "wins"?
2) Is it possible to calculate the individual consumer surplus as monetary utility minus price and whichever option has the higher surplus, "wins"?

Or would it be better to simply conduct one common CBC despite the above mentioned problem?
asked Jun 21, 2015 by Ulf

1 Answer

0 votes
Very tricky, I think, to separate these two in different surveys as a between-respondents design.  The consistency of respondents to use the "none" category is at stake.  The None usage depends on context issues within the survey (such as the complexity of the survey task, whether all attributes are shown in full or partial profile, the length of the survey, etc.).  By splitting the sample into two different surveys and comparing the results between respondent groups, you're also subjecting yourself to differences due to sampling error.

I would naturally battle hard to figure out an appropriate way to have respondents compare the two ways of acquiring the product within the same discrete choice questionnaire.  Do some pretesting and see if respondents seem to be able to manage.  Debrief the respondents.  Ask them questions about how confused or not they were with the tradeoffs involved.
answered Jun 23, 2015 by Bryan Orme Platinum Sawtooth Software, Inc. (170,015 points)
Thanks for your reply. I actually meant to combine both designs in one questionnaire (with less choice sets for each design, of course) and thus have them both answered by ALL respondents. I would then compare probabilities and consumer surplus only within-respondents to derive a conclusion, which option is preferred although it was not part of the other option's design. Does that make more sense?
It makes sense to do this to reduce the sampling error for a between-subject design I mentioned...but you still will have the challenge that you are putting  your faith on the scaling of the "none" alternative between two sets of CBC questions that have slightly different context.  This is the remaining worry I have.  I don't know how big the problems may be for doing such, but I just call your attention to it for consideration.