Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How can MBC and CBC results be compared?

Dear Experts

In order to answer our research question we have conducted two independent studies. One time using the MBC method, the other time using CBC. The data of both studies have been analyzed through HB estimation. Subsequent market simulations produced contradicting results (in order of prices and product bundles)

Are there any scientific approaches to analyze the contradicting results? Also can you recommend any papers that cover the impact of different approaches of data collection on the results?

asked May 6, 2013 by anonymous
retagged May 6, 2013 by Walter Williams

2 Answers

+1 vote

If you use two very different design and modeling approaches there are any number of things that can cause the results to differ.  Did you have exactly the same sets of attributes, levels, brands, prices?  Did your respondents have the same qualifications and characteristics.  Were respondents making the exact same kinds of choices?  Were they choosing in the exact same context?  My guess is that the answer to at least one of the questions above is "no" or else you would have used the same methodology (CBC/MBC) both times.  It's where you say the "no" that you should start seeking an answer.

answered May 6, 2013 by Keith Chrzan Platinum Sawtooth Software, Inc. (72,475 points)
+1 vote
It is expected that CBC results and MBC results do not lead to the same underlying estimated utilities and predicted choice probabilities.  This has been examined and found multiple times and it is good that you have confirmed this once again.  CBC asks the respondent to do something quite different from MBC: pick one pre-configured alternative from a set of pre-configured alternatives.  MBC asks people to configure each item on the menu until an overall bundle has been constructed that the respondent prefers.

Rich Johnson, Jon Pinnell, and I found quite different results when comparing CBC to MBC in our paper we presented in 2006: "SIMULATING MARKET PREFERENCE WITH “BUILD YOUR OWN” DATA" downloadable from https://www.sawtoothsoftware.com/download/techpap/2006Proceedings.pdf

Also, Bakken also found in two earlier papers that the information provided or utilities estimated from comparable CBC and MBC attribute setups led to different utilities:

Bakken, David and Len Bayer (2001), “Increasing the Value of Choice-Based Conjoint with „Build Your Own‟ Configuration Questions,” Sawtooth Software Conference Proceedings, pp 99-110.  downloadable from https://www.sawtoothsoftware.com/download/techpap/2001Proceedings.pdf

Rice, Jennifer and David Bakken (2006), “Estimating Attribute Level Utilities from „Design Your Own Product‟ Data—Chapter 3,” Sawtooth Software Conference Proceedings, pp 229-238.
downloadable from https://www.sawtoothsoftware.com/download/techpap/2006Proceedings.pdf

So, this is a strong argument that if the consumer is facing a decision where that person uses a menu to construct their preferred alternative, then the CBC approach will probably not mimic this appropriately and probably not lead to quite as good as predictions of actual market behavior as MBC.  But, if the respondent selects one from many pre-configured product alternatives (and doesn't have the opportunity to multi-select and build her preferred product), then CBC would be the more appropriate conjoint tool to model preferences.
answered May 6, 2013 by Bryan Orme Platinum Sawtooth Software, Inc. (162,060 points)
Dear Bryan, Dear Keith

Firstly let me thank you for responding to my question, your help is very much appreciated.

Your remarks are very helpful to my research but also make me even more curious.

The data i am working with have been collected by a professional research company. The two studies (MBC/CBC) - although very different in nature - have been designed (in terms of price levels and attributes) to obtain comparable results as effectively as possible.

The conducted research (and obtained results) is particularly interesting since the examined products are of digital nature and can therefore be offered as a menu as well as a bundle. The question would now be can the simulated shares and revenues be compared and therefore can the different methods be used to determine whether products should be offered as a menu choice or as a bundle.