Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Regarding the sample sizes required for MBC


I understand that MBC requires large sample sizes, and in fact I have noticed the following rule of thumb in regards to the preferred amount of data: [#respondents x #menu_tasks]/price_levels should be around 1000. However, I'm interested in MBC's capability such as the one described in the manual (pasted below):

"In pharma studies, we often hear of projects involving CBC-looking tasks with multiple drugs and a patient profile in the prologue. The task involves showing a patient profile with varying characteristics, an array of drugs with varying characteristics, and doctors are asked to select which drug or combination of drugs they would prescribe to a patient with those characteristics."

In such a task, price is not included as an attribute being measured, so I'm wondering what rule of thumb I need to apply to estimate the sample size I need. Would I just look at whichever attribute measured has the largest amount of levels? Within reason, can I conclude that I can add more menu tasks per respondent to make up for a smaller sample size in order to reach 1000 times?

asked Mar 21, 2013 by anonymous

1 Answer

+1 vote
That's a nice and simple rule-of-thumb.  But, it's just a starting point.  The more independent moving parts (attributes), the more parameters to estimate, and the larger sample size needed to obtain equal precision as a smaller design.  That simple rule of thumb you refer to was developed back in the day (early 1990s) when CBC was limited to just 6 attributes and we only knew about aggregate analysis.  So, Rich Johnson, who developed the formula assumed it would be applied to pretty straightforward and clean situations.

With the pharma example, there are moving pieces for the patient characteristics and also the drug characteristics.  But, if you can make some simplifications such as linear estimation of price parameters (requires just one beta rather than a separate beta for k-1 price levels, where k is the total number of price levels within the attribute) then you can do quite a bit even with limited sample sizes.  Of course, you cannot escape sampling error due to small sample sizes.  That will always be a limitation.  (Except the obvious case where the total population of interest is just 50 people and you are able to interview 48 of the 50, for example.  That sample of 48 respondents is associated with really low sampling error since they represent nearly a complete census.)

I recommend people generate dummy respondent data (essentially random answers) for the MBC design they plan to field and that they estimate aggregate logit models of the type they eventually plan to use when the final data are collected.  Then, they can observe whether the logit estimation converges and the standard errors have reasonable (small) size.  You can look at standard errors with 1000 people, 500 people, 100 people, 50 people, etc. and see the differences.

Often times with pharma research and doctors we just cannot expect to obtain the same precision that we can shoot for with general consumer samples.  But, the decision comes down to whether the data will provide measurably better insights than mere guesses.  It usually turns out that even though the confidence levels aren't as tight with studies involving doctors than studies involving beverage consumers, the research is still worth every penny due to the signal relative to noise and the confidence intervals one still may obtain even from what seem to be smallish sample sizes.

So, in short, one could look at rules of thumb from CBC as a starting point (as you mention).  But, much better to actually build experimental design plans, generate random data, set up the models you plan to estimate in MBC, and estimate using aggregate logit.
answered Mar 22, 2013 by Bryan Orme Platinum Sawtooth Software, Inc. (133,765 points)