Do I have enough sample for a "simple" menu based experiment?

Question

Hi MBC experts,
I have a specific problem (for which I consider to use MBC) but I worry that the sample available for my experiment (as few as 40respondents) is too small. On the other hand, my model is not very complex. Basically I just look for reassurance here from somebody with some experience in the field.

So here is the deal:
My client is considering the launch of a new product. The attractivity of that product is very much dependent on the availability of added services around the product that the customer can than buy in addition to the plain product. I am interested in the impact of the availability of these 17 prospective new services (and not at all the prices!). It is a pure availability design.

So I picture an experiment in which the respondent is faced with say 12 to 15 tasks, in each of which he is presented with a subset of these services and then asked to select the services that he considers relevant and would like to use in combination with the product or choose the none-option (i.e. to stick with the traditional product). So I model a) the decision for the new product conditional on the availability of the services and b) the decision to actually use these services conditional on the available selection of other services. So technically I can do this seperately with regular binary logistic regression without Logit/MNL.

Question #1: Does anybody have other suggestions for a smart experiment and the modelling afterwards keeping in mind the small sample?

Question #2: Will MBC be able to handle the estimation and simulation? It seems quite basic...

I probably will only have 20 parameters (incl. cross-effects) to estimate at max. My guess is that I will be able to discard some non-significant effects when specifying the model. With the available sample size each service will be displayed roughly 200-300 times depending on how I set up my restrictions on the design (e.g. minimum of 8 services available in each task). That sounds kind of plenty but unfortunately I have not done this before.

Question #3: Is anybody willing to share their gut-feeling about the sample-size issue with me?

I look forward to your responses. They are much appreciated!
Alex from Germany

Answer 1

Well phrased question and thinking.

Some researchers (Chris Chapman formerly of Microsoft and now at Google) is well experienced with using CBC studies to guide business decisions.  He is fond of saying that he sometimes learns enough from 20 to 40 well sampled respondents to make many key business decisions, if the differences shown in the conjoint study (market simulator) are large enough to easily surpass some go/no go threshold.  

But, if the business decisions are more complex and involve a more subtle discovery of differences in the conjoint results that require larger sample sizes to detect, then certainly larger sample sizes are needed.

MBC software would also treat your problem as a series of separate binary logits (estimated either via aggregate logit or HB).  If you feel you can manage those models on your own and link them together to build your simulator, then there really isn't a need to buy the software.  That's what the MBC software does (though it does go beyond that to do some nice combinatorial selections simulations, which are harder to pull off on your own).  

You have to decide how much comfort you have with 200-300 representations of an item in the design across respondents.  You can generate some dummy data, and run aggregate logit, and get some feel for the margins of error on the parameters of interest.  The models will work, the worry is just how much precision you require.

Comments

Thank you Bryan,
I think I will check out the potential precision with dummy data as you suggested.
Best,
Alex