Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

question to smrt with logit


I have two questions concerning the market simulations with aggregate logit.
First of all, I would like to know if there is way to calculate hold out hit rates. I was just able to analyse the MAE with CBC/logit using SMRT. Moreover, I would like to do sensitivity analysis using my utilities from CBC/logit. I used share of preference (I have an alternative specific design) and in my outputs I can not find any standard error, since I need this to have certainty about significant differences in market shares by changing levels of attributes.

Thank you for your support.

asked Nov 2, 2016 by isabelle (180 points)

1 Answer

0 votes
We strongly recommend against using aggregate logit for building market simulators and performing hit rate analysis.  Aggregate logit is an old approach from the 1970s and 1980s that is good when you first start to look at your choice data, in the sense that it is fast and gives a quick overall (average) summary.

But, it computes an average solution across the entire sample.  In other words, it acts as if it was one respondent who took a very long survey (and completed all the choice tasks that were actually completed by multiple respondents).

Aggregate logit is blind to differences between people (heterogeneity) and we therefore do not report standard errors and confidence intervals.  It also isn't very accurate to use aggregate logit to compute hit rates.

We strongly recommend you use HB instead for building your market simulator, getting the confidence intervals, and computing hit rates.  HB is  automatically built into your Lighthouse Studio (SSI Web) software--assuming you are using a recent version from the last few years.
answered Nov 2, 2016 by Bryan Orme Platinum Sawtooth Software, Inc. (132,290 points)
Thank you for your answer and your recommendation. I did an HB analysis and I think there is some random noise in my data which makes some individual preferences look like there is an overall preference for a level. Is this possible?
I do a line extension and did two equal studies with 250 respondents each. My HB results show different preferences for levels for both studies. I can not find any explanation for this. I did sensitivity analysis to check for signficant market share changes and those show significant differences between both studies und between both levels. My idea was to check what logit tells my about significant influences of part worth utlities. And it tells me that there are no significant part worth utilities for this attribute. Like you can imagine I look for the best product to maximize the revenue of the whole product line.  Do you think it is a better idea to look for segments by using Latent Class?

Thank you again in advance.
Perhaps I am not understanding your comments and questions.  There is always random noise in conjoint analysis utilities, whether for HB or aggregate logit.  Plus, when you compare two equivalent studies that were conducted among two different groups of respondents, there will be random noise but also sampling error that causes differences between the results of the studies.

Latent Class is a good technique for discovering segments of respondents who have similar preferences within each segment, but different preferences between the segments.  Some researchers, therefore, use Latent Class to find "needs-based segments".