Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to derive the prior variance for any pricing study?

I am doing some researches to improve the precision of the utility estimation for the pricing studies. There is a paper on Optimal HB Priors, where I have found some benchmark given for adjusting the prior degrees of freedom and prior variance. It will vary by sample size and the number of attributes used in a study.

I have done a study with 575 sample size with two attributes. For one attribute, there are more than 100 levels and the other attribute is price which has 5 levels. The suggested prior degrees of freedom are 10 and the prior variance is 1.2.

I have also tried to estimate utility with increasing the prior variance ("Increasing the prior variance tends to place more weight on fitting each individual's data and places less emphasis on "borrowing" information from the population parameters") to 10 where I am getting a better hit rate and closer market alignment to share with compare to the suggested one.

In both the cases I have estimated utility with the assumption that price is linear and it has an interaction with the other attribute.

But the price sensitivity of the model with compare to the suggested one is going up. If we go with the suggested one the share of a major brand in the market is going down from 6.8 to 6.0. On other the utility with prior variance of 10 the share of the same brand is going down from 7.2 to 5.0.

Can someone throw some light on this phenomenon?

Thanks for the help in advance.
asked May 31, 2018 by sanmit1988 (460 points)
edited May 31, 2018 by sanmit1988

1 Answer

0 votes
 
Best answer
I'm curious regarding how you are estimating hit rate (raw hit rate true/false per observation or probability of the hit from the logit equation) and how many holdout tasks you are employing.  Keith Chrzan (our colleague) has found that unless you have at least 5 or more holdout choice tasks, hit rate reliability for comparing one model vs. another may be low.  So, you need to have plenty of holdout tasks to be reasonably confident that changes you are making to the utility estimation actually are having a statistically significant improvement.

Next, when you increase the prior variance, you allow respondent-level utility models to have higher fit (you are increasing the low-level model fit) and you reduce the Bayesian smoothing to the population means (the upper-level model fit).  Higher fit in logit models (our HB employs a logit model) means larger utilities and bigger differences between best and worst utilities within an attribute.  That's why when you increase the prior variance you are seeing price sensitivity increase.  The sensitivity to the first attribute (I'm assuming SKU?) will also be seen to increase.
answered May 31, 2018 by Bryan Orme Platinum Sawtooth Software, Inc. (163,515 points)
selected May 31, 2018 by sanmit1988
Hi Bryan,

Thanks a lot for the prompt response.

We are estimating the raw hit rate by true/false per observation and there are 3 holdout task.

I got your point that, with increase in prior variance means larger utilities and bigger differences between best and worst utilities within an attribute.

Kindly help me with the following questions:

1) Is there any way we can derive an optimal prior variance for a particular study with two attributes and one of them will have more than 100 levels?

2) Increase in prior variance actually helps to improve the precision of the attribute with higher number of levels, as it create bigger differences between best and worst utilities within an attribute, but for price as an attribute, where the level will be maximum 5 or 6, the same might not be true. Is there any way we can creates bigger differences between best and worst utilities of those attributes which higher levels without impacting the price utility?

Thanks for the help in advance.
...