None Threshold & Calibration to Purchase Likelihood
Top  Previous  Next

One of the benefits of traditional CBC modeling is the ability to include a "None" option. However, interpretation of the strength of the None parameter as well as appropriate application within market simulations have been difficult issues in CBC research. Adaptive CBC can estimate a None threshold using two approaches:

·None parameter estimated from Screening Tasks  
·None parameter inferred from purchase intent scales via Calibration Concepts  
None Parameter Estimated From Screening Tasks

The Screening Tasks present product concepts to the respondent and ask if each is a possibility or not. These tasks are coded for utility estimation as binary choices of the product concept versus a threshold of acceptability. The parameter estimate for that threshold of acceptability may be used as a surrogate for the None parameter within choice modeling. However, recognize that this None parameter has some characteristics that depart from the traditional accepted practice of computing a None threshold for CBC projects:

·It is scaled based on binary choices (associated with labels such as "a possibility" and "not a possibility") rather than in the context of a purchase/choice among multiple available options in a marketplace.  
·The None parameter estimated from ACBC's binary tasks will generally be stronger (higher) than from traditional CBC tasks. The product concepts presented in ACBC surveys are generally of higher utility and closer to the respondent's ideal than typical CBC questionnaires. When respondents become conditioned to having the opportunity to choose among generally more relevant and desirable concepts, they react more negatively to concepts of moderate utility that might have seemed acceptable to them within the context of a standard CBC questionnaire.  
None Parameter Inferred from Calibration Concepts

Some users may want to calibrate the None parameter (threshold) using 5-point purchase likelihood questions. This was suggested by Jon Pinnell in a paper entitled, "Alternative Methods to Measure Changes in Market Size in Choice Based Conjoint" presented at the 2006 SKIM/Sawtooth Software European Conference Event. In justifying the 5-point purchase likelihood intent scale, Pinnell reports: "While the 2-point buy/no-buy is consistent with the historical use of a none alternative in choice modeling, we prefer a 5 point follow-up question in that it provides more information and is also consistent with most methods outside discrete choice modeling to gauge market size and product appeal. In addition, many marketers have developed internal normative databases reflecting the five point purchase intent scale and in-market performance and have used those to develop action standards based on the five point purchase intent scale."

Our use of the rating scale is in the tradition of concept testing with a five-point Likert scale and departs from choice theory and multinomial logit assumptions. Given the problems in obtaining proper scale for the None parameter under sound choice theory, many will consider this departure to be reasonable and defensible--especially if the firm has prior data regarding how Likert scales map to eventual product acceptance.

Unlike ACA, our calibration procedure does not rescale part-worth utilities by multiplying by a slope and adding an intercept. The only purpose of the calibration concept data within ACBC is to estimate a new threshold None utility value (replacing the None threshold computed from the Screening concepts). It is entirely optional to include these calibration concept questions in the ACBC survey (as one can simply retain the original None parameter).

To estimate a new None utility using the Calibration Concept data, click Analysis + Estimate ACBC None Threshold... Select an existing .hbu file as the Source File (this file contains your part-worth utilities). Specify a new file with .hbu extension as the Destination File (this file will be created using the original utilities plus the newly estimated None utility).

 
A key assumption when estimating a new value for the None utility is the point on the Likert scale that should represent the buy/no buy threshold for respondents. Obviously, this could differ by respondent. However, we generalize that the buy/no buy threshold on the scale is constant across respondents and further assume that you as the forecaster can identify that appropriate scale point (easy, right?).

The scale points on the Likert scale are typically labeled as follows:

5 Definitely Will Buy  
4 Probably Will Buy  
3 Might or Might not Buy  
2 Probably Will not Buy  
1 Definitely Will not Buy  


If, for example, you indicate that the None threshold is the product utility corresponding to a 3.2 on the scale, we estimate that utility (for each respondent) by referring to the already-estimated part-worth utilities and the calibration concept data. The calibration ratings are regressed (at the individual level) on the final utilities that resulted from HB or Monotone Regression. We'd recommend that respondents be shown five or six calibration concepts, to get reasonably stable estimates from the regression. Using the respondent-specific estimated slope (Beta1) and intercept (C), we can compute the utility corresponding to a 3.2 on the Likert scale using the equation (3.2 - C) / Beta1.

The estimation of the None threshold from the purchase likelihood scale is a post hoc procedure that does not involve re-estimating the part-worths. As it uses OLS, the computation is nearly instantaneous, and the analyst can quickly compute new None threshold parameters using different assumptions about the point on the rating scale that corresponds to the threshold point for buy/no buy.

When you calibrate the None parameter, you create a new .hbu file. Only two fields are modified in that file: the None parameter value and the fit statistic. The calculation of the None parameter was described directly above. The fit statistic is the correlation resulting from the calibration regression step (the square root of the R-squared). When this None utility threshold is used within the market simulator, a new "product" with utility equal to the None threshold value is introduced in competition with the other product concepts within that market simulation. The share captured by the "None" product reflects the percent of the market projected not to buy a product within this scenario.

The default setting for the point on the rating scale corresponding to the buy/no buy threshold is 3.0. We have no evidence that this is proper for your project needs, and recommend you decide for yourself the appropriate setting. Using a recent dataset shared with us, we found that using a 2.8 on the 5-point purchase likelihood scale led to a None parameter from the OLS calibration that on average was about the same magnitude as estimated via HB using the binary judgments of the Screener section (where the binary choices were labeled "A possibility" or "Won't work for me"). Thus, using the default 3.0 as the buy/no buy threshold resulted in a slightly stronger None parameter utility than was initially estimated using the Screener section information.


Exception Handling:

Given respondent error, regressions at the individual level based on so few data points may sometimes lead to irrational outcomes, such as negative slopes (suggesting a negative relationship between estimated utilities and scores on the Likert scale). Also, if the respondent answers the calibration concepts using the same scale point, there is no variation to model. Therefore, we must handle these exceptions.

If the estimated slope is negative, the None threshold is set equal to the average utility for products exceeding the threshold point on the Likert scale minus 1. If the slope is negative and no products are above the threshold, we set the none utility equal to the largest utility of any product plus 1. If the slope is negative and all products are above the threshold, we set the none utility equal to smallest utility of any product minus 1. If there is no variation in the Likert scale responses, we do the following: if all products are rated equally at a point below the threshold point on the scale, we set the none utility equal to the largest utility of any product plus 1; if all products are rated equally on or above the threshold, we set the None utility equal to the smallest utility of any product minus 1. With all these exceptions, a fit statistic of 0 is written to the new utility file.

Because the None utility is estimated using the equation (UserSpecifiedScalePoint - ConstantFromRegression)/Beta, a small positive Beta near zero can cause the None utility to become extremely large in absolute magnitude. To guard against the most extreme cases, if the estimated beta is between 0 and 0.05, we set it to 0.05.

For a recent dataset shared with us, 88% of the sample had calibration data that were well behaved and did not require exception handling (there was variation in the dependent variable and the estimated slopes were positive and larger than 0.05). Of the 12% that required exception handling, 5% had tied ratings (across four calibration concepts), 6% had negative slopes, and 1% had slopes between 0 and 0.05 that we trimmed to 0.05.