Estimating Part-Worth Utilities

Estimating Part-Worth Utilities

Once you have downloaded your data (saving the studyname_data.sqlite file within your study folder) or read your paper-and-pencil data into SSI Web, you are ready to estimate part-worth utilities. CVA employs different statistical methods for estimating the separate part-worths for the attribute levels in your study. A unique set of part-worths is estimated for each individual, and the utility run is saved to a .UTL file for importing into the SMRT Market Simulator.

Ordinary Least Squares and Monotone Regression

When you click the Analysis menu, you can select whether to estimate utilities via Ordinary Least Squares (OLS) or Monotone Regression . (If your system includes the CVA/HB module, you can additionally choose HB estimation.)

OLS is the method of calculation traditionally used in most conjoint studies. However, OLS is not appropriate for conjoint data consisting of rank orders.

For OLS to be appropriate, we must assume the data are "scaled at the interval level." By this, we mean that the data are scaled so that real differences in the things being measured are communicated by the arithmetic differences in their values. Fahrenheit temperature, for instance, has an interval scale. The difference between 70 and 80 degrees is exactly as large as the difference between 80 and 90 degrees. In the social sciences and in marketing research we are usually willing to assume that rating scale values possess this kind of scaling.

However, it is usually not reasonable to make such an assumption about rank order data. Suppose a respondent were to rank 30 concept statements in terms of his likelihood of buying each concept. In the absence of other information, we would probably expect the concepts to have a normal distribution of buying likelihood. If so, then we would expect there to be larger "real" differences between concepts with extreme ranks (such as 1 versus 2, or 29 versus 30) than those in the center of the distribution (such as ranks 14 and 15).

When the data are rank orders, it is more appropriate to use a method of calculation that does not assume that the data represent anything more than rank orders. That is the case with nonmetric methods, and in particular with the monotone regression method provided in CVA.

There is also another reason why we have provided a nonmetric method of calculation within CVA: With such methods it is easier to constrain calculated utilities to conform to the researcher's expectations.

Conjoint utilities are often observed to violate principles of common sense. For example, in pricing studies it sometimes turns out that respondents seem eager to pay higher prices rather than lower prices. This may accurately reflect some respondents' behavior; price is sometimes taken as an indicator of product quality, and respondents may suspect that a low price reflects poor quality.

However, it is customary to explain to the respondent that "everything else is equal," and that the attributes are to be considered independently of one another. Under those conditions, if a high price level receives a higher utility then a low price level, we are likely to conclude that the respondent was simply confused. Rather than discard data for that respondent, it is often useful to provide additional information to the calculating program in the form of "constraints." For example, we may tell the calculating program that the utility for $1.00 must be no lower than the utility for $1.25.

With nonmetric methods it is easy to enforce such constraints. Since these methods are iterative, all that is necessary is to insure that the successive estimates at each stage obey the specified inequalities.

With OLS it is much more difficult to enforce such constraints. CVA provides that capability in a limited way: after the least squares solution is computed, then it is adjusted by "tying" values that violate specified inequalities. However, this is an inelegant way of solving the problem. When the data contain many violations of common sense relationships, then the nonmetric method provides a better way of enforcing desired constraints. However, more recent research has shown that HB estimation provides yet an even better way for estimating utilities that are less prone to reversals and for additionally enforcing utility constraints.

Additional Information

We have not provided details regarding utility estimation in this section. Please see How CVA Calculates Utilities for more information about OLS and Monotone Regression. Also see an introductory paper available at www.sawtoothsoftware.com in the Technical Papers library entitled "Analysis of Traditional Conjoint Using Excel: An Introductory Example." For information regarding HB estimation, please see two papers also available in the on-line Technical papers library: "The CVA/HB Technical Paper" and "Monotonicity Constraints in Choice-Based Conjoint with Hierarchical Bayes."

Settings for Utility Estimation

From Analysis | Calculate CVA Utilities using <OLS/Monotone>... if you click the Settings… button, you can review or change the settings for the estimation. The controls depend slightly whether you are using Ordinary Least Squares or Monotone Regression Estimation:

OLS Settings

Monotone Regression Settings

Additional Utility Constraints…: When you click this button, a new dialog opens in which you can specify Utility Constraints or override previously defined a priori orders for attributes. Utility constraints are useful if you have attributes that have a specific order such as price. It usually doesn't make sense that a higher price would be preferred to a lower price. However, utilities (especially when calculated at the individual level) sometimes include such "reversals." The researcher can smooth some of the noise out of conjoint data by providing additional information about the order of attribute levels. However, researchers often find that constraining utilities will slightly hurt share prediction accuracy. The benefit of adding constraints is that they can improve individual-level classification and prediction.

Important Note: If you specified a priori order when you specified attributes and levels, CVA will automatically constrain utilities of these ordered attributes to have no reversals. If you do not wish to apply constraints, you should override these a priori specifications by editing the Constraints.

When you are satisfied with your settings, close the dialog and click the Prepare for CVA OLS Computation... (or Prepare for CVA Monotone Regression Computation...) button. CVA automatically calculates part-worths for each respondent and provides a summary report on the screen. Respondents with no variation in the dependent variable or too many missing answers are not included in the utility run. The utility run is saved to a .UTL file for use in the SMRT Market Simulator.

Moving the Data into SMRT

SMRT stands for "Sawtooth Software Market Research Tools" and contains the market simulation tool used to analyze data resulting from any of Sawtooth Software's conjoint analysis systems. We'll now describe how you move data from your CVA project within the SSI Web Platform into SMRT.

When you calculated part-worth utilities using Analysis | Calculate CVA Utilities Using Ordinary Least Squares... (or via Monotone regression or HB) this process creates a text-only format file containing case IDs and part-worth estimates called STUDYNAME OLS.utl.

To move the CVA part-worth utility data into SMRT for analysis:

Start the SMRT software by clicking Start | Program Files | Sawtooth Software | Sawtooth Software SMRT.

Within SMRT, create a new study for analyzing the results (select File | New and choose a folder and a studyname, making sure to specify that this is a CVA study type). You can choose any folder or studyname that you want, as this new study functions independently of your original SSI Web study.

Import the STUDYNAME OLS.utl file (file of respondent part-worth utilities) into your SMRT study by clicking (from the SMRT software menu) Analysis | Run Manager | Import… and following the prompts for importing CVA utilities (selecting CVA (Traditional Conjoint) Utilities (*.utl) as the file type, and browsing to your .utl file). Once you import your utilities, you can select Analysis | Market Simulator and begin simulations.

Click close to return to the main menu. Now that you have imported the utilities, the Market Simulator menu item becomes active. From the main menu, click Analysis | Market Simulator.

You will note that there is a list of available Utility Runs (your recently computed run is available), and a list of Simulation Scenarios (currently empty). Click Compute!, and a report of average utilities and importances for your data set is displayed in the report window (these data should match the similar report that was earlier displayed within SSI Web, directly following utility estimation).