Sawtooth Software: The Survey Software of Choice

Special Features of CBC Software for Packaged Goods and Beverage Research

This article is an excerpt from a more complete paper with the same title available for downloading from our Technical Papers library at www.sawtoothsoftware.com.

Choice-Based Conjoint (CBC) software has been used extensively over the last decade for a variety of conjoint analysis problems. Among Sawtooth Software customers, its use has now eclipsed the use of ACA. CBC-type questionnaires are the most widely used conjoint method among all market researchers.

CBC particularly has found widespread use in packaged goods and beverage research. CBC’s popularity for this type of research is due to a number of benefits:

  • Asking respondents to make choices among sets of products is more realistic than rating them individually.
  • CBC can measure interaction effects more effectively than traditional conjoint (when using aggregate estimation routines, or HB which leverages estimates of population parameters), which often occur among brand, packaging, and price.
  • The resulting market simulation tool can estimate shares of choice, demand curves, and substitution (including cannibalization) effects.
  • The CBC questionnaire can be made to look quite a bit like choosing products from actual store shelves.

Under favorable conditions, CBC can produce quite accurate predictions of market share. Of course, predictions are never perfect. We believe the bulk of evidence suggests that CBC is one of the most useful research tools available for testing pricing, packaging, repositioning, and new product introductions for packaged goods. It cannot perfectly predict market shares or estimate elasticity curves, but we shouldn’t expect nor require it to do so. There are many other elements that influence market share, such as product life cycle, prominence of shelf facing, promotions, and distribution, to name a few. Under the assumptions of equal availability and information, CBC is able to predict useful shares of choice which, when applied in the context of what-if market simulations, can significantly increase the likelihood of making profitable marketing decisions.

Shelf-Facing Presentation

The newest version of our CBC/Web Advanced Design Module supports “shelf-facing” presentation, as shown above. We needed to implement three new features in the software to support the shelf-facing look.

  1. In the shelf display shown previously, there are 29 different products. Previous versions of CBC/Web only supported up to 15 levels per attribute and a maximum of 16 concepts per task, so it was impossible to show so many unique products on the screen at once. The new version of the CBC/Web Advanced Design Module can include up to 100 levels for an attribute and up to 100 concepts within a task, which should offer great flexibility for showing quite complex packaged goods displays.
  2. Some package sizes are larger than others (or the researcher may want to include more units in a graphic to represent more linear shelf space), so the software needed to support differing widths of product concepts. Also, we needed to make the software flexible so that if multiple rows of products were displayed, the number of products shown per shelf did not need to be constant.
  3. We needed to allow the researcher to specify that the brands should not change positions on the screen across choice tasks (suppress the randomization of level order for brand). We expect most researchers choosing shelf-type display will prefer fixed positions for brands. But, if desired, brands can have randomized position, either across tasks, or held constant within a respondent interview but randomized across respondents.

CBC/Web leverages web browser technology to display the product concepts. This leads to a great deal of flexibility in the programming and in fielding the questionnaire. HTML and JavaScripting elements can be added by the author, to add elements such as pop-ups showing more detail about a package (detailed image and text) if the graphic is clicked. The surveys can be either fielded over the Web or in CAPI mode, from a PC or laptop not connected to the Internet, or even via paper-and-pencil. These benefits lead to greater realism in the interview, and greater flexibility for interviewing respondents.

The Generic “Conjoint-Style” Case

In the 1970s, researchers began to use conjoint analysis for business problems. With traditional conjoint analysis, each attribute had multiple levels, and the levels for each attribute could generically combine with all other levels of the other attributes. For example, a small conjoint study might have three brands, three sizes, and three prices:

Brand:
Brand A
Brand B
Brand C

Package Size:
Small
Medium
Large

Price:
$1.00
$1.25
$1.50

With a generic, balanced conjoint design, each level of price occurs an equal number of times with each brand and size level. This immediately posed limitations. The prices for different brands, or especially package sizes, in reality might be quite different, but the conjoint interview couldn’t reflect that.

The Prohibitions Trap

One of the most common mistakes CBC users make is to try to specify unique price ranges for different brands using level prohibitions. For example, some users have expanded the number of prices, and created prohibitions between, say, package size and price. The table below represents such a “prohibitions table,” where the “Xs” indicate combinations that are prohibited.

  $0.75 $1.00 $1.25 $1.50 $1.75
Small       X X
Medium X       X
Large X X      

Indeed, the prohibitions table above leads to a CBC survey in which the Small package is generally shown at lower prices, and the Large package at higher prices. The combinations presented to respondents seem more realistic. But, the resulting data are often poor, or entirely unusable. That is because the researcher is still asking the CBC software to estimate the part worth utilities for package and price as if they were independent (main effects). But, because the levels within the attributes are strongly correlated, one often cannot estimate the separate effects of the levels with good precision. Sometimes, the prohibitions specified may be so extreme that the part worths cannot be estimated at all (given the current main-effects model specification).

CBC software includes a Test Design function that shows the relative efficiency values for the main effects, and can thus point to specific problem areas in the design. If you include any prohibitions between attributes, you must use the Test Design function prior to fielding the study.

Conditional Pricing

Due to the problems discussed in the previous section, researchers sought ways to customize the price ranges for the different brands and package sizes, but that would still lead to efficient estimates for main effects and interactions. Conditional pricing offers a solution. (CBC for Windows has always offered conditional pricing, but conditional pricing wasn’t available within CBC/Web until the most recent release.)

With conditional pricing, a “look-up” table is provided. We can use the example from the previous section to create a conditional pricing table.

  Low Price Medium Price High Price
Small $0.75 $1.00 $1.25
Medium $1.00 $1.25 $1.50
Large $1.25 $1.50 $1.75

Notice that we still have specified just two attributes for our design, each with three levels. But, we have created a series of conditional prices that are shown in the questionnaire, depending on the package size and price level. The interview looks correct, because the right combinations of prices are shown with the package sizes. But, more importantly, the data no longer are hindered by any prohibitions. We’ve solved these initial problems, but left ourselves with slightly more challenging issues for back-end analysis.

By default, the software still estimates the main effects for the brands and prices. However, these main effects are no longer interpreted as the preference for each level, holding all else constant. All else was not held constant. For example, the main effect for the Large package size captures the preference for Large package sizes given the average prices shown for Large packages. Thus, included in the parameter estimate for Large package size is a negative utility intercept to compensate for the average increased prices shown with Large packages.

When one builds conditional pricing tables, this often leads to the need to estimate additionally the interaction between the attributes involved in the conditional pricing grid. This is especially the case if the price differences from level to level, for each package size, were not so uniform as portrayed in the grid above. With aggregate logit and latent class, additional interactions are often required. That is because many interactions observed at the aggregate level are just due to unrecognized heterogeneity (i.e. the same people who prefer premium brands are also less price sensitive). With individual-level estimation (HB), if the interactions are principally due to unrecognized heterogeneity, one can often obtain excellent models with main effects estimation only.

We’ve discussed conditional pricing with respect to one attribute: Package Size. However, in our CBC software it is possible to make prices conditional on the combination of up to three attributes other than price. Researchers studying packaged goods categories often need to create conditional prices, depending on the package size and the brand. But this often leads to the “brand/package size prohibitions trap,” described in the next section.

The Brand/Package Size Prohibitions Trap

Clients often approach CBC users with a list of, say, 18 brand and package size combinations, where the prices also need to be unique for these SKUs. The CBC user recognizes that the past CBC software only permitted up to 15 levels per attribute. So, the only way to represent all 18 brand and package size combinations was to specify brand and package size as separate attributes, associate a conditional graphic with those combinations, and prohibit any of the brand and package size combinations that didn’t apply. For example, perhaps there were 6 unique brands and 5 unique package sizes. That leads to 30 possible combinations, of which the researcher needs to prohibit 12 of them. This is clearly the same problem as the “prohibitions trap” described earlier, and such prohibitions may lead to an inefficient (or even deficient) design.

Even if the prohibitions between brand and package were very modest, leading to reasonable efficiencies for main effects, this procedure would not permit the estimation of interactions between brand and package. If the preference for a package really depends on the brand attached to it, main effects estimation will lead to improper conclusions. Many CBC researchers have used too many prohibitions over the years, with two results: reduced overall design efficiency and the inability to assess whether modeling the main effects for brand independent of package size provided an adequate fit to represent peoples’ preferences for joint brand/package combinations. We suspect this has been one of the most common mistakes committed with our CBC software over the years.

An adequate solution to this problem was not available in CBC software until most recently, with the most recent release of CBC/Web Advanced Design Module. In this situation, the researcher needs to be able to specify the brand/package size combinations as a single attribute, with all 18 levels. This explicitly accounts for the independent preference for each brand/package combination (the interaction between brands, packages and price). Once the 18-level attribute is specified, a conditional pricing table can be specified for these 18 brand/package levels.

Packaged Goods Research and Market Simulations

Sawtooth Software provides different market simulation approaches for competitive contexts: first choice, share of preference, share of preference with correction for product similarity (not suggested, but available for historical purposes only), and Randomized First Choice (RFC). RFC’s approach provides correction (share reduction) for pairs of products that are defined similarly in terms of their attribute levels. In general, we have found RFC to perform well in methodological studies involving, say, five or more attributes, especially when using aggregate logit or latent class results. There is evidence that RFC provides some benefit over standard share of preference simulations when using individual-level part worths (such as from HB), but the benefits are not as dramatic as with group-based models.

Recently, we’ve recognized that RFC may be less useful or not useful at all for two-attribute studies involving brand/package and conditional prices. In conditional pricing tables, level “1” for one brand/package may mean $2.00, but level “1” for another brand/package might mean “$5.25.” However, RFC is blind to this difference, assuming only that since they both share level 1 for price, they must be identically defined on price. The resulting correction may not be desirable.

We generally recommend using HB estimation for CBC studies, and especially if using conditional pricing tables we recommend that you test whether RFC or share of preference best fit holdout observations. Make sure to tune the exponent (scale factor) for the different models to best predict holdouts prior to comparing results. Recent evidence involving packaged goods research and real-world sales data suggests that RFC may offer little benefit over share of preference in these cases. If this finding holds, you can save a great deal of computing effort using the faster share of preference method, which will especially pay off if using the computationally-intensive Advanced Simulation Module for optimization searches.