
	Technical Papers Downloads Pricing & Ordering Your Cart

Sawtooth Software » Education » Sawtooth Solutions

Education

Support

SS Winter 2004

Special Features of CBC Software for Packaged Goods and Beverage Research

This article is an excerpt from a more complete paper with the same title available for downloading from our Technical Papers library at www.sawtoothsoftware.com
Choice-Based Conjoint (CBC) software has been used extensively over the last decade for a variety of conjoint analysis problems. Among Sawtooth Software customers, its use has now eclipsed the use of ACA. CBC-type questionnaires are the most widely used conjoint method among all market researchers.
CBC particularly has found widespread use in packaged goods and beverage research. CBC’s popularity for this type of research is due to a number of benefits:

Asking respondents to make choices among sets of products is more realistic than rating them individually.

CBC can measure interaction effects more effectively than traditional conjoint (when using aggregate estimation routines, or HB which leverages estimates of population parameters), which often occur among brand, packaging, and price.

The resulting market simulation tool can estimate shares of choice, demand curves, and substitution (including cannibalization) effects.

The CBC questionnaire can be made to look quite a bit like choosing products from actual store shelves.
Under favorable conditions, CBC can produce quite accurate predictions of market share. Of course, predictions are never perfect. We believe the bulk of evidence suggests that CBC is one of the most useful research tools available for testing pricing, packaging, repositioning, and new product introductions for packaged goods. It cannot perfectly predict market shares or estimate elasticity curves, but we shouldn’t expect nor require it to do so. There are many other elements that influence market share, such as product life cycle, prominence of shelf facing, promotions, and distribution, to name a few. Under the assumptions of equal availability and information, CBC is able to predict useful shares of choice which, when applied in the context of what-if market simulations, can significantly increase the likelihood of making profitable marketing decisions.

Shelf-Facing Presentation
The newest version of our CBC/Web Advanced Design Module supports “shelf-facing” presentation, as shown above. We needed to implement three new features in the software to support the shelf-facing look.

In the shelf display shown previously, there are 29 different products. Previous versions of CBC/Web only supported up to 15 levels per attribute and a maximum of 16 concepts per task, so it was impossible to show so many unique products on the screen at once. The new version of the CBC/Web Advanced Design Module can include up to 100 levels for an attribute and up to 100 concepts within a task, which should offer great flexibility for showing quite complex packaged goods displays.

Some package sizes are larger than others (or the researcher may want to include more units in a graphic to represent more linear shelf space), so the software needed to support differing widths of product concepts. Also, we needed to make the software flexible so that if multiple rows of products were displayed, the number of products shown per shelf did not need to be constant.

We needed to allow the researcher to specify that the brands should not change positions on the screen across choice tasks (suppress the randomization of level order for brand). We expect most researchers choosing shelf-type display will prefer fixed positions for brands. But, if desired, brands can have randomized position, either across tasks, or held constant within a respondent interview but randomized across respondents.
CBC/Web leverages web browser technology to display the product concepts. This leads to a great deal of flexibility in the programming and in fielding the questionnaire. HTML and JavaScripting elements can be added by the author, to add elements such as pop-ups showing more detail about a package (detailed image and text) if the graphic is clicked. The surveys can be either fielded over the Web or in CAPI mode, from a PC or laptop not connected to the Internet, or even via paper-and-pencil. These benefits lead to greater realism in the interview, and greater flexibility for interviewing respondents.
The Generic “Conjoint-Style” Case
In the 1970s, researchers began to use conjoint analysis for business problems. With traditional conjoint analysis, each attribute had multiple levels, and the levels for each attribute could generically combine with all other levels of the other attributes. For example, a small conjoint study might have three brands, three sizes, and three prices:
Brand:
Brand A
Brand B
Brand C
Package Size:
Small
Medium
Large
Price:
$1.00
$1.25
$1.50
With a generic, balanced conjoint design, each level of price occurs an equal number of times with each brand and size level. This immediately posed limitations. The prices for different brands, or especially package sizes, in reality might be quite different, but the conjoint interview couldn’t reflect that.
The Prohibitions Trap
One of the most common mistakes CBC users make is to try to specify unique price ranges for different brands using level prohibitions. For example, some users have expanded the number of prices, and created prohibitions between, say, package size and price. The table below represents such a “prohibitions table,” where the “Xs” indicate combinations that are prohibited.

$0.75 $1.00 $1.25 $1.50 $1.75

Small X X

Medium X X

Large X X

Indeed, the prohibitions table above leads to a CBC survey in which the Small package is generally shown at lower prices, and the Large package at higher prices. The combinations presented to respondents seem more realistic. But, the resulting data are often poor, or entirely unusable. That is because the researcher is still asking the CBC software to estimate the part worth utilities for package and price as if they were independent (main effects). But, because the levels within the attributes are strongly correlated, one often cannot estimate the separate effects of the levels with good precision. Sometimes, the prohibitions specified may be so extreme that the part worths cannot be estimated at all (given the current main-effects model specification).
CBC software includes a Test Design function that shows the relative efficiency values for the main effects, and can thus point to specific problem areas in the design. If you include any prohibitions between attributes, you must use the Test Design function prior to fielding the study.
Conditional Pricing
Due to the problems discussed in the previous section, researchers sought ways to customize the price ranges for the different brands and package sizes, but that would still lead to efficient estimates for main effects and interactions. Conditional pricing offers a solution. (CBC for Windows has always offered conditional pricing, but conditional pricing wasn’t available within CBC/Web until the most recent release.)
With conditional pricing, a “look-up” table is provided. We can use the example from the previous section to create a conditional pricing table.

Low Price Medium Price High Price

Small $0.75 $1.00 $1.25

Medium $1.00 $1.25 $1.50

Large $1.25 $1.50 $1.75

Notice that we still have specified just two attributes for our design, each with three levels. But, we have created a series of conditional prices that are shown in the questionnaire, depending on the package size and price level. The interview looks correct, because the right combinations of prices are shown with the package sizes. But, more importantly, the data no longer are hindered by any prohibitions. We’ve solved these initial problems, but left ourselves with slightly more challenging issues for back-end analysis.
By default, the software still estimates the main effects for the brands and prices. However, these main effects are no longer interpreted as the preference for each level, holding all else constant. All else was not held constant. For example, the main effect for the Large package size captures the preference for Large package sizes given the average prices shown for Large packages. Thus, included in the parameter estimate for Large package size is a negative utility intercept to compensate for the average increased prices shown with Large packages.
When one builds conditional pricing tables, this often leads to the need to estimate additionally the interaction between the attributes involved in the conditional pricing grid. This is especially the case if the price differences from level to level, for each package size, were not so uniform as portrayed in the grid above. With aggregate logit and latent class, additional interactions are often required. That is because many interactions observed at the aggregate level are just due to unrecognized heterogeneity (i.e. the same people who prefer premium brands are also less price sensitive). With individual-level estimation (HB), if the interactions are principally due to unrecognized heterogeneity, one can often obtain excellent models with main effects estimation only.
We’ve discussed conditional pricing with respect to one attribute: Package Size. However, in our CBC software it is possible to make prices conditional on the combination of up to three attributes other than price. Researchers studying packaged goods categories often need to create conditional prices, depending on the package size and the brand. But this often leads to the “brand/package size prohibitions trap,” described in the next section.
The Brand/Package Size Prohibitions Trap
Clients often approach CBC users with a list of, say, 18 brand and package size combinations, where the prices also need to be unique for these SKUs. The CBC user recognizes that the past CBC software only permitted up to 15 levels per attribute. So, the only way to represent all 18 brand and package size combinations was to specify brand and package size as separate attributes, associate a conditional graphic with those combinations, and prohibit any of the brand and package size combinations that didn’t apply. For example, perhaps there were 6 unique brands and 5 unique package sizes. That leads to 30 possible combinations, of which the researcher needs to prohibit 12 of them. This is clearly the same problem as the “prohibitions trap” described earlier, and such prohibitions may lead to an inefficient (or even deficient) design.
Even if the prohibitions between brand and package were very modest, leading to reasonable efficiencies for main effects, this procedure would not permit the estimation of interactions between brand and package. If the preference for a package really depends on the brand attached to it, main effects estimation will lead to improper conclusions. Many CBC researchers have used too many prohibitions over the years, with two results: reduced overall design efficiency and the inability to assess whether modeling the main effects for brand independent of package size provided an adequate fit to represent peoples’ preferences for joint brand/package combinations. We suspect this has been one of the most common mistakes committed with our CBC software over the years.
An adequate solution to this problem was not available in CBC software until most recently, with the most recent release of CBC/Web Advanced Design Module. In this situation, the researcher needs to be able to specify the brand/package size combinations as a single attribute, with all 18 levels. This explicitly accounts for the independent preference for each brand/package combination (the interaction between brands, packages and price). Once the 18-level attribute is specified, a conditional pricing table can be specified for these 18 brand/package levels.
Packaged Goods Research and Market Simulations
Sawtooth Software provides different market simulation approaches for competitive contexts: first choice, share of preference, share of preference with correction for product similarity (not suggested, but available for historical purposes only), and Randomized First Choice (RFC). RFC’s approach provides correction (share reduction) for pairs of products that are defined similarly in terms of their attribute levels. In general, we have found RFC to perform well in methodological studies involving, say, five or more attributes, especially when using aggregate logit or latent class results. There is evidence that RFC provides some benefit over standard share of preference simulations when using individual-level part worths (such as from HB), but the benefits are not as dramatic as with group-based models.
Recently, we’ve recognized that RFC may be less useful or not useful at all for two-attribute studies involving brand/package and conditional prices. In conditional pricing tables, level “1” for one brand/package may mean $2.00, but level “1” for another brand/package might mean “$5.25.” However, RFC is blind to this difference, assuming only that since they both share level 1 for price, they must be identically defined on price. The resulting correction may not be desirable.
We generally recommend using HB estimation for CBC studies, and especially if using conditional pricing tables we recommend that you test whether RFC or share of preference best fit holdout observations. Make sure to tune the exponent (scale factor) for the different models to best predict holdouts prior to comparing results. Recent evidence involving packaged goods research and real-world sales data suggests that RFC may offer little benefit over share of preference in these cases. If this finding holds, you can save a great deal of computing effort using the faster share of preference method, which will especially pay off if using the computationally-intensive Advanced Simulation Module for optimization searches.
Go back to Index

Call for Papers--Eleventh Sawtooth Software Conference

On October 4-8, we will hold our eleventh Sawtooth Software Conference in San Diego, California. Our research conference brings together some of the best minds in our industry to talk about practical issues in computer/Web interviewing and quantitative market research. It is not a sales-oriented event for our software, but a chance to exchange ideas and receive education from a variety of sources and perspectives. Papers presented at our previous Sawtooth Software Conferences are cited frequently in journal articles.
We're looking for exceptionally strong papers. If you'd like to be on the program, please respond promptly (by April 2, email: bryan@sawtoothsoftware.com) with a one-page abstract describing your proposed paper, with special attention to the findings and what the audience will "take away" from the presentation. You must also include a 50-word description of your paper to include in the conference brochure, should your abstract be accepted.
We are interested in papers on a variety of subjects, including Web interviewing, market segmentation, scale development, customer satisfaction modeling, conjoint/choice analysis, perceptual mapping, hierarchical Bayes methods, forecasting, pricing research, market simulations and case studies. These papers need not involve Sawtooth Software's programs or approach.
In an effort to provide more balance to the program, we are encouraging papers that are NOT about conjoint/choice modeling. We are also placing special emphasis this year on case studies and nuts-and-bolts aspects of conducting quantitative market research and delivering results to managers. For all topics, we are eager to see evidence of managerial relevance, external validity, profit impact, etc.
Presenters receive a complimentary conference registration. To be accepted, a paper must show promise of being sufficiently practical to be of use to the least sophisticated members of the audience, while having enough substance to be of interest to the most sophisticated members. In addition to standard presentation slides, authors are required to submit a journal-quality written paper for publication in the Conference Proceedings.
We strive for the highest quality in our conferences. If your abstract is accepted, a member of the steering committee will review early drafts of your presentation and offer suggestions. Authors are expected to consider these suggestions conscientiously and rework their presentations as needed. Sawtooth Software reserves the right to remove any author from the program or proceedings that fails to meet deadlines or produce high quality work.
Sawtooth Software Conference 2004 Steering Committee Members are:

Bryan Orme, Sawtooth Software
Karlan Witt, Customer Metrics Group
Dick Wittink, Yale University
Joel Huber, Duke University
Go back to Index

New Advances Shed Light on HB Anomalies

Hierarchical Bayes estimation for choice data represents one of the most successful new developments in our field. HB has proven robust for full-profile CBC projects, and tests comparing HB to other methods of part worth estimation have generally favored HB. However, two anomalies specific to HB estimation have caused us some puzzlement and concern.
“Omitted” Level Estimation under Effects Coding
In “The Joys and Sorrows of Implementing HB Methods for Conjoint Analysis” presented at the 1999 HB Conference at Ohio State, our Chairman, Rich Johnson, showed that estimation of the last (omitted) level for each attribute under effects coding was problematic. The part worth for the omitted level (the reference level, constrained to be negative the sum of the other levels within the same attribute) had larger variance than other levels within the same attribute. The more levels for an attribute, the more pronounced the inflation of variance for the omitted level. Rich suggested the opposite was true for dummy coding (the omitted level has a lower variance relative to the other parameters). He concluded, “Thus, with HB, unlike OLS, it makes a difference how the data are coded. I don’t know what the practical consequences of this will turn out to be. But, like many other things about Bayesian analysis, it was a surprise to me.”
We have since learned that for most data sets the bias in variance for the omitted level under effects coding has been of little practical consequence. But, for particularly sparse data sets, especially when there are many levels within an attribute, the estimation of the omitted level can be severely compromised. Not only is the variance amplified, but for the most extreme cases, we’ve noted just recently that the point estimate itself can be significantly biased in the negative direction.
Partial-Profile CBC and HB Anomaly
In a paper presented at the 2001 Sawtooth Software Conference (“The Effects of Disaggregation with Partial Profile Choice Experiments”), Jon Pinnell and Lisa Fridley analyzed nine different partial-profile CBC studies that had been conducted by three different research agencies. Using our CBC/HB software and an aggregate logit solution, they compared the ability of the respective part worths to predict each individual’s choices to different random choice tasks that were “held out.” We would naturally expect that the HB solution should improve individual-level classification rates relative to an aggregate model where all respondents are pooled. Surprisingly, the authors found that HB did worse in four of the nine data sets, and offered no improvement for two others. We’ve puzzled over this finding, as it seems that the data borrowing mechanism in HB should appropriately leverage the more robust information available from the population parameters relative to the relatively sparse data available at the individual level. Only recently do we have an explanation and a better solution.
Improving the Prior Covariance Matrix Specifications
In the hierarchical Bayes world, we begin with a prior assumption about individual and population parameters and update that information as new data are added. The population-level priors consist of a vector of means and a covariance matrix. The degrees of freedom are also specified for the prior covariance matrix, indicating how much weight should be given to the priors versus the data.
Academics have generally suggested using zero for the means, a covariance matrix proportional to an identity matrix (some positive constant across the diagonal, and zeros in the off-diagonal elements), and degrees of freedom for the covariance matrix equal to the number of parameters to be estimated (plus a small integer constant). When there is enough information available at the individual level relative to the number of parameters to be estimated, these prior assumptions have very little effect on the posterior part worth estimates. However, under extreme conditions such as having many levels of an attribute under effects coding, or estimating many parameters from sparse partial-profile CBC designs, proper specification of the prior covariance matrix matters.
With direction from Peter Lenk of the University of Michigan (a leading academic in HB) we have modified how our CBC/HB software sets the prior covariance matrix. If effects-coding is specified, we introduce appropriate negative covariances in the off-diagonal elements of the prior covariance matrix, to reflect the fact that levels within each attribute are necessarily negatively correlated. This step resolves the problems stated above for estimating the part worths for omitted levels. Next, we permit the user to set the prior variance along with the degrees of freedom for the prior covariance matrix, thereby tuning the assumed between-respondent variance and relative contribution of the priors versus the data. This tuning can be important for modeling sparse data sets, such as is the case with some partial-profile CBC designs.
We have examined two of the partial-profile CBC data sets that Pinnell and Fridley found problematic when they used our previous CBC/HB software. The assumptions regarding the priors in the old CBC/HB software seemed to lead to overfitting at the individual level (and reduction in hit rate accuracy to holdouts) for these data sets. Using the new CBC/HB software, we decreased the prior variance assumption (assuming greater homogeneity in the population) and increased the degrees of freedom for the prior covariance matrix (increasing the weight for the prior). The hit rates for the new HB solutions improved, and were now slightly higher than aggregate logit in both cases. We believe the small remaining difference in hit rate is due to a combination of low heterogeneity in the sample and little information available for each individual relative to the number of parameters to be estimated.
The important take-away for partial-profile CBC is that the previous failures that Pinnell and Fridley illustrated for HB were not inherent to HB or partial profile designs, but were a result of the fixed priors we used in our software that were suboptimal for these particular data sets. With the newest CBC/HB software (v3), the researcher can tune the priors, avoiding overfitting in these unusual cases. We should emphasize, however, that the defaults in both the previous and current versions of CBC/HB software seem to work very well for most CBC data sets in practice--especially full-profile designs.
To learn more about the enhancements in the newest version of CBC/HB software, please refer to the CBC/HB v3 Technical Paper in our Technical Papers library at www.sawtoothsoftware.com.
More Flexible Priors in HB-Regression
The ability to tune the prior variance and degrees of freedom for the prior covariance matrix may be even more valuable in the context of our generalized HB program for regression-based problems, called HB-Reg. With our other conjoint-based HB systems, we could make reasonable assumptions about the relative scaling of the dependent variable and the conjoint part worth coefficients. In contrast, HB-Reg is a generalized system for analyzing many types of user-formatted data, and we cannot make general assumptions regarding the scaling of the variables and the related measures of variance. The previous version of HB-Reg assumed a fixed prior variance for betas. If there was relatively sparse information available at the respondent level (which is often the case) and if the data were scaled much differently from the prior variance assumption, the estimation could be sub-optimal or even incorrect.
The new version of HB-Reg (version 3) permits the user to modify the prior settings as described previously for CBC/HB. Additionally, for very advanced users we’ve provided a way to supply a user-specified prior covariance matrix. These improvements should increase the overall applicability and value of HB-Reg. The new Windows interface in version 3 should also make it easier to use, and the graphical display of the parameter estimates by iteration leads to easier determination of convergence.
Go back to Index

Advanced Conjoint Analysis Training in Europe: May 13-14, 2004

We are pleased to announce advanced conjoint analysis training co-sponsored by Sawtooth Software and our European distributor, SKIM, to be held in Rotterdam, Netherlands. This two-day training is designed for those who already know the essentials of conjoint analysis. Sawtooth Software’s President Christopher King and Vice President Bryan Orme will teach most of the sessions. The level of presentations will be more technical than typical Sawtooth Software trainings. Preliminary topics include:

Advanced CBC designs: theory and practical examples
Design testing: going beyond the Test Design function in CBC
Compromising orthogonality: prohibitions and utility balance
Aggregate and disaggregate re-weighting of Price in ACA
Mathematics and design principles behind ACA
MaxDiff and paired comparisons: design, coding, and estimation
Analytical examples using CBC/HB and HB-Reg software
Latent Class, ICE, and HB: critical results and comparisons
Product optimization searches
Perceptual mapping
Please contact info@skim.nl for more information, or watch SKIM Software division’s website at www.skim.nl. We’ll also keep you posted regarding details via Enews, our email newsletter. We distribute Enews about every 8-10 weeks. If you are not already receiving Enews, please register your email address at our website www.sawtoothsoftware.com.