[Download Paper]

"Adaptive Conjoint Analysis" (ACA) is software for conjoint (trade-off) analysis. The term "adaptive" refers to the fact that the computer-administered interview is customized for each respondent. Data are analyzed as the interview progresses, and we choose questions likely to reveal the most about the respondent's values in the shortest time. ACA is an excellent alternative to full-profile conjoint when you have a large number of attributes. This paper provides a description of the adaptive technique, including technical details.

[Download Paper]

"Adaptive Choice-Based Conjoint" (ACBC) is a new approach for adaptive choice-based conjoint studies. The interview has three main phases: 1) BYO (configuration) phase, 2) Consideration phase, 3) Choice phase. This follows the common proposition that buyers develop consideration sets and then choose a final product from within the consideration set. The interview adapts to each respondent, giving a more relevant, interactive experience. ACBC is recommended for experienced conjoint analysts and for projects involving about 5 or more attributes. Traditional brand-package-price studies should continue to be conducted under the standard (non-adaptive) CBC software.

[Download Paper]

Hierarchical Bayes is a relatively new technique that can be used in estimating part worths for conjoint analysis experiments. HB has been described favorably in several recent journal articles. Its strongest point of differentiation is its ability to provide estimates of individual part worths given limited information from each individual. It does this by "borrowing" information from other individuals.

This technical paper describes the intuition and math behind HB, including results that suggest that ACA utilities computed using HB are generally superior to those generated by the ACA system under OLS. ACA/HB utilities generally produce better hit rates and more accurate share predictions of holdout validation data. Furthermore, ACA/HB provides a more theoretically sound way to combine information from ACA's priors and pairs.

We at Sawtooth Software are not experts in Bayesian data analysis. In producing this software we have been helped by several sources. We have benefited particularly from the materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing Association's Advanced Research Techniques Forum.

[Download Paper]

The Advanced Simulation Module (ASM) extends the capabilities of the standard market simulator software for ACA, CBC, and CVA to enable product optimization searches, based on the criteria of utility, share, purchase likelihood, revenue, profit or cost minimization. Search routines include hill-climbing methods, exhaustive search, and Genetic Algorithms. Product optimizations are well-suited for finding optimal products considered alone, or relative to a set of competitors. Cost information can be associated with attribute levels in the study. With cost information, the analyst can perform profit maximization searches, or can search for products that maximize some performance threshold relative to a cost limit specified by the user.

[Download Paper]

"Choice-Based Conjoint" Analysis (CBC) is an integrated component within the SSI Web platform for conducting choice-based conjoint studies. The main characteristic distinguishing choice-based conjoint analysis from other types is that the respondent expresses preferences by choosing concepts from sets of concepts, rather than by rating or ranking them. This paper discusses the method of choice-based conjoint analysis from a practitioner-oriented point of view, and describes Sawtooth Software's CBC System for choice-based conjoint analysis in some detail. It also provides suggestions about how to select a particular conjoint method from the variety of those available, considering characteristics of the research problem at hand.

[Download Paper]

Some CBC projects do not fit the traditional mold (full-profile, common attributes, limited attributes and levels). The Advanced Design Module for CBC gives the researcher additional capabilities:

• Alternative-specific plans
• Partial-profile interviewing format
• Capacity extended to 30 attributes
• Capacity extended to 254 levels per attribute, and 100 concepts per task (CBC/Web only)
• Shelf-facing display (CBC/Web only)

This paper covers the intuition and quantitative concepts behind these more advanced approaches. With the Advanced Design Module, researchers are better equipped to handle a variety of challenging requests and modeling opportunities.

[Download Paper]

The CBC Latent Class Module is an add-on system to the CBC System for Choice-Based Conjoint. Latent Class analysis is a technique for dividing respondents into segments having similar preferences. Latent Class simultaneously estimates utilities for each segment and the probability that each respondent belongs to each segment.

Latent Class can improve the quality of marketing simulations over traditional aggregate logit modeling. It helps reduce IIA (red bus/blue bus) problems. This module also lets you fit linear terms to quantitative attributes.

[Download Paper]

Hierarchical Bayes is a relatively new technique for computing individual- level part worths from CBC data. HB has been described favorably in numerous journal articles. Its strongest point of differentiation is its ability to provide estimates of individual part worths given only a few choices by each individual. It does this by "borrowing" information from other individuals.

This technical paper describes the intuition and math behind HB, including results that suggest that HB is generally superior relative to aggregate approaches for estimating individual's choices and aggregate share predictions. We at Sawtooth Software are not experts in Bayesian data analysis. In producing this software we have been helped by several sources. We have benefited particularly from the materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing Association's Advanced Research Techniques Forum.

[Download Paper]

"Convergent Cluster & Ensemble Analysis" (CCEA) is software for doing cluster and cluster ensemble analysis. CCEA uses k-means as its standard cluster algorithm. However, the newer Ensemble Analysis included in the software is shown to produce better results for artificial datasets generated with known group membership. The procedure for the ensemble analysis is described in detail. Comparisons to k-means as provided by our previous CCA software are shown.

[Download Paper]

"Conjoint Value Analysis" (CVA) is an integreated component within our SSI Web platform for full-profile conjoint analysis. CVA interviews can be conducted using the software's built-in web-based interviewing program or data can be gathered using a paper-and-pencil approach. CVA can be used to manage all aspects of card-sort (single-concept) or pairwise comparison conjoint studies. This paper provides a description and technical details of CVA.

[Download Paper]

Hierarchical Bayes is a relatively new technique for computing individual- level part worths from conjoint data. HB has been described favorably in several recent journal articles. Its strongest point of differentiation is its ability to provide estimates of individual part worths given limited information from each individual. It does this by "borrowing" information from other individuals.

This technical paper describes the functionality of the software and math behind HB. We at Sawtooth Software are not experts in Bayesian data analysis. In producing this software we have been helped by several sources. We have benefited particularly from the materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing Association's Advanced Research Techniques Forum.

[Download Paper]

Hierarchical Bayes is a relatively new technique for computing individual- level estimates of regression coefficients or part worths. HB has been described favorably in several recent journal articles. Its strongest point of differentiation is its ability to provide estimates of individual parameters given only a few observations by each individual. It does this by "borrowing" information from other individuals.

HB-Reg is a generalized software program for running Regression-based HB. The user provides the data in an Excel-compatible .csv file. Potential uses for HB-Reg include traditional ratings-based conjoint experiments, customer satisfaction studies, or price elasticity measurement from scanner data.

This technical paper describes the intuition and math behind HB, including results that suggest that HB is generally superior relative to aggregate approaches for estimating individual's regression coefficients or part worths for conjoint experiments. In producing this software we have been helped by several sources. We have benefited particularly from the materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing Association's Advanced Research Techniques Forum.

[Download Paper]

This paper describes the technical procedures used in the MaxDiff System. MaxDiff (best-worst) scaling is a trade-off method for measuring the importance or preference for multiple items, such as brands, product features, political platforms, advertising claims, etc. Any time you are considering using a rating scale, ranking scale, or constant sum scale for multiple items, you can consider using MaxDiff.

The MaxDiff methodology, originally invented by researcher and academic Jordan Louviere, has gained in popularity over the last five years. Papers on MaxDiff have won "best presentation" awards at recent ESOMAR and Sawtooth Software research conferences. It has many similarities to, but is distinctively different, from conjoint methodology and is appropriate for a wider range of research opportunities.

Sawtooth Software’s MaxDiff System may be used for conducting web-based, CAPI, or paper-based MaxDiff studies. The software also supports asking the "best" half of the question only (not requiring respondents to identify the "worst" item in each set). The software may also be used for Method of Paired Comparisons research. Individual-level estimation of item scores employs Sawtooth Software’s popular hierarchical Bayes (HB) engine. Results may also be analyzed with the integrated Latent Class procedure for segmentation analysis.

[Download Paper]

Increasingly, stated preference choice projects involve Menu-Based Choice scenarios (MBC) where respondents can select one to multiple options from a menu. This is not surprising, given the fact that buyers commonly are allowed to customize products and services (mass customization).

The software takes a data processing and analysis process that can take experienced analysts from one to two weeks to do and compresses the timeline to 1 to 2 days.

MBC requires more expertise to use properly than our other conjoint analysis tools. The user should have solid background in CBC and multivariate statistical modeling, especially in terms of building models (regression, MNL) and the theory behind coding independent variables. While the software manages most details involving the data processing, independent variable coding, model estimation and simulations, the user must understand and direct the process intelligently.

[Download Paper]

This article is for those wanting a quick and understandable introduction to conjoint analysis. The basics of conjoint measurement are demonstrated using an example about optimizing golf balls in terms of price, durability, and performance.

[Download Paper]

Conjoint analysis has become the most popular and useful way to measure respondents’ preferences for simple to complex offerings and predict market choices. This article is taken from Chapter 1 of Getting Started with Conjoint Analysis, a book written by Sawtooth Software president Bryan Orme. It provides a non-technical managerial overview of the technique. It describes the history of the method, the various flavors, its practical uses, and recent developments that have made conjoint analysis even more powerful.

[Download Paper]

This paper (a chapter from the book Getting Started with Conjoint Analysis) illustrates how conjoint can be used to provide managers strategic marketing information that is intuitive and actionable. It explains how Choice-Based Conjoint (CBC) can be used to measure brand equity and determine brand sensitivity. The article focuses on how to get managers to "buy in" to conjoint, and some pitfalls to avoid when presenting conjoint data.

[Download Paper]

Sawtooth Software offers different conjoint analysis packages, including choice-based (discrete choice) methologies (CBC, ACBC), as well as the older ratings based approaches (ACA, CVA). This paper discusses the main differences between these approaches and offers suggestions regarding applicability to different research situations.

[Download Paper]

Covers the essentials for interpreting conjoint analysis data, including part worths, importances, shares of preference and "counting" analysis. The framework for interpreting results is developed from formal definitions of scaled data: Nominal, Ordinal, Interval, and Ratio. Common errors in interpreting conjoint analysis are highlighted.

[Download Paper]

Conjoint analysis has been a great success story for the marketing research industry. This paper outlines its development from the late 1960s through the early 2000s. The earliest conjoint analysis approaches were based on either full-profile card sort, or Johnson's tradeoff matrix. Later, Adaptive Conjoint Analysis and discrete choice (CBC) applications dominated. The use of CBC accelerated in the 1990s due largely to the introduction of CBC software in 1993 and the development of HB methods in the mid to late 1990s. The author states: "Much of the recent research and development in conjoint analysis has focused on doing more with less: stretching the research dollar using IT-based initiatives, reducing the number of questions required of any one respondent with more efficient design plans and HB (“data borrowing”) estimation, and reducing the complexity of conjoint questions using partial-profile designs." Since 2000, there has been increased interest in the use of optimization routines, greater realism (including "virtual shopping" environments) and real-time adaptive CBC routines.

[Download Paper]

This article conveys the basics of conjoint utility estimation using a common software tool: Microsoft's Excel. It covers dummy-coding and experimental design issues for full-profile conjoint analysis (single concept). When using Excel to perform the steps described in this article, you'll need Excel's Analysis Toolpak add-in with Regression Analysis.

[Download Paper]

This article provides an excellent case study and tutorial regarding how to bring sophisticated methods like conjoint analysis to an organization.

The author (Goodwin) discusses how conjoint has been adopted at Lifetime Products, Inc., including success stories and suggestions for obtaining buy-in from management. He outlines his history of progression in conjoint methods, from card-sort conjoint, to CBC, to part-profile CBC, and finally to adaptive CBC (ACBC).

[Download Paper]

It is advisable to include holdout choice tasks in conjoint interviews even though they may not appear to be needed for the main purpose of the study. This paper, originally published in Sawtooth Solutions, Spring 1997, and updated in 2010, lists the benefits of holdout choices and provides general guidance on how to construct them.

[Download Paper]

The Internet is fast becoming a viable way to conduct market research surveys for many situations. This article reports on a Full-Profile conjoint experiment conducted over the Internet. Single-Concept and Pairwise approaches are compared in terms of conjoint importances, utilities and reliability for predicting holdout choice concepts. Very little difference is found in any of the measures, suggesting that both approaches work well in practice for computerized Full-Profile conjoint.

[Download Paper]

Sample size considerations for conjoint analysis are often quite different from those for traditional market research surveys. This paper covers such topics as sampling error versus measurement error, confidence intervals, sampling for small populations, and how the choice of market simulation method affects the precision of results. The differences between ACA, traditional conjoint (CVA), and CBC are discussed with respect to sample size decisions. Finally, the paper reviews sample sizes commonly used by conjoint practitioners, and provides some rules-of-thumb and general recommendations.

[Download Paper]

Because conjoint utilities are often difficult for non-researchers to understand, researchers sometimes try to convert those to monetary equivalents. This practice is usually a poor use of conjoint analysis, and often misleading. The typical approaches ignore competitive factors and base the analysis on the average respondent. The author suggests that the worth of adding incremental features to products can be better determined through competitive market simulation scenarios.

[Download Paper]

This paper reports on an industry survey of 159 commercial research agencies in Europe regarding their use of conjoint analysis during the period July 1986 through June 1991. ACA is shown to be the most popular conjoint software in Europe. The authors (Wittink, Vriens, and Burhenne) provide an excellent discussion of general conjoint issues, and provide insight regarding the industry's adoption of this popular technique.

[Download Paper]

This 1987 article by Norein Klein is often cited in the literature. Abstract:

Some adaptive conjoint analysis methods reduce the attribute space by allowing the respondent to state which attribute levels are completely unacceptable. Utilities are not estimated for these levels, and it is assumed in later choice simulations that respondents would never choose alternatives that possess these levels. This procedure allows a more efficient estimation of conjoint utilities, but its value depends on whether the judgments of acceptability are consistent with respondents' behavior in later choices. In the study reported here, 15 percent of all choices contained an attribute level previously designated unacceptable, indicating some inconsistency between the judgments and choices. However, the overall accuracy of choice predictions was unaffected by the initial elimination of alternatives with unacceptable levels. The practical implications of these findings, and the relationship of judgments of acceptability to decision strategies are discussed.

[Download Paper]

Despite over 20 years of conjoint research and hundreds of methodological papers, very little has been published in the way of formal tests of whether conjoint really works in predicting real-world purchase decisions. The authors (Orme, Alpert and Christensen) argue that the holdouts typically used in conjoint validation studies are not very realistic, especially for high-involvement categories. The authors present results from a small pilot study. Respondents completed ACA, full-profile card sort, regular holdout choices and a "Super Holdout Task." The Super Holdout Task took 10 minutes and was an attempt to create a more realistic holdout. The authors compare results from the two types of holdout tasks and detect no significant differences. The authors find no significant difference in holdout hit rates for the Super Holdout Task for ACA and full-profile, though full-profile maintains a small edge. Full profile and CBC importances are shown to be steeper than ACA importances. The authors call for more realistic holdout tasks and urge those who have the resources to publish real-world validations of conjoint analysis. Originally presented at the 1997 Sawtooth Software Conference.

[Download Paper]

Joel Huber, Duke University, points out that respondents adopt different strategies for answering different types of conjoint questions. Researchers should understand these simplification strategies and match the right method to the context of actual marketplace decisions. Huber summarizes the strengths of the methods as follows:

• Self-explicated models are best in the case of many attributes, where expectations about levels and associations among attributes are stable. They work better in predicting decisions about independent alternatives than for competitive contexts.
• Paired comparisons are most appropriate for modeling markets in which alternatives are explicitly compared with one another, approximating a deeper search of a broad range of attributes, and where within-attribute value steps are smooth and approximately linear.
• Full-Profile works best when it is desirable to abstract from short run beliefs, when market choices reflect simplification toward the most important variables, and the decision focus is more within alternative rather than explicitly made using side-by-side comparisons between options.
• Choice is most appropriate for simulating immediate response to competitive offerings, when decisions are made based on relatively few attributes with substantial aversion to the worst levels of each attribute, and when consumers make decisions based on comparative differences among attributes.

In contrast to what is becoming popular agreement regarding the superiority of choices, Huber cautions that choices may not always work better than more traditional approaches. (Originally published in our 1997 Sawtooth Software Proceedings.)

[Download Paper]

Three ways to treat overall price in conjoint analysis experiments are discussed: traditional approach, conditional price, and continuous/summed price.

The traditional pricing method treats price as a separate attribute with a fixed set of price points that apply to all products. Prices are varied independently of the features. The problem with treating price in this traditional manner is that products with the best features are sometimes shown at the lowest prices (and products with the worst features are sometimes shown at the highest prices). This can lead to dominated choices and lack of realism.

With conditional pricing, incremental amounts are added to the price for premium brands or features, so enhanced products are generally shown at higher prices. One uses a look-up table to determine actual prices shown in the questionnaire. Currently, Sawtooth Software's CBC software allows price to be conditional on up to 3 other attributes.

Continuous/Summed pricing generalizes the idea of conditional prices (beyond the software limitations of just three attributes). Also, it estimates the effect of overall price as a linear coefficient, rather than as a part-worth utility function. After summing the prices across the feature components, price is varied by an additional random component specified by the researcher. One of the challenges of continuous pricing is that the price variable is moderately to strongly correlated with other attributes, depending on the design. A simulation study investigating the stability of the price coefficient within summed pricing is shown. The continuous pricing option is currently available as a standard option in Sawtooth Software's tools, though a power user could implement it with additional work. Continuous pricing will be a part of the forthcoming Adaptive CBC software.

[Download Paper]

Joel Huber of Duke University provides an insightful discussion on the history and theoretical underpinnings of conjoint analysis. He traces its development from its roots in psychometrics to its enthusiastic adoption by the market research community. The original paper is quite dated (originally published in our 1987 Sawtooth Software Conference Proceedings), but Joel Huber and Bryan Orme have added additional footnoted commentary from a 2004 perspective. This paper continues to be an excellent resource for today's conjoint practitioners.

[Download Paper]

The Number Of Levels effect is this: attributes with more levels in general tend to achieve higher importance than attributes defined on fewer levels. The authors (Wittink, Huber, Zandan and Johnson) present research which seeks to identify the cause. Behavioral versus algorithmic explanations are investigated. Their findings support the algorithmic hypothesis. The authors conclude that ACA is less susceptible to the number of levels effect than traditional full-profile conjoint methods in part due to the utility balance of the graded pairs.

[Download Paper]

In this article (originally published in Sawtooth News), Jon Pinnell and Pam Olsen of IntelliQuest present a case history of how Choice-Based Conjoint (CBC) was used successfully in pricing research conducted for a technology client. Derivation of demand curves and Price Elasticity of Demand is discussed. The Disk-By-Mail approach is also discussed as a viable way of collecting market research data.

[Download Paper]

This paper discusses a real-world case study using Choice-Based Conjoint. The Choice-Based Conjoint approach is shown to be a powerful method for uncovering brand x price interactions. The authors show that neglecting that important interaction would have been a bad assumption for their research study. This paper was originally presented at the ART Forum and is an excellent paper for researchers interested in Discrete Choice Modeling.

[Download Paper]

This paper discusses successful strategies for using CBC properly, and warns against common pitfalls. Topics include: using prohibitions, determining number of attribute levels to include, sample size, precision of estimates, counting analysis versus logit analysis, whether to include "None" in the questionnaire and in analysis, calibrating CBC results to market shares, and IIA and the red bus/blue bus problem.

[Download Paper]

Sawtooth Software's founder, Rich Johnson, describes the events that led to the development of the CBC software product. This document provides insights into key developments that have made CBC the most commonly used conjoint software product for conducting conjoint-related studies. An enjoyable read to develop an appreciation for the key people and main forces behind the creation of a classic.

[Download Paper]

"Choice-Based Conjoint" Analysis (CBC) is an integrated component within our SSI Web platform for conducting choice-based conjoint studies. The main characteristic distinguishing choice-based conjoint analysis from other types is that the respondent expresses preferences by choosing concepts from sets of concepts, rather than by rating or ranking them. This paper discusses the method of choice-based conjoint analysis from a practitioner-oriented point of view, and describes Sawtooth Software's CBC System for choice-based conjoint analysis in some detail. It also provides suggestions about how to select a particular conjoint method from the variety of those available, considering characteristics of the research problem at hand.

[Download Paper]

This paper demonstrates why recognizing differences between segments or respondents results in more predictive and valid choice simulations than simple aggregate models. Latent Class and ICE solutions are shown to better handle the traditional "Red bus/Blue bus" problem, cross-elasticities and interaction effects than the equivalent main-effects model using aggregate logit. Though cross-effects and IIA violations can be modeled in the aggregate, it requires modeling expertise and software other than CBC. The author concludes that it is beneficial to start with underlying utilities that are less susceptible to the "Red bus/Blue bus" problem.

[Download Paper]

This paper, by practitioner Jon Pinnell, was first delivered at the 2004 Sawtooth Software Conference, as a comment on Joel Huber's paper (also available for download within this library) entitled "Conjoint Analysis: How We Got Here and Where We Are--An Update." Jon gives many practical pieces of advice based on his years as a researcher using CBC analysis. His suggestions include: use choice-based rather than ratings-based tasks; use randomized rather than fixed designs; use more alternatives per task; use first-choices rather than allocations or full ranks; use HB; and to be cautious with partial-profile designs.

This article is a good review of best practice, with suggestions based on numerous methodological and commercial CBC studies.

[Download Paper]

CBC is a popular tool for studying brand and price effects for packaged goods and beverages. Under the proper conditions, it can produce quite accurate predictions of buyer behavior. The purpose of this document is to discuss some of the common approaches (and mistakes made) with past versions of our CBC software, and to point out some new capabilities available with the latest version of the Advanced Design Module for CBC/Web.

Common mistakes made were based on excessive use of level prohibitions within CBC software. Also, given the previous limitations of no more than 15 levels for brand and 16 concepts per choice task, it was difficult for researchers to represent the variety of unique brands available to buyers. The new version of CBC/Web Advanced Design Module offers up to 100 levels per attribute and 100 concepts per task, and supports a realistic "Shelf-Display" where the choice tasks look like shelves in a store, with products resting on the shelves.

[Download Paper]

The authors (Orme and Heft) provide evidence that, under proper conditions, conjoint analysis can accurately predict what buyers do in the real world. Their results are based on CBC interviews conducted in grocery stores, where the CBC results were used to predict actual sales for three product categories of packaged goods from those same stores with good success.

A second purpose of the paper is to show that capturing heterogeneity (reflecting differences in preference between groups or individuals) with Latent Class or ICE can improve predictions. Many complex effects (substitution, cross-effects and interactions) can be accounted for with disaggregate Main Effect models. The authors note that complex terms can be built into large aggregate logit models, but that such models risk overfitting. Moreover, that approach places a great deal of responsibility on the analyst to choose the right combination of terms. This paper was originally presented at the 1999 Sawtooth Software Conference.

[Download Paper]

In this article, the author (Orme) uses random split-sample experiments to test different ways of asking CBC and Adaptive CBC (ACBC) questionnaires. Specifically, he examines:

• Use of minimal overlap vs. modest overlap for CBC questionnaires (modest overlap seems to improve results)
• ACBC for small designs--just 4 attributes (ACBC is shown to work as well as CBC)
• Placing “Unacceptable” screening questions prior to “Must-Have” screening questions in ACBC (seems to work better)
• Individual-level (customized) utility constraints in ACBC (no benefit shown for this 4-attribute data set, though benefits should be greater for larger, more demanding designs)
• Whether giving respondents a “consistency game” will improve their data and their experience (minimal gains in fit observed, but with the risk of annoying about 1/5 of the respondents)

[Download Paper]

When planning a choice-based conjoint study, one must decide how many choice tasks to give each respondent. Too many may produce biased or noisy results, and too few will reduce precision. We re-analyzed data from 21 commercial studies, conducted in several countries and languages, with widely varying product categories, to see how results depend on the number of tasks respondents are given. [The paper was awarded "Best Presentation" at the ART Forum at Beaver Creek in June 1996.]

[Download Paper]

In this paper (originally published in our 1997 Sawtooth Software Conference Proceedings) Tom Pilon of TRAC, Inc. presents some additional types of analysis that can be done using standard CBC data. He reports results from a beer study, and shows how cross-elasticities for brands can be calculated (by regressing the log of choice volume on the log of price) and incorporated into a market simulator.

Pilon argues that the standard logit simulator which assumes constant cross-elasticity across brands was not entirely realistic for the beer market. A cross-elasticity simulator lets brands that compete closely (perceived as close substitutes) take relatively more share from one another as a result of price changes than from brands which are not perceived to be as substitutable. Pilon also demonstrates how to convert a cross-elasticity matrix into a "brand similarities matrix" for use in an MDS perceptual map. Brands which compete closely with one another are situated close to one another on the map.

[Download Paper]

This article is adapted from a presentation given at the 1998 A/R/T Forum by Joel Huber, Duke University. Huber compares three different methods for accounting for heterogeneity in CBC modeling: Hierarchical Bayes (HB), Latent Class, and ICE (Individual Choice Estimation). Three data sets are used to compare the merits of these approaches. HB and ICE are shown to outperform Latent Class in all aspects.

He concludes: "The important result is that although HB is more theoretically elegant than ICE, our experience suggests that both methods work equally well in practice." Latent Class is shown to provide estimates of aggregate shares nearly as accurate as HB and ICE, but "Latent Class, for its part, does a poor job of predicting individual choices unless its weights are allowed to be negative, as they are with ICE."

[Download Paper]

Huber, Wittink, Johnson and Miller report on a methodological study which featured an ACA interview between two short Choice-Based Conjoint modules. The study sought to discover if 1) the Choice-Based Conjoint utilities shifted when interrupted by ACA; 2) if the results from ACA were different from Choice-Based Conjoint; and 3) if ACA can be modified to approximate the Choice-Based results. They conclude that the answer to all three questions is yes. The authors suggest that ACA may be better at predicting how buyers would choose given sufficient information and time, whereas Choice-Based Conjoint results portray customers who are primarily motivated by brand name and price, and who have little time to make a decision. Originally published in 1992 Sawtooth Conference Proceedings.

[Download Paper]

In this paper, Rich Johnson and Bryan Orme from Sawtooth Software take an entirely new approach from their previous attempts at adaptive CBC. The new approach mimics the purchase process of formulating a consideration set using non-compensatory heuristics (such as “must have” or “must avoid” features), followed by a more careful tradeoff of alternatives within the consideration set using compensatory rules.

Adaptive CBC involves three core stages: 1) Build-Your-Own (BYO) Stage, 2) Screening Stage, and 3) Choice Tasks Stage. Rich and Bryan conducted a split-sample experiment comparing the new approach to traditional CBC. They found that respondents liked the adaptive survey more and felt it was more realistic--even though it took about double the time as traditional CBC. Furthermore, part-worths developed from ACBC were more predictive of holdout tasks than traditional CBC, despite the methods bias in favor of CBC for predicting the CBC-looking holdouts.

[Download Paper]

Rich Johnson, a key person in the development of conjoint analysis methods over the last 30 years, gives his perspective on Adaptive CBC (ACBC). What role does it play in the evolution of conjoint methods? What about the respondent burden? What about respondent engagement? What strengths does ACBC have over other approaches? This short essay contains pearls of wisdom.

[Download Paper]

"Adaptive Choice-Based Conjoint" (ACBC) is a new approach for adaptive choice-based conjoint studies. The interview has three main phases: 1) BYO (configuration) phase, 2) Consideration phase, 3) Choice phase. This follows the common proposition that buyers develop consideration sets and then choose a final product from within the consideration set. The interview adapts to each respondent, giving a more relevant, interactive experience. ACBC is recommended for experienced conjoint analysts and for projects involving about 5 or more attributes. Traditional brand-package-price studies should continue to be conducted under the standard (non-adaptive) CBC software.

[Download Paper]

This article is from a peer-reviewed journal article on Sawtooth Software’s ACBC (Adaptive CBC) approach. Indeed, most ACBC research has been published at the Sawtooth Software and joint SKIM/Sawtooth Software conferences, which aren’t subject to a formal academic review process. We’re pleased to point you to an article recently published in a peer-reviewed healthcare economics journal entitled “The Patient: Patient-Centered Outcomes Research”.

The article is authored by Charles E. Cunningham, Ken Deal, and Yvonne Chen.

The authors review previous ACBC studies involving consumer electronics, home purchases, PCs, automobiles, fast food, mobile telephone, chemical products, and consumer durables. They conclude:

"A review of randomized trials and cross-over studies suggests that, although ACBC surveys require more time than conventional approaches to CA, informants find ACBC surveys more engaging. In most studies, ACBC surveys yield lower standard errors, improved prediction of hold-out task choices, and better estimates of real-world product decisions than conventional choice-based CA surveys."

[Download Paper]

Bryan Orme and Rich Johnson report on another methodological test of their Adaptive CBC procedure (comparing it to standard non-adaptive CBC). This time, they study home purchases and employ shorter questionnaires, to see if similar results can be achieved with substantially less information. They use a slightly improved design algorithm as compared to their previous test reported in their 2007 article, "A New Approach to Adaptive CBC". Also, they test the use of a "most likelies" question rather than the standard BYO question.

Adaptive CBC again is shown to have greater internal predictive validity than standard CBC. Reducing the length of the questionnaires doesn't seem to reduce the predictive ability of Adaptive CBC for this test, though the authors caution that some information must have been lost. The "most likelies" option is shown to be a viable approach, though Orme and Johnson caution that the utilities are mis-informed by at least a small degree when using this approach.

[Download Paper]

In this article, the author (Orme) uses random split-sample experiments to test different ways of asking CBC and Adaptive CBC (ACBC) questionnaires. Specifically, he examines:

• Use of minimal overlap vs. modest overlap for CBC questionnaires (modest overlap seems to improve results)
• ACBC for small designs--just 4 attributes (ACBC is shown to work as well as CBC)
• Placing “Unacceptable” screening questions prior to “Must-Have” screening questions in ACBC (seems to work better)
• Individual-level (customized) utility constraints in ACBC (no benefit shown for this 4-attribute data set, though benefits should be greater for larger, more demanding designs)
• Whether giving respondents a “consistency game” will improve their data and their experience (minimal gains in fit observed, but with the risk of annoying about 1/5 of the respondents)

[Download Paper]

Validity studies that include data on actual sales are hard to obtain. In this article, Chris Chapman of Microsoft Corporation compares CBC and Adaptive CBC (ACBC) in terms of a number of measures, including ability to predict actual market shares for a consumer electronics device. ACBC produced slightly better estimates of market share. Also, Chapman cites evidence that ACBC may obtain more precise estimates than CBC, indicating need for lesser sample size (with reduction in fielding costs). Price sensitivity was also greater for ACBC than CBC.

Chapman conludes: "In short, the performance of ACBC for our CE product was similar to CBC and somewhat better in alignment with market data. We believe future research would be useful to determine whether this pattern of results (better prediction; higher price sensitivity; lower standard deviation) continues with other product categories."

[Download Paper]

This article discusses Menu-Based Choice (MBC), and how task position affects the results. Specifically, later tasks show greater price sensitivity than earlier tasks.

[Download Paper]

The author discusses practical concerns in analyzing MBC studies. He highlights the value of using randomized designs for MBC questionnaires, as they allow for straightforward counting analysis. Counting analysis helps the analyst decide which effects (own and cross-effects) are important to include in subsequent modeling. Three different approaches for building logit-based simulators are discussed: 1) Volumetric CBC Model, 2) Exhaustive Alternatives Model, and 3) Serial Cross-Effects Model. Both aggregate logit and HB models are estimated. All approaches worked quite well for the data set described here. The Volumetric CBC Model is the easiest to perform, but has weaknesses and is not a theoretically sound approach. The Exhaustive Alternatives model is more theoretically pleasing, but practically is limited to use with smaller menu problems that don't include very many options on the menu. The Serial Cross-Effects model strikes a good balance between flexibility and soundness.

[Download Paper]

The Market Simulator is usually considered the most important tool resulting from a conjoint project. The simulator is used to convert raw conjoint (part-worth utility) data into something much more managerially useful: simulated market choices. Products can be introduced within a simulated market scenario and the simulator reports the percent of respondents projected to choose each. A market simulator lets an analyst or manager conduct what-if games to investigate issues such as new product design, product positioning, and pricing strategy.

This paper covers the topic from an intuitive and strategic standpoint. It explains why interpreting average part worths or importances falls short, and the additional benefits of conducting appropriate simulations. Three common strategic questions that simulators can respond to are listed, and examples are provided using hypothetical data. The examples include new product introduction, repositioning existing products, price sensitivity measurement, and line extensions.

[Download Paper]

Conjoint simulators have been very useful for transforming part-worth utility values into the more concrete and managerially appealing shares of preference. Such simulators let the analyst play "What-If" games with real market scenarios, such as estimating the impact of pricing changes, product design modifications, or the effect of a line extension. However, traditional conjoint simulators based on the BTL or logit model have suffered from IIA problems. A common example is that of the red bus company that repaints half of its fleet blue and nearly doubles its predicted market share. Similar or identical products placed in IIA simulators tend to result in "share inflation." The first choice model, while not susceptible to IIA difficulties and unrealistic share inflation for similar offerings, typically produces shares of preference that are too extreme relative to real world behavior. Also, first choice models are inappropriate for use with logit or latent class models.

In the family of Sawtooth Software products, a Model 3 "Correction for Product Similarity" has been offered to deal with problems stemming from product similarity. However, this model is often too simplistic to accurately reflect real world behavior. The authors propose a new method called "Randomized First Choice (RFC)" for tuning market simulators to real world behavior. RFC adds random variation to both attribute part-worths and to the product utility, and simulates respondent choices under the first choice rule. RFC can be tuned to reflect any similar product substitution behavior between the extreme first choice rule and the IIA-grounded logit rule. RFC is shown to improve predictions of holdout choice tasks (reflecting severe differences in product similarity) for logit, latent class, ICE and hierarchical Bayes. The greatest gains were for the aggregate methods. The disaggregate methods, while less in need of corrections for product similarity, still benefit from RFC.

[Download Paper]

Market simulations from conjoint data often do not closely predict actual market shares. That is to be expected, as the model doesn't incorporate many real-world factors that critically affect market shares (such as distribution, awareness, time on the market, etc.). The authors argue that the best approach is to understand (and explain to others) the assumptions within the conjoint model, and to use the market simulator as-is-- focusing on its strengths, rather than making it do something it often cannot (predict market shares). Researchers over the years have (for better or worse) adjusted shares of preference to match known targets or market shares. The Sawtooth Software simulator offers an "external effect" correction to do this. However, it remains a "dangerous" practice, and the documentation warns against its use. The authors investigate how different methods for adjusting shares affect the fundamental properties of the market simulator, in terms of substitution effects, elasticities, and cross-elasticities. They find that the method used in the Sawtooth Software simulation tool has some undesirable properties. A method for adjusting part-worths at the individual level is also tested, and shown to perform better. Perhaps the most valuable section of this paper (and a very defensible adjustment) is the section dealing with corrections for distribution.

[Download Paper]

Randomized First Choice is a new market simulation technique that shows promise for reducing IIA problems, especially when using aggregate utilities. It combines the strengths of the first choice rule and the share of preference (logit) rule. Conjoint simulators have traditionally used part worths as point estimates of preference. Most recently, Hierarchical Bayes (HB) draws and Randomized First Choice (RFC) reflect uncertainty (error distributions) about part worths. RFC makes simplifying assumptions. HB draws, though theoretically more complete, have some unexpected properties. The authors (Orme and Baker) find that RFC with point estimates performs slightly better than using HB draws during simulations. Using RFC on point estimates avoids having to use the enormous HB draws files. The authors present two reasons why HB draws did not perform as well: a reverse number of levels effect, and an excluded levels effect. This paper was delivered at the 2000 Sawtooth Software Conference.

[Download Paper]

This paper compares four design strategies for choice-based experiments: catalog-based designs for full-profile experiments, recipe-based designs for partial profile experiments, computer optimized designs using SAS OPTEX software; and randomized designs using CBC software. The authors (Chrzan and Orme) compare these strategies in terms of design efficiency and their ability to capture particular effects (main, cross- and alternative-specific effects, and interactions).

CBC software is found to create optimal or near-optimal designs in all cases, with the exception of designs in which many more first-order interactions are modeled relative to main effects. SAS OPTEX software is shown to provide optimal or near-optimal designs in all cases for which it is applicable. For main-effect only designs, minimal level overlap strategies are favored. For higher-order effects, level overlap within tasks is desirable. A special case is demonstrated in which carefully chosen prohibitions between attribute levels can actually improve the efficiency of designs. D-efficiency computation using CBC and SPSS software is detailed in the appendix. This paper was voted "Most Valuable Presentation" at the 2000 Sawtooth Software Conference.

[Download Paper]

This paper (presented at our 1997 Conference) by Warren Kuhfeld of SAS Institute is a good overview of design principles for traditional conjoint. Kuhfeld introduces the concept of design efficiency and argues that orthogonality in conjoint experiments was generally required in days when computers were not widely available. If an orthogonal design was used, relatively simple formulas were available for hand or calculator ANOVA computations. Today, general linear models such as OLS do not require orthogonality for the unbiased estimation of effects.

Warren explains the principles of orthogonality and balance, introduces the measure of D-efficiency, and compares two computerized search routines for finding efficient experimental designs: SAS's PROC OPTEX procedure, and Sawtooth Software's CVA designer. For the size of designs commonly used in conjoint experiments, Warren found the CVA routine to find designs about 97% as efficient as OPTEX, but that CVA's designs tended to be more balanced. He also found CVA easier to use than OPTEX. Warren concluded: "For small problems like you would typically encounter in a full-profile conjoint study, CVA seems to do an excellent job. However, for larger and more difficult problems, it often fails to find more efficient designs that can be found with PROC OPTEX."

[Download Paper]

This article provides a case study regarding how MaxDiff and Cluster Ensemble analysis can be used to segment a population. Sawtooth Software conducted an online study among US respondents just prior to the 2008 presidential election between John McCain and Barack Obama.

The 2008 Political Landscape study measured what policy positions would make people most and least want to vote for a US presidential candidate. We studied 25 different policy positions, including such items as:

• Ensure the long-term health of Social Security
• Bring the troops home from Iraq
• Restrict carbon emissions to reduce global warming
• Reduce the federal deficit

We used MaxDiff (best-worst) scaling to measure the influence of these items on respondents’ preference for a presidential candidate. And, of course, democrats and republicans have different priorities when it comes to policies and positions.

Following estimation of individual-level scores from MaxDiff, we used our CCEA (Convergent Cluster & Ensemble Analysis) software to find segments of respondents that had similar preferences.

[Download Paper]

"Convergent Cluster & Ensemble Analysis" (CCEA) is software for doing cluster and cluster ensemble analysis. CCEA uses k-means as its standard cluster algorithm. However, the newer Ensemble Analysis included in the software is shown to produce better results for artificial datasets generated with known group membership. The procedure for the ensemble analysis is described in detail. Comparisons to k-means as provided by our previous CCA software are shown.

[Download Paper]

Cluster Ensemble Analysis is a relatively new technique to market researchers. The process involves combining existing cluster or segmentation solutions (an ensemble of candidate segmentation solutions) to produce a consensus solution that has stronger characteristics than any of the separate candidate solutions.

Retzer et al. from Maritz test different approaches to developing consensus solutions: Direct Approach, Feature Based Approach, Pair-Wise Approach, and the Sawtooth Software method as implemented in CCEA software. The authors conclude: "It's clear from the results that, for these data, the Direct and Sawtooth approaches outperform all others." They also demonstrate a graphical method to depict ensemble diversity. It's important that the candidate solutions in the ensemble be of good quality, and it is also important that they be diverse. The authors comment that the CCEA software "proved quite capable of handling the difficult and critical task of generating a diverse ensemble."

[Download Paper]

Maximum Difference (MaxDiff, or best/worst) scaling is a relatively new technique for measuring the importance or preference of multiple items. In MaxDiff tasks, respondents see sets of items (typically 4 to 6). In each set, respondents indicate which item is most important (preferred) and least important (preferred). Steve Cohen describes the methodology and presents results for a methodological study comparing MaxDiff measurement with monadic ratings and paired comparisons, and also a case study focusing on using MaxDiff for segmentation work. MaxDiff is shown to provide results that have greater between-item and between-respondent discrimination, and greater predictive accuracy than either monadic ratings or paired comparisons. Steve won the "best presentation" award with this paper at the 2003 Sawtooth Software Conference.

[Download Paper]

This paper describes the technical procedures used in the MaxDiff/Web System. MaxDiff (best-worst) scaling is a trade-off method for measuring the importance or preference for multiple items, such as brands, product features, political platforms, advertising claims, etc. Any time you are considering using a rating scale, ranking scale, or constant sum scale for multiple items, you can consider using MaxDiff.

The MaxDiff methodology, originally invented by researcher and academic Jordan Louviere, has gained in popularity over the last five years. Papers on MaxDiff have won "best presentation" awards at recent ESOMAR and Sawtooth Software research conferences. It has many similarities to, but is distinctively different, from conjoint methodology and is appropriate for a wider range of research opportunities.

Sawtooth Software’s MaxDiff/Web system may be used for conducting web-based, CAPI, or paper-based MaxDiff studies. The software also supports asking the "best" half of the question only (not requiring respondents to identify the "worst" item in each set). The software may also be used for Method of Paired Comparisons research. Individual-level estimation of item scores employs Sawtooth Software’s popular hierarchical Bayes (HB) engine. Results may also be exported to Sawtooth Software’s Latent Class system for segmentation analysis.

[Download Paper]

This article offers a case study demonstrating how best/worst scaling may be used for estimating the price sensitivity of automobile buyers to different car options, such as warranty, anti-lock brakes, and keyless entry. Using best/worst scaling, the author (Keith Chrzan), shows how price sensitivity curves may be developed for each car option, presented on a common scale. Chrzan contrasts the best/worst approach with conjoint analysis, and explains the benefits of using best/worst for this particular application rather than conjoint analysis. The results validate closely to self-reported past purchase behavior for options on the most recently purchased car.

Chrzan provides some background on best/worst, and describes how Sawtooth Software's latent class and HB software may be used during analysis. This paper was voted "best presentation" at the 2004 Sawtooth Software Conference.

[Download Paper]

The author (Orme) presents results from two studies testing a new procedure called Adaptive MaxDiff Scaling. Rather than focus equal attention on estimating respondents' preferences (or importances) for best AND worst items, A-MaxDiff focuses attention on estimating best/most important items with greater precision. The interview adapts to each respondent, learning from prior responses. Items marked "worst" are discarded from further consideration. The questionnaire proceeds in stages. In the first stage, K items are shown per set. In each subsequent stage, K-1 items are shown per set, until the respondent is doing paired comparisons among the surviving (most preferred) items. Later tasks reflect increased utility balance.

The results show better hit rates for "best" items in holdouts relative to standard MaxDiff. Average population parameters are essentially identical between standard and adaptive forms of MaxDiff. Respondents take slightly less time to complete the adaptive survey, and they perceive it to be more enjoyable and less monotonous than standard MaxDiff. Orme argues that A-MaxDiff should be especially preferred when simulation methods such as TURF are used with MaxDiff data. The main drawback is decreased precision of estimates for "worst" items.

[Download Paper]

The authors investigate how the number of items per MaxDiff set affects dropout rates, survey length, positional bias, parameter equivalence, and predictive validity. Three commercial studies are analyzed, where the number of items per set varied from 3 items/set to 8 items/set. The number of items/set has the most influence on task length, with respondents taking significantly longer to complete 8 items/set rather than 3 items/set. Statistically significant differences among the parameters were found, but the authors note that the overall results would lead to similar managerial decisions. The predictive validity tests "hint that 3 items per question may produce slightly worse predictions than questions with more items." They conclude: "Given the slight evidence of poorer hit rates and poorer out-of-sample for 3 items per question we recommend using 4 or 5 items per question in maxdiff experiments."

[Download Paper]

Traditional MaxDiff analysis leads to relative importance/preference scores. But, there is no possible way for respondents to express that (for example) all the items are important or none of the items are important. Some researchers have worried that the relative nature of the MaxDiff judgments and resulting scale means that meaningful differences between respondents or segments of respondents are lost.

This article describes a practical way, proposed by Jordan Louviere (inventor of MaxDiff), to anchor the scale for each respondent based on an important/not important threshold. A dual-response questioning device is straightforward to include in MaxDiff questionnaires, and in analysis. The author (Orme) provides empirical evidence that the dual-response approach leads to meaningful discrimination among respondents and items, beyond the information provided by the standard MaxDiff tradeoffs. The pros and cons of the approach are discussed.

[Download Paper]

Maximum Difference Scaling is widely used to measure the relative values of items/attributes. Despite the strengths of MaxDiff, some analysts would prefer data that represented more than just relative scores; they would prefer absolute scores scaled with respect to each respondent’s importance threshold. In this article by Kevin Lattery (Maritz Research), Kevin tested two methods for anchoring MaxDiff scores to a threshold: Dual-Response MaxDiff suggested by Louviere and a more direct method asking respondents to choose which attributes are above a threshold (using 2-point scale grid questions).

He determined that theoretically (using synthetic respondent data) the direct method would be superior, especially as the number of attributes shown in a MaxDiff task increases. With six or more attributes shown per screen the indirect dual-response method should not be used, and even five attributes per screen may not capture individual anchoring that well. In comparing the two methods with human respondents, showing only four attributes per screen, results were very similar. The rank order of utilities at the respondent level was nearly identical. However, the anchoring in the direct method was more biased by the context of the total set of attributes. So if it is important for one to have a more neutral anchor for utilities then the indirect dual-response method may be slightly better, assuming four (and certainly no more than five) attributes are shown per screen.

(Originally published in the 2010 Sawtooth Software Proceedings).

[Download Paper]

This document describes a software package from Sawtooth Software for designing MaxDiff (best/worst) experiments. It also describes the dummy coding procedure for estimating effects from MaxDiff data, as may be applied within Sawtooth Software’s Latent Class or HB software systems.

[Download Paper]

This paper communicates results of a Monte Carlo simulation study on how the precision of estimates for MaxDiff (best/worst) experiments is affected by:

• Number of items presented per set,
• Number of sets presented to each respondent,
• Number of items in the overall study.

Results show that it may not be useful to ask more than about 5 items per set. The data also suggest that displaying each item 3 or more times per respondent works well for obtaining reasonably precise individual-level estimates with HB. Asking more tasks, such that the number of exposures per item is increased well beyond 3, seems to offer significant benefit, provided respondents don't become fatigued and provide data of reduced quality..

[Download Paper]

This paper compares different methods of obtaining individual-level scores for MaxDiff surveys at the individual level: Simple counting, individual-level logit, and HB. Key to the success for all these methods was having enough information available for each respondent to estimate stable scores.

The author (Orme) finds that counting analysis provides reasonable population estimates of scores, but that the individual-level scores can lack precision. Precision is better under the logit model estimation methods: either individual-level logit or HB, which "borrows" information across the sample to improve the individual-level logit scores for individuals.

Despite the simplicity of the counting approach and its weaknesses, it tends to do quite well in predicting responses to holdout choices. But, across-respondent variance (heterogeneity) tends to be weaker than the other methods studied.

[Download Paper]

This paper was originally published in the March 2000 Quirk's Marketing Research Review. HB has been receiving a lot of attention lately. Until recently, desktop PCs weren't powerful enough to handle typical data sets and commercial software wasn't available. Now HB is accessible to mainstream market researchers. HB is receiving so much attention because it consistently matches or beats traditional OLS estimation for individual-level parameters, and can estimate individual-level models for choice-based conjoint (CBC) data. Traditional aggregation methods confound heterogeneity with noise. By modeling the heterogeneity in the data, HB can achieve more precise estimates. This usually leads to more accurate models, whether the researcher is interested in aggregate or individual-level predictions. This paper gives examples of how HB can be applied to traditional regression-based problems (like customer satisfaction data sets), ACA data or choice-based conjoint (CBC). It explains why HB is beneficial for each of those applications.

[Download Paper]

This paper was originally created for the 2009 Sawtooth Software Conference. This paper focuses on what happens during the estimation of CBC/HB utilities. It takes a naïve approach assuming no knowledge of statistics or math beyond basic algebra. It begins by providing a brief overview about the issues surrounding individual level estimation and why CBC/HB receives the attention it does. The paper then goes into more detail about what the algorithm is intuitively doing and what each step is trying to accomplish.

The paper is non-technical in nature and is written for the absolute beginner. It is intended to be at a level that can be given to a beginning junior analyst or a non-mathematically inclined client.

[Download Paper]

In this paper, Rich Johnson provides an intuitive example to explain Bayesian analysis. He explains how Bayesian analysis differs from conventional statistics. Rich introduces Bayes' rule, and talks about how HB can be applied to conjoint analysis and other marketing research problems where respondents provide multiple observations. Rich reports findings demonstrating that HB generally performs better than traditional methods for conjoint and regression problems.

[Download Paper]

Hierarchical Bayes is a relatively new technique for computing individual- level part worths from CBC data. HB has been described favorably in numerous journal articles. Its strongest point of differentiation is its ability to provide estimates of individual part worths given only a few choices by each individual. It does this by "borrowing" information from other individuals.

This technical paper describes the intuition and math behind HB, including results that suggest that HB is generally superior relative to aggregate approaches for estimating individual's choices and aggregate share predictions. We at Sawtooth Software are not experts in Bayesian data analysis. In producing this software we have been helped by several sources. We have benefited particularly from the materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing Association's Advanced Research Techniques Forum.

[Download Paper]

Hierarchical Bayes estimation for choice data represents one of the most successful new developments in our field. HB has proven robust for ratings-based conjoint, ACA, and full-profile CBC projects. Tests comparing HB to other methods of part worth estimation have generally favored HB. However, two anomalies specific to HB estimation have caused us some puzzlement and concern.

• The "omitted" level in effects coding for conjoint analysis results in overstatement of the variance, and in extreme cases (very sparse data and very many levels within an attribute) biased point estimates.
• HB was demonstrated to have problems in individual-level estimation for some partial-profile CBC data sets.

This paper shows that the problems above can be controlled or even solved by setting proper "priors" in HB. The anomalies therefore do not point to a weakness in HB methods, but simply illustrate that we were not defining the models properly in certain circumstances. HB researchers should be aware of the kinds of data sets that can challenge "generic" HB estimation under Sawtooth Software's default settings, and learn to manage these through more proper specification of the priors.

[Download Paper]

The authors (Keith Sentis and Lihua Li) investigated whether the assumption in HB of a single multivariate normal distribution to reflect the population negatively affected the estimated utilities if segments existed with quite different utilities. The authors studied seven actual CBC data sets, systematically excluding some of the tasks to serve as holdouts for internal validation. Keith and Lihua found that whether one ran HB on the entire sample, or whether one segmented first (K-means, demographic, or Latent Class) prior to estimating utilities, the upper-level model assumption in HB of normality did not decrease the fit of the estimated utilities to the holdouts. It seemed unnecessary to segment first before running HB.

[Download Paper]

This paper was originally delivered at the 1999 Hierarchical Bayes Conference at Ohio State University by Rich Johnson. Rich recaps his experience with HB methods, particularly as they relate to conjoint analysis. He points out that HB usually either matches or beats traditional estimation of part worth utilities. He also illustrates some interesting artifacts regarding HB estimation dealing with distortions in the variances of estimates.

[Download Paper]

The basic (generic) hierarchical Bayes estimation that the first versions of Sawtooth Software's CBC/HB program supported assumed that respondents were drawn from a single, multivariate-normal distribution. All respondents were "shrunk" to some degree or another toward the population means. This article describes relaxing the assumption of a single normal population via covariates included in the upper-level model of the hierarchy. Covariates are segmentation variables that are predictive of respondent's choices. When used in HB, they allow shrinkage to respondent-specific locations in the distribution, depending on the characteristics of the covariates. The results can lead to modest improvement in predictive accuracy, but substantial improvement in terms of differences between segments of respondents on the means. This is of substantial benefit to segmentation research, where the generic model could obscure the true differences between respondents.

[Download Paper]

Conjoint analysts often discover that some of the part worths don't conform to expectations. We generally expect low prices to be preferred to high prices, high performance to low performance, etc. But, with individual-level CBC part worths, we often notice violations (reversals) of rational ordering. The author (Rich Johnson) investigates six alternative approaches to constraining part worths for two commercial data sets within the context of HB estimation. Two of the best methods presented in this paper are implemented in CBC/HB v1.5 software. Rich concludes: "If the primary purpose of the study is to predict individual choices, then it appears desirable to enforce monotonicity constraints. On the other hand, if the primary purpose of the study is to predict aggregate measures such as market shares, monotonicity constraints appear less helpful, and may occasionally even be harmful."

[Download Paper]

Greg Allenby and Peter Rossi describe the history of HB methods as they relate to marketing research methods. They describe the theory behind HB, the challenges in implementing HB methods for marketing research in the 1990s, and what they see as the future applications of HB within marketing research. They predict that over the next 10 years, HB will enable researchers to develop more rich models of consumer behavior. We will extend the standard preference models to incorporate more complex behavioral components, including screening rules in conjoint analysis (conjunctive, disjunctive, compensatory), satiation, scale usage, and inter-dependent preferences among consumers. New models will approach preference from the multitude of basic concerns and interests that give rise to needs. Common to all these problems is a dramatic increase in the number of explanatory variables.

[Download Paper]

TURF stands for "Total Unduplicated Reach and Frequency" and is an optimization search routine for finding optimal sets of items (brands, claims, attributes) that motivate (reach) respondents. This introductory paper describes how TURF works within our software offering TURF optimization called MaxDiff Analyzer. Many different types of data may be used with our TURF routine including Likert scales, consideration (pick any), and MaxDiff.

[Download Paper]

Traditional TURF optimization uses an exhaustive search approach, examining all possible portfolios (sets of items) of a specific size en route to discovering the optimal portfolio(s). However, problems can quickly become too large for exhaustive search to compute within reasonable time frames. A shortcut, heuristic approach is described called Stepwise TURF + Swaps that finds all 300 top portfolios in three very large commercial data sets. The heuristic searches take at most a few minutes, whereas the exhaustive search would take many days or years to compute.

[Download Paper]

In this paper, Rich Johnson, Sawtooth Software's founder, recounts the history of Adaptive Conjoint Analysis (ACA). He traces its development and theoretical roots from the early trade-off matrices through today's current software solution. This paper, originally presented at the 2001 Sawtooth Software Conference, is an excellent primer on the theory and mechanics of Adaptive Conjoint Analysis.

[Download Paper]

Though we've been told that ACA is remarkably easy to use, our users run into problems from time to time. 18 of the most common pitfalls are discussed. Most of the pitfalls listed focus on design and interpretation issues. A must read for ACA users, originally published in Sawtooth Solutions, Spring 1996.

[Download Paper]

"Adaptive Conjoint Analysis" (ACA) is software for conjoint (trade-off) analysis. The term "adaptive" refers to the fact that the computer-administered interview is customized for each respondent. Data are analyzed as the interview progresses, and we choose questions likely to reveal the most about the respondent's values in the shortest time. ACA is an excellent alternative to full-profile conjoint when you have a large number of attributes. This paper provides a description of the adaptive technique, including technical details.

[Download Paper]

The author responds to recent criticism of conjoint methods and compares ACA with self-explicated methods. The author provides his perspective on using ACA, some shortcomings of the method, and recent developments that have eliminated or reduced those shortcomings. The benefits of using HB estimation for modeling ACA part worths are demonstrated using an actual ACA project.

[Download Paper]

Results from a Monte Carlo simulation are reported showing that ACA's estimation technique is unbiased. The accuracy of utility estimation under various questionnaire conditions (e.g. 2 attributes vs. 5 attributes in the pairs section) is shown. The research concludes that ACA is accurate at estimating respondents' utilities, provided they respond consistently.

[Download Paper]

As market researchers, we strive to create conjoint models that are predictive of actual purchase behavior. But it is difficult to both design good validation studies and convince the owners of the data to let one publicly share the results. This paper by James J. Tumbusch of MarketVision Research, Inc. (originally published in the 1991 Sawtooth Software Conference Proceedings) presents a compelling real-world validation of conjoint and ACA. Conjoint predictions are compared with traditional concept testing for four different frequently purchased product categories. High correlations were found--even though the conjoint and concept test samples were independent, and the conjoint studies preceded the concept test data collection by up to 12 months.

[Download Paper]

Although ACA has proven to be a useful and popular technique over two decades, many researchers have argued that the self-explicated importance question in the ACA "Priors" may be a weak link. The self-explicated importances can be confusing to some respondents, and the metric information provided by the responses may flatten the final derived importances. In this article, the authors report on a split-sample methodological study that tests an altered version of ACA (that skips the Importance questions) against the original ACA. The modified ACA is shown to produce directionally better share predictions of holdout choice tasks than the original ACA, with a lower overall interviewing time. Also, there is evidence that the information provided by the self-explicated importances is not only flatter than that contributed by the conjoint Pairs section, but the information is different.

[Download Paper]

The Number-of-Levels (NOL) Effect is troublesome in conjoint research. One can increase the apparent importance of an attribute by adding more levels. Past research has shown that ACA (Equal Weighting) is less susceptible to the NOL effect than traditional conjoint and CBC. There is yet another way to reduce the NOL effect even further with ACA by using the "Optimal Weighting" feature available in ACA Version 4. This paper demonstrates the difference in attribute importances between the Equal Weighting and Optimal Weighting options, and quantifies the reduction in the effect for two data sets.

The author concludes: "The optimal weighting method appears to have been a nice addition to ACA. It probably deserves more credit than it has been given. We . . . have believed for some time that optimal weighting provided modest improvements to ACA utilities . . . We now recognize that (it) plays a significant role in reducing the NOL effect, and recommend that ACA users use optimal weighting, especially when the number of levels varies across attributes in the design."

[Download Paper]

Huber, Wittink, Fiedler and Miller present a methodological study comparing ACA with traditional full profile conjoint. 300 respondents completed both an ACA and a full profile task administered one concept at a time by computer about preferences for refrigerators. The order of the two conjoint tasks was rotated to control for order effects. The authors compare how well results from the two methods predict hold-out concepts. They conclude: "For both five . . . and nine attributes we find that ACA outperforms the full profile method. We have broken down the overall performance results in a number of ways, and observe that ACA maintains an edge, compared to computer-administered full-profile."

[Download Paper]

Even though ACA is one of the most popular conjoint analysis techniques, it has been shown often to understate the importance of price. This article by Peter Williams demonstrates a method of adjusting the price utilities within ACA studies using holdout choice tasks. The holdout tasks typically include a stripped down product at a cheap price, a mid-range product at a medium price, and a feature-rich product at a premium price.

Williams suggests counting the number of times respondents choose higher-priced alternatives (this requires multiple choice tasks), and segmenting respondents based on that scored variable. Then, he suggests developing a separate weighting factor for the ACA price utilities for each segment. He demonstrated the technique with an ACA study involving 969 respondents. He found that no adjustment was necessary for the group that preferred high-priced, high-quality offers; a scaling factor of 2 for price was required for the mid-price segment; and a scaling factor of slightly more than 4 was required for the price sensitive segment.

[Download Paper]

Some researchers have discovered that they can obtain better overall results by combining ACA with a second conjoint module. The author, Jon Pinnell of IntelliQuest, refers to this as "Multistage Conjoint" or "Dual Conjoint" and describes a procedure for combining ACA with a full-profile conjoint or choice-based conjoint exercise. ACA is used to estimate utilities for a wide range of features, while the second conjoint focuses on a limited set of features and issues such as price sensitivity. The author reports that ACA studies which include price as just one of many attributes (more than about 10) tend to assign too little weight to price. Pinnell suggests ways to increase the price weight by bridging price information from the second conjoint module.

[Download Paper]

The 4-point importance scale used in the self-explicated priors in ACA has been a focus of some debate. Specifically, it has been suggested that the 4-point scale is too coarse. William McLauchlan studied the effects of using different scales in ACA, and presented the results via this paper at the 1991 Sawtooth Software Conference. In cooperation with Sawtooth Software, three versions of ACA were created and tested: 1) ACA Version 3.0, 2) ACA Version 3.0 with a 9-point importance scale, 3) Analog ACA, which elicited prior utilities using a finely-graded analog scale. McLauchlan found no significant differences in predictability of hold-out concepts among the three versions of ACA. However, the analog version of ACA took significantly more time for respondents to complete.

[Download Paper]

Many studies have indicated that the predictive validity of partworth estimates in conjoint analysis can be improved by imposing constraints on utility estimates. In this paper, two types of constraints are investigated: within- and across-attribute. Prior information can be imposed (i.e. a priori order for quantitative variables such as Price) or collected during the interview (i.e. self-explicated rankings and importance ratings). The authors (van der Lans, Wittink, Huber and Vriens) investigate the effect of imposing constraints on conjoint utilities for full-profile and ACA. Hit rates for hold-out concepts are shown to increase significantly for full-profile when utility constraints are imposed. ACA hit rates on average are higher than full-profile, and are not significantly improved by imposing constraints. Originally published in 1992 Sawtooth Software Conference Proceedings.