Technical Papers  Pricing & Ordering  Your Cart

Sawtooth Software picks

These represent our favorite articles, along with some papers that deserve a look even though they didn't make it onto the other suggested readings lists.

• A Short History of Conjoint Analysis (2009)
  
  [Download Paper]     

  Conjoint analysis has been a great success story for the marketing research industry. This paper outlines its development from the late 1960s through the early 2000s. The earliest conjoint analysis approaches were based on either full-profile card sort, or Johnson's tradeoff matrix. Later, Adaptive Conjoint Analysis and discrete choice (CBC) applications dominated. The use of CBC accelerated in the 1990s due largely to the introduction of CBC software in 1993 and the development of HB methods in the mid to late 1990s. The author states: "Much of the recent research and development in conjoint analysis has focused on doing more with less: stretching the research dollar using IT-based initiatives, reducing the number of questions required of any one respondent with more efficient design plans and HB (“data borrowing”) estimation, and reducing the complexity of conjoint questions using partial-profile designs." Since 2000, there has been increased interest in the use of optimization routines, greater realism (including "virtual shopping" environments) and real-time adaptive CBC routines.
• History of ACA (2001)
  
  [Download Paper]     

  In this paper, Rich Johnson, Sawtooth Software's founder, recounts the history of Adaptive Conjoint Analysis (ACA). He traces its development and theoretical roots from the early trade-off matrices through today's current software solution. This paper, originally presented at the 2001 Sawtooth Software Conference, is an excellent primer on the theory and mechanics of Adaptive Conjoint Analysis.
• History of Sawtooth Software's CBC Program (2011)
  
  [Download Paper]

  Sawtooth Software's founder, Rich Johnson, describes the events that led to the development of the CBC software product. This document provides insights into key developments that have made CBC the most commonly used conjoint software product for conducting conjoint-related studies. An enjoyable read to develop an appreciation for the key people and main forces behind the creation of a classic.
• Introduction of Quantitative Marketing Research Solutions in a Traditional Manufacturing Firm: Practical Experiences (2009)
  
  [Download Paper]

  This article provides an excellent case study and tutorial regarding how to bring sophisticated methods like conjoint analysis to an organization.

  The author (Goodwin) discusses how conjoint has been adopted at Lifetime Products, Inc., including success stories and suggestions for obtaining buy-in from management. He outlines his history of progression in conjoint methods, from card-sort conjoint, to CBC, to part-profile CBC, and finally to adaptive CBC (ACBC).

• CBC vs. ACBC: Comparing Results with Real Product Selection (2009)
  
  [Download Paper]

  Validity studies that include data on actual sales are hard to obtain. In this article, Chris Chapman of Microsoft Corporation compares CBC and Adaptive CBC (ACBC) in terms of a number of measures, including ability to predict actual market shares for a consumer electronics device. ACBC produced slightly better estimates of market share. Also, Chapman cites evidence that ACBC may obtain more precise estimates than CBC, indicating need for lesser sample size (with reduction in fielding costs). Price sensitivity was also greater for ACBC than CBC.

  Chapman conludes: "In short, the performance of ACBC for our CE product was similar to CBC and somewhat better in alignment with market data. We believe future research would be useful to determine whether this pattern of results (better prediction; higher price sensitivity; lower standard deviation) continues with other product categories."

• Assessing the Monetary Value of Attribute Levels with Conjoint Analysis:
      Warnings and Suggestions
  
  [Download Paper]     

  Because conjoint utilities are often difficult for non-researchers to understand, researchers sometimes try to convert those to monetary equivalents. This practice is usually a poor use of conjoint analysis, and often misleading. The typical approaches ignore competitive factors and base the analysis on the average respondent. The author suggests that the worth of adding incremental features to products can be better determined through competitive market simulation scenarios.
• MaxDiff/Web Technical Paper (2007)
  
  [Download Paper]     

  This paper describes the technical procedures used in the MaxDiff/Web System. MaxDiff (best-worst) scaling is a trade-off method for measuring the importance or preference for multiple items, such as brands, product features, political platforms, advertising claims, etc. Any time you are considering using a rating scale, ranking scale, or constant sum scale for multiple items, you can consider using MaxDiff.

  The MaxDiff methodology, originally invented by researcher and academic Jordan Louviere, has gained in popularity over the last five years. Papers on MaxDiff have won "best presentation" awards at recent ESOMAR and Sawtooth Software research conferences. It has many similarities to, but is distinctively different, from conjoint methodology and is appropriate for a wider range of research opportunities.

  Sawtooth Software’s MaxDiff/Web system may be used for conducting web-based, CAPI, or paper-based MaxDiff studies. The software also supports asking the "best" half of the question only (not requiring respondents to identify the "worst" item in each set). The software may also be used for Method of Paired Comparisons research. Individual-level estimation of item scores employs Sawtooth Software’s popular hierarchical Bayes (HB) engine. Results may also be exported to Sawtooth Software’s Latent Class system for segmentation analysis.

• Accuracy of HB Estimation in MaxDiff Experiments (2005)
  
  [Download Paper]     

  This paper communicates results of a Monte Carlo simulation study on how the precision of estimates for MaxDiff (best/worst) experiments is affected by:

  • Number of items presented per set,
  • Number of sets presented to each respondent,
  • Number of items in the overall study.

  Results show that it may not be useful to ask more than about 5 items per set. The data also suggest that displaying each item 3 or more times per respondent works well for obtaining reasonably precise individual-level estimates with HB. Asking more tasks, such that the number of exposures per item is increased well beyond 3, seems to offer significant benefit, provided respondents don't become fatigued and provide data of reduced quality..
• How Many Questions Should You Ask in Choice-Based Conjoint Studies? (1996)
  
  [Download Paper]     

  When planning a choice-based conjoint study, one must decide how many choice tasks to give each respondent. Too many may produce biased or noisy results, and too few will reduce precision. We re-analyzed data from 21 commercial studies, conducted in several countries and languages, with widely varying product categories, to see how results depend on the number of tasks respondents are given. [The paper was awarded "Best Presentation" at the ART Forum at Beaver Creek in June 1996.]
• The Benefits of Accounting for Respondent Heterogeneity in Choice Modeling (1998)
  
  [Download Paper]     

  This paper demonstrates why recognizing differences between segments or respondents results in more predictive and valid choice simulations than simple aggregate models. Lclass and ICE solutions are shown to better handle the traditional "Red bus/Blue bus" problem, cross-elasticities and interaction effects than the equivalent main-effects model using aggregate logit. Though cross-effects and IIA violations can be modeled in the aggregate, it requires modeling expertise and software other than CBC. The author concludes that it is beneficial to start with underlying utilities that are less susceptible to the "Red bus/Blue bus" problem.
• Dealing with Product Similarity in Conjoint Simulations (1999)
  
  [Download Paper]     

  Conjoint simulators have been very useful for transforming part-worth utility values into the more concrete and managerially appealing shares of preference. Such simulators let the analyst play "What-If" games with real market scenarios, such as estimating the impact of pricing changes, product design modifications, or the effect of a line extension. However, traditional conjoint simulators based on the BTL or logit model have suffered from IIA problems. A common example is that of the red bus company that repaints half of its fleet blue and nearly doubles its predicted market share. Similar or identical products placed in IIA simulators tend to result in "share inflation." The first choice model, while not susceptible to IIA difficulties and unrealistic share inflation for similar offerings, typically produces shares of preference that are too extreme relative to real world behavior. Also, first choice models are inappropriate for use with logit or latent class models.

  In the family of Sawtooth Software products, a Model 3 "Correction for Product Similarity" has been offered to deal with problems stemming from product similarity. However, this model is often too simplistic to accurately reflect real world behavior. The authors propose a new method called "Randomized First Choice (RFC)" for tuning market simulators to real world behavior. RFC adds random variation to both attribute part-worths and to the product utility, and simulates respondent choices under the first choice rule. RFC can be tuned to reflect any similar product substitution behavior between the extreme first choice rule and the IIA-grounded logit rule. RFC is shown to improve predictions of holdout choice tasks (reflecting severe differences in product similarity) for logit, latent class, ICE and hierarchical Bayes. The greatest gains were for the aggregate methods. The disaggregate methods, while less in need of corrections for product similarity, still benefit from RFC.

• An Overview and Comparison of Design Strategies for Choice-Based Conjoint (2000)
      Analysis
  
  [Download Paper]     

  This paper compares four design strategies for choice-based experiments: catalog-based designs for full-profile experiments, recipe-based designs for partial profile experiments, computer optimized designs using SAS OPTEX software; and randomized designs using CBC software. The authors (Chrzan and Orme) compare these strategies in terms of design efficiency and their ability to capture particular effects (main, cross- and alternative-specific effects, and interactions).

  CBC software is found to create optimal or near-optimal designs in all cases, with the exception of designs in which many more first-order interactions are modeled relative to main effects. SAS OPTEX software is shown to provide optimal or near-optimal designs in all cases for which it is applicable. For main-effect only designs, minimal level overlap strategies are favored. For higher-order effects, level overlap within tasks is desirable. A special case is demonstrated in which carefully chosen prohibitions between attribute levels can actually improve the efficiency of designs. D-efficiency computation using CBC and SPSS software is detailed in the appendix. This paper was voted "Most Valuable Presentation" at the 2000 Sawtooth Software Conference.

© 2012 Sawtooth Software, Inc. All rights reserved.