Optional Workshop and Tutorials
Optional 1.5-Day Workshop
Monday 8 am - 5 pm & Tuesday 8 am - Noon	CBC WORKSHOP Aaron Hill and Brian McEwan, Sawtooth Software
	The CBC Workshop is designed to introduce participants to discrete choice (CBC) analysis through an interactive, hands-on workshop. The course will include an overview of conjoint analysis techniques and methodology, with specific attention to CBC/Web. Attendees will receive practical experience creating surveys in Sawtooth Software’s SSI Web system, and an introduction to conducting market simulations using Sawtooth Software's online market simulator. Topics include: Conjoint methodology overview Formulating attributes and levels Designing conjoint experiments Analyzing CBC data using Counts, Latent Class, and hierarchical Bayes (HB) Using market simulators to estimate preference for competitive products in market scenarios, including price sensitivity Best practices / common mistakes related to CBC. Participants will create several conjoint surveys and analyze sample data using a team-oriented case study approach. Cost is $750.
Optional Half-Day Tutorials
	Tutorial workshops are being offered to provide opportunities for a more in-depth learning experience. Each tutorial will be led by an outstanding professional with pertinent research and teaching experience. These classes are offered only on Tuesday. Tutorials are optional and are an additional cost ($225 for one tutorial, $375 for two tutorials). Please note that you must register separately for the tutorials.
Tuesday 8 am - Noon	ADVENTURES IN ADVANCED CBC APPLICATIONS Bryan Orme, Sawtooth Software, and David Lyon, Aurora Market Modeling
	In 2009, Sawtooth Software held an advanced workshop for choice researchers called “Turbo CBC” that focused on a number of challenging and interesting extensions and applications of choice modeling, including: alternative-specific designs, bundling, menu-based choice models, and pricing models with special constraints (such that premium products are always higher priced than non-premium products). In this advanced session, we intend to show you some of the more interesting and useful examples from that course. We’ll stretch you and open your eyes to the possibilities within the family of discrete choice methods. The techniques we describe usually involve custom work that goes beyond the standard options provided in Sawtooth Software programs. This course intended for researchers with significant experience in CBC and strong command of analytics.
Tuesday 8 am - Noon	ADVANCES IN MARKET SEGMENTATION ANALYSIS Joseph Retzer, Market Tools
	The focus of this tutorial will be on new unsupervised learning (Cluster Analysis) algorithms and models. It will begin with a short review of some well known methods while paying particular attention to the Cluster Ensemble approach. Unsupervised methods covered will include: K-means PAM (Partitioning Around Medoids) Finite Mixture Modeling Clustering Objects on Subsets of Attributes (COSA) Cluster Ensembles Supervised learning techniques (e.g. CART) will also be discussed with emphasis on the relatively new and highly regarded Random Forest algorithm. In addition, Semi-Supervised Learning (SSL) algorithms, which combine information from both supervised and unsupervised learning, will be reviewed. SSL can be useful in producing clusters of high quality that are also informative to pre-specified marketing strategy. Finally, an illustration of effective post hoc cluster profiling, and presentation of cluster analysis results in general, will be demonstrated. Example code will be provided, typically in R, but also in Sawtooth Software, and Latent Gold when appropriate.
Tuesday 8 am - Noon	SEGMENTATION AND PREDICTION IN PRACTICE: Applications of Latent GOLD®, LG Choice, and CORExpress™ Jay Magidson, Statistical Innovations and Tony Babinec, AB Analytics
	In this tutorial we will use popular software to account for heterogeneity in data by identifying actionable segments and developing predictive models that allow us to better understand the segments. Output will be interpreted from a practical perspective. For each analysis we will begin with basic models understandable by non-statisticians and then introduce some advanced features. Latent GOLD -- We will uncover segments with ratings-based conjoint data and show how to use the output to customize a separate product for each segment. An advanced feature includes the use of a C-factor (random effects) as an alternative to centering. LG Choice -- We will identify respondents that differ in their brand preference and price sensitivity. We will interpret the output and show how it may be used to develop an Excel-based simulator. Advanced features include the incorporation of scale factors into the model. We will also analyze max-diff data and show how it can be integrated with ratings to improve the reliability of the estimates. CORExpress – We will show how segments can be described/predicted using exogenous variables. Recent advances in high dimensional data analysis allow inferences to be made for large numbers of attributes, respondent characteristics and interactions. We show how these advances can be used to simplify a model to include only the most important attributes/predictors/interactions.
Tuesday 8 am - Noon	AN INTRODUCTION TO THE APPLICATION OF CONJOINT ANALYSIS IN HEALTH CARE A. Brett Hauber Ph. D., RTI Health Solutions John F P Bridges, Ph. D., Johns Hopkins Bloomberg School of Public Health.
	This is a Conjoint Analysis in Health Care tutorial. Sawtooth Software attendees may also enroll, if interested. This short course will introduce participants to conjoint analysis, to motivate its use in health and to present the ISPOR Checklist for Good Research Practices. The course will be facilitated by a “hands on” group task that will demonstrate the value of the ISPOR Checklist for Good Research Practices as a means to guide the development of a conjoint analysis to elicit preferences and/or values of patients, physicians, and other decision makers. Applications to health care will also be discussed, including measuring willingness to pay, estimating healthy-year equivalents, identifying/valuing patient relevant outcomes, reweighting existing health outcomes scales, developing efficiency frontiers and estimating benefit-risk tradeoffs. Course participants will learn the conceptual and empirical basis for using conjoint analysis to elicit preferences in outcomes research. The course will introduce participants to both the conceptual basis for quantifying decision-maker preferences for medical interventions and the practical design and analytical issues that must be addressed in order to obtain valid empirical preference estimates. This course is designed for clinicians, policy makers, researchers, patient advocates/researchers with limited experience in applying conjoint analysis or other stated-preference methods in health care.
Tuesday 1 - 5 pm	ADVANCES IN MARKET SEGMENTATION (repeat session) Joe Retzer, Market Tools
Tuesday 1 - 5 pm	RESEARCH FOR SOLID PRICING DECISIONS David Lyon, Aurora Market Modeling, LLC
	Pricing decisions go right to a firm’s bottom line, so pricing research is one of the most important areas of marketing research. A number of very different approaches are used to measure price sensitivity, and sorting out their competing claims of accuracy, realism and simplicity can be very confusing. While none are a panacea, all have their place, and this session will focus on understanding the basics, pros and cons of each to help establish what to use when. We will discuss a number of approaches to the pricing problem, all ones based on surveys, including simple willingness-to-pay questions, monadic designs, van Westendorp, and its Newton-Miller-Smith variant, ratings-based conjoint and discrete choice modeling (or choice-based conjoint).
Tuesday 1 - 5 pm	INTRODUCTION TO ADAPTIVE CBC (ACBC) Bryan Orme, Sawtooth Software
	In the nearly two years since its commercial software release, Sawtooth Software’s ACBC program has captured a lot of interest and usage. It seems to offer a fine combination of the benefits of the industry-standard CBC methodology and the innovative ACA adaptive approach. Respondents and clients find ACBC surveys more engaging than CBC, and for problems involving about five attributes or more, recent research has been quite favorable for ACBC. ACBC leverages the idea that respondents use non-compensatory rules (such as “must-haves” or “unacceptables”) when creating consideration sets. Then, they select an option within the consideration set using more in-depth tradeoff strategies. In this introductory session, we’ll explain how ACBC works, show examples, and demonstrate how to program an ACBC survey within the SSI Web system.
Tuesday 1 - 5 pm	POWER TRICKS FOR SSI WEB USERS (Aaron Hill and Justin Luster, Sawtooth Software)
	Even though SSI Web is easy to begin using, there is an amazing degree of flexibility and power awaiting the adventurous and advanced user. The course will demonstrate a number of power tricks that will open your eyes to new possibilities to accomplish challenging tasks and impress your clients. You’ll see how you can take on new work and problems you previously thought could not very easily be done. Many of these tricks involve Perl, JavaScript, and CSS. We will teach you the basics of these technologies and how you can apply them to create powerful SSI Web surveys. Some of the topics will include: Free Format questions, custom JavaScript validation, advanced constructed lists, advanced formatting options with CSS, how to search and replace text across your whole study, how to include Flash, how to build drag and drop sliders, surveys for the iPhone, and how to include audio and video into your surveys. Sample study code will be shared.
Tuesday 1 - 5 pm	APPLYING BEST-WORST SCALING IN HEALTH CARE Dr. Terry Flynn and Professor Jordan Louviere Centre for the Study of Choice, UTS, Australia
	This is a Conjoint Analysis in Health Care tutorial. Sawtooth Software attendees may also enroll, if interested. Best-worst scaling (BWS) is increasingly being used to elicit more information from respondents than traditional discrete choice/conjoint tasks. By asking about the least preferred option as well as the most preferred one, it considerably increases the amount of information obtained about the respondent's utility function. In so doing, researchers can gain a better understanding of differences in respondents' choice consistency (which is vital in demonstrating unbiasedness of the estimates). This short course will be run by Dr. Terry Flynn (Centre for the Study of Choice, UTS, Australia) with contribution from Professor Jordan Louviere, the inventor of BWS. It will introduce three types of BWS and, using applications in health, illustrate their ability to estimate individual level models. These models will have considerable benefits in valuing both quality of life and patient reported outcomes
Main Conference Sessions
Wednesday October 6, 2010
Session 1: Screening Rules and Complex Search Space
Ely Dahan Princeton University and Claremont’s Drucker School of Management	What Drives Me? A Novel Application of Real-Time, Adaptive, Web-Based Conjoint Analysis to Individual Vehicle Consideration Set Formation
	We combine conjoint, user configurators, population data, and crowd wisdom in a novel way. This integrated decision aide helps individuals form vehicle consideration sets through adaptive, choice-based questioning, with estimates updated continuously and response errors allowed. We build on Dahan’s prior CARDS best paper presented at the 2007 Sawtooth Software Conference.
Sören W. Scholz, Bielefeld University, Beate Sarnowski, Marie Schuir, Interrogare, and Reinhold Decker, Bielefeld University	Analyzing Consumers’ Screening Rules by Means of Virtual Online Shops
	Many product categories comprise hundreds of product alternatives. It is well known that consumers apply non-compensatory screening rules to limit their information processing in these purchase situations. This paper discusses the application of a virtual online shop to analyze the pre-screening of choice options and the formation of consideration sets in online surveys.
Keith Chrzan, John Zepp, and Joseph White, Maritz Research	The Success of Choice-Based Conjoint Designs When Respondents Make Lexicographic Choices
	Respondents making non-compensatory choices may be poorly represented by minimal overlap design strategies assuming a compensatory MNL choice process. We show this for artificial respondents and then assess the magnitude of the potential problem for a variety of design strategies using two large commercial studies in which pretest respondents exhibited non-compensatory decision processes.
Break
Session 2: Menus and Bundling
Bryan Orme, Sawtooth Software	Menu-Based Choice Modeling Using Traditional Tools
	We’re commonly facing multi-check, menu-based pricing problems. Randomized designs make these relatively easy to design, but data processing and analysis can be challenging. Using a real dataset with 1600 respondents, we investigate different ways to model choices for options on new automobiles using Sawtooth Software tools, especially CBC/HB.
Chris Moore and Corrine Moy, GfK NOP	Analyzing Pick n’ Mix Menus via Choice Analysis to Optimize the Client Portfolio
	Menu-based choice research is becoming more prevalent in our industry, following the trends toward mass customization. This case study conducted by GFK NOP for a restaurant chain fuses a series of cross-effects logit models to optimize pricing for key dishes to ultimately increase net profit for the client.
Jack Horne, Silvo Lenart, Bob Rayner, and Paul Donagher, Market Strategies International	An Empirical Test of Bundling Techniques for Choice Modeling
	We compare and contrast two methodologies for product/service bundling research: a CBC approach that uses availability effects to estimate a feature’s underlying intrinsic utility (conditional on prices offered); and the binomial choice-modeling approach developed by Leichty, Ramaswamy and Cohen (2001) for the analysis of build-your-own data.
Lunch
Session 3: Non-Conjoint Methods
Kevin Lattery, Maritz Research	Anchoring Maximum Difference Scaling Against a Threshold: Dual Response and Direct Binary Responses
	Maximum Difference Scaling is widely used to measure the relative value of a set of attributes. This presentation considers two methods for anchoring MaxDiff scores to a threshold: Dual-Response MaxDiff suggested by Louviere and a more direct method asking respondents to choose which attributes are above a threshold.
Karen Buros, Radius Global Market Research	Directing Product Improvements from Consumer Sensory Evaluations
	Using consumer evaluations to guide product development is problematic when a product fails to achieve its goals. This paper explores an alternative to Penalty Analysis to understand which attributes play a greater role in driving satisfaction using Latent Class regression and Excel-based simulation.
Rosanna Garcia, Northeastern University, and Ting Zhang, Xi’an Jiaotong University	Policy Implications on the Diffusion of Alternative Fuel Vehicles: An Agent-Based Modeling Approach
	This paper demonstrates the use of an agent-based model to investigate factors that can speed the diffusion of eco-innovations. In our model, we use choice-based conjoint data to elicit heterogeneous consumer preferences for alternative fuel vehicles (AFVs). In a series of experiments, mechanisms are considered for speeding the adoption of AFVs.
Break
Session 4: Respondent Interaction with Choice Questionnaires
Bob Goodwin, Lifetime Products	The Impact of Respondents’ Physical Interaction with the Product on Adaptive Choice Results
	A manufacturer of folding furniture wanted to determine the potential impact of respondents’ physical interaction with the product on the precision of adaptive choice results. Split-sample ACBC studies were conducted using online and mall-intercept field methods. Market simulation results were then validated using actual product sales and market share distributions.
Martin Meißner, Sören W. Scholz, and Reinhold Decker, Bielefeld University, Germany	Using Eye Tracking and Mouselab to Examine How Respondents Process Information in Choice-based Conjoint Analysis
	Information processing is a black box in CBC. However, cognitive psychology has shown that most respondents limit information processing when making choices. This paper investigates respondents’ information acquisition behavior by means of process tracing techniques and discusses the impact on the design of CBC surveys with respect to information overload, interview length, unacceptable attribute levels and respondent learning.
Thursday October 7, 2010
Session 5: Joint Sawtooth Software & Conjoint in Health Care Session
Liana Fraenkel, Yale University School of Medicine, VA Connecticut Healthcare System	The Value of Conjoint Analysis in Health Care for the Individual Patient
	As patients play a greater role in health-related decisions, their assessment of risks and benefits of treatment and cancer screening becomes a critical factor. In this paper, we will describe the value of conjoint analysis as a tool to enable patients to construct their preferences and to better understand the impact of specific attributes on choices.
Marsha Wittink, Knashawn Morales and Mark Cary, University of Pennsylvania School of Medicine	Personalizing Treatment for Depression: Developing Values Markers
	Despite existing effective depression treatments, poor adherence is prevalent. Analogous to genetic markers, profiles of genetic variation related to treatment response, this research proposes to identify values markers, profiles of values related to the attributes of treatment that patients value most. Values markers can provide specific guideposts for how to design personalized depression treatment.
Break
Session 6: Issues in Designing Discrete Choice Studies
Paul Johnson, Western Wats	Conjoint Design Effect on Respondent Engagement throughout a Survey
	Adaptive CBC has been shown to be more enjoyable for respondents than CBC, but enjoyment doesn’t always mean respondent fatigue is reduced. We examine fatigue metrics and other questions typically asked in a survey outside the conjoint portion in a split cell design to compare conjoint design and placement effects.
Marco Hoogerbrugge and Eline van der Gaast, SKIM Analytical	Sales Promotion in Conjoint Analysis
	This presentation is about sales promotion as an attribute in conjoint studies. It will start with a general theoretical background, followed by discussing various types of sales promotions: price discount, extra volume, etc.
Jane Tang and Andrew Grenville, Vision Critical	How Many Questions Should You Ask in CBC Studies? – Revisited Again
	The previous two papers of the same title conclude that respondents can reasonably handle a large number of choice tasks (up to 20), and there is increasing reliability in predicting holdout tasks at between the 10-15 tasks mark. We reason this may be partly a result of respondents increasingly relying on heuristic simplifying rules.
Lunch
Session 7: Portfolio Optimization
Matthew Selove, USC	The Strategic Importance of Accuracy in Conjoint Design
	Improved accuracy in conjoint analysis has important strategic implications. Even if two models provide unbiased partworths, competitive game theory shows that the more-accurate model (with lower error variance in an HB CBC model) implies product differentiation is more profitable. I illustrate the theory by varying accuracy in a student-apparel application.
Christopher N. Chapman, Microsoft and James L. Alford, Nytec	Product Portfolio Evaluation Using Choice Modeling and Genetic Algorithms
	We describe using genetic algorithm (GA) models to find near-optimal product portfolios in the presence of competition, using individual-level part worths from CBC and ACBC models. We describe how we have used this method to evaluate the performance of product portfolios, inform optimal portfolio size, and generate hypotheses about product opportunities.
Break
Session 8: Covariates in HB
Keith Sentis and Valerie Geller, Pathfinder Strategies	The Impact of Covariates on the Quality of HB Estimates
	Following on from Sentis & Li (2001) and Frazier et al. (2009), we address HB analyses in the context of segmentation by examining the impact of covariates on the quality of the estimated utilities. For this project, we focus on 10 commercial datasets from studies involving both services and FMCG with sample sizes ranging from 400 to 6,000.
Peter Kurz, TNS Infratest Forschung GmbH, and Stefan Binner, bms marketing research + strategy	Added Value through Covariates in HB Modeling
	We analyze the validity of predictions (compared to real data) using traditional HB (total and subgroups), covariates and split approach for multinational and multi-channel studies. The paper will describe conditions for successful application of covariates as well as reasons for failure and will provide guidelines for the application of covariates.
Friday October 8, 2010
Concurrent Break-Out Session:	How to Get Things Right and Avoid Potential Errors with Sawtooth Software’s CBC: A Critique of Sample CBC Studies
Session 9: Advanced Topics
Thomas Eagle, Eagle Analytics of California	Modeling Demand Using Simple Methods: Joint Discrete/Continuous Modeling
	We describe the joint discrete/continuous volumetric model for choice experiments and compare it to two other forms of simple volumetric models: regression-based and choice based models using several actual data sets. The ease of use and flexibility of the approach to handling very complex menu based choice experiments is described.
Stuart Drucker, Drucker Analytics	Recent Developments in PLS Modeling for Customer Loyalty and Brand Retention
	Partial least squares (PLS) is a structural equation modeling approach that builds on the strengths of factor analysis and regression analysis for key driver modeling (i.e. customer loyalty/satisfaction research). We will present potential enhancements to conventional PLS that include estimation of higher-order key driver relationships, more stable estimation of scoring weights within drivers, and non-linearity of key drivers.
Break
Session 10: Top-Down and Bottom-Up CBC
Don Marshall, TVG, Siu-Shing Chan, University of Pennsylvania, and Joseph Curry, Sawtooth Technologies, Inc.	A Head-to-Head Comparison of the Traditional (Top-Down) Approach to Choice Modeling with the New (Bottom-Up) Approach
	At the 2009 Sawtooth Software Conference Professor Jordan Louviere discussed published research indicating problems with the traditional approach to choice modeling leading to misleading results. He then presented a technique to eliminate these biases. We will report on a direct comparison of the new approach with the traditional approach (CBC).
Ralph Wirth, GfK Group	HB-CBC, HB-Best-Worst-CBC or No HB at All?
	We present results of a research project aiming at (1) developing a flexible HB-model for estimating individual utility parameters based on best-worst choices, (2) empirically testing different possible probabilistic best-worst choice models and (3) systematically comparing the developed HB-Best-Worst-CBC approach with the standard HB-CBC approach and a non-HB approach suggested by Louviere et al. (2008) by means of a simulation study.
Conference Adjourned