Optional Workshops and Tutorials
Optional 1.5-Day Workshops
Monday 8 am - 5 pm & Tuesday 8 am - Noon	CBC MODELING WORKSHOP Aaron Hill and John Howell, Sawtooth Software
	The CBC Modeling Workshop is designed to introduce participants to discrete choice (CBC) analysis through an interactive, hands-on workshop. The course will include an overview of conjoint analysis techniques and methodology, with specific attention to CBC/Web. Attendees will also receive practical experience creating surveys in Sawtooth Software’s SSI Web system. Topics include: Conjoint methodology overview Formulating attributes and levels Designing conjoint experiments Analyzing CBC data using Counts, Logit, Latent Class, and hierarchical Bayes (HB) Using market simulators to estimate preference for competitive products in market scenarios, including price sensitivity Best practices / common mistakes related to CBC. Participants will create several conjoint surveys and analyze sample data using a team-oriented case study approach. Cost is $750.
Monday 8 am - 5 pm & Tuesday 8 am - Noon	ADAPTIVE CHOICE (ACBC) WORKSHOP Bryan Orme and Justin Luster, Sawtooth Software
	Adaptive CBC is a new discrete choice method that leverages three interactive stages: BYO, Screening Tasks, and Choice Tasks. It takes advantage of the idea that buyers often have non-compensatory cutoff rules (must-haves and unacceptables) that they use to build consideration sets of products. Once a consideration set is formed, buyers trade off the features among considered products in making a final purchase. With adaptive CBC, the products shown to respondents are customized to be relatively near their preferred (BYO) choice, rather than randomly drawn from the entire range of possibilities. Only users who are quite experienced in CBC and proficient in programming surveys within SSI Web should attend. Attendees will use a demo copy of ACBC software to program three separate Adaptive Choice surveys. The class covers both design and analysis of ACBC surveys. Cost is $750.
Optional Half-Day Tutorials
	Tutorial workshops are being offered to provide opportunities for a more in-depth learning experience. Each tutorial will be led by an outstanding professional with pertinent research and teaching experience. These classes are offered only on Tuesday. Tutorials are optional and are an additional cost ($225 for one tutorial, $375 for two tutorials). Please note that you must register separately for the tutorials.
Tuesday 8 am - Noon	CHOICE MODELLING BEST PRACTICES Jeff Brazell and Cindy Ford, The Modellers
	This tutorial is for the research analyst or client-side researcher with intermediate-level experience conducting and analyzing choice projects. It is not an introduction to the methodology nor is it aimed at academics. The main focus is the difficult practical issues that arise when executing choice modelling projects and how to ensure that you get good results. First, issues related to correctly setting up choice tasks will be addressed such as level of task complexity and more accurate response variables. Second, critical experimental design issues such as strategies for defining alternatives and variables, partial profiles, interactions, and design restrictions will be explored. Third, estimation issues such as constraining estimates (e.g., price coefficients), and estimating more sophisticated models HB models (e.g., Threshold, non-compensatory, simultaneous calibration) will be discussed. Fourth, post-estimation issues such as post-hoc corrections for IIA, calibration, assumptions and validity, forecasting, and optimization will be examined. Finally, we will demonstrate and discuss advanced simulators and talk about issues in optimization and interpretation. The result of the tutorial will be a greater ability to execute and manage complex choice projects, with an eye toward better foreseeing complexity and analysis issues that will arise, avoiding costly mistakes and ensuring more accurate, actionable results.
Tuesday 8 am - Noon	ADVANCES IN MARKET SEGMENTATION ANALYSIS Joseph Retzer, Maritz Research
	The focus of this tutorial will be on new unsupervised learning (Cluster Analysis) algorithms and models. It will begin with a short review of some well known methods while paying particular attention to the Cluster Ensemble approach. Unsupervised methods covered will include: K-means PAM (Partitioning Around Medoids) Finite Mixture Modeling Clustering Objects on Subsets of Attributes (COSA) Cluster Ensembles Supervised learning techniques (e.g. CART) will also be discussed with emphasis on the relatively new and highly regarded Random Forest algorithm. In addition, Semi-Supervised Learning (SSL) algorithms, which combine information from both supervised and unsupervised learning, will be reviewed. SSL can be useful in producing clusters of high quality that are also informative to pre-specified marketing strategy. Finally, an illustration of effective post hoc cluster profiling, and presentation of cluster analysis results in general, will be demonstrated. Example code will be provided, typically in R, but also in Sawtooth Software, and Latent Gold when appropriate.
Tuesday 8 am - Noon	METHODS, DESIGNS, AND APPLICATIONS OF CONJOINT ANALYSIS IN HEALTH Deborah A. Marshall, University of Calgary, and Ken Deal, PhD, McMaster University Additional Facilitators: Joel Huber, Duke University, Reed Johnson, RTI Health Solutions, Liana Fraenkel, Yale University, Charles Cunningham, McMaster University
	This is a Conjoint Analysis in Healthcare tutorial. Sawtooth Software attendees may also enroll, if interested. This workshop is designed to provide a broad introductory overview to conjoint analysis methods for healthcare researchers, and will help researchers expand their skill sets by exposing them to a variety of successful health research projects that leverage conjoint analysis methodology. After a brief introduction to how conjoint analysis works and why it is used, the workshop will illustrate different approaches to conjoint analysis and demonstrate how conjoint analysis has been used in a variety of healthcare related applications. A number of specific case studies will be presented with interactive discussions of practical issues. This workshop is designed for researchers with limited exposure to conjoint analysis in health applications who are interested in exploring and expanding their knowledge of alternative conjoint analysis methods and designs. Attending this workshop should help those in health research better appreciate the presentations in the parallel Sawtooth Software Conference.
Tuesday 1 - 5 pm	CHOICE MODELLING BEST PRACTICES (repeat session) Jeff Brazell and Cindy Ford, The Modellers
Tuesday 1 - 5 pm	ADVANCES IN MARKET SEGMENTATION ANALYSIS (repeat session) Joseph Retzer, Maritz Research
Tuesday 1 - 5 pm	BECOMING A CBC/HB POWER USER AND MORE Eleanor Feit, University of Michigan, Jeffery Dotson, The Ohio State University, John Howell, Sawtooth Software
	CBC/HB and the other Sawtooth Software hierarchical Bayes packages have become standard tools for many market researchers. Because the software was designed to be fast and easy to use, some common market research problems cannot be solved with the default settings in the software. This tutorial will teach you how to deal with some of these problems, using advanced features of CBC/HB and the R statistical language when necessary. Some of the topics covered will be: Using custom designs with CBC/HB Using informative priors with HB models Dealing with small sample sizes Combining data from multiple sources Managing draws files How to include additional respondent information in HB models Segmentation strategies with HB Data Metropolis-Hastings algorithms, with specific focus on Discrete Choice and Response models Note: This is an advanced class on hierarchical Bayes. The class is intended to present practical solutions to common problems that cannot be easily solved with Sawtooth Software’s packaged HB solutions. Because of the nature of the class, participants should have some familiarity with Bayesian statistics and be comfortable with programming ideas. Many of the solutions presented will require some degree of custom programming. Familiarity with the R language is helpful, but not required.
Tuesday 1 - 5 pm	INTRODUCTION TO CIW PROGRAMMING (David Squire and Brian McEwan, Sawtooth Software)
	This hands-on workshop introduces attendees to Sawtooth Software’s web interviewing system, SSI Web. Emphasis will be placed on developing general surveys for market research using SSI Web’s CiW module. We’ll discuss: Composing surveys including question types, list building, skip patterns, quota control, graphics, and data verification Fielding surveys including respondent passwords, merged fields, fields passed in from or out to another survey, basic server setup, and inviting respondents to take the survey At the end of the workshop we’ll show a preview of advanced CiW programming using Perl, JavaScript, and other technologies. Please bring your own laptop with either SSI Web or SSI Web Demo version loaded (version 6.4). Classroom exercises will be assigned and discussed. A CD of assignments, solutions, and classroom slides will be distributed. Some exposure to HTML is helpful, but not necessary.
Tuesday 1 - 5 pm	CRITICAL EVALUATION OF CONJOINT ANALYSIS STUDIES: A CHECKLIST FOR GOOD PUBLICATIONS Brett Hauber, Research Triangle Institute, and John F. P. Bridges, Bloomberg School of Public Health, Johns Hopkins University
	This is a Conjoint Analysis in Healthcare tutorial. Sawtooth Software attendees may also enroll, if interested. The purpose of this workshop is to identify the stages of study design and analysis in applications of conjoint analysis to health and to provide an overview and discussion of critical study parameters that researchers should consider at each stage of a study. The workshop will be structures using a checklist of key issues for researchers to consider in conducting conjoint-analysis studies and reporting results. Each item in the checklist and its importance will be discussed and each item will be illustrated with real-world examples. In addition, this workshop will provide an overview of current practices in conjoint analysis. Finally, the workshop will address recommendations for preparing manuscripts to be submitted to peer-reviewed scientific journals and for evaluating the quality of published studies. The workshop will include presentations, hands-on exercises, and discussions of real-world applications. This workshop is designed for researchers at all levels of experience and expertise, but likely will be most useful for those new to the research area and for those who have limited experience with using conjoint analysis in health applications. For those new to conjoint analysis, the overview of current practice and discussion of key elements of conjoint analysis studies will provide a comprehensive introduction to conjoint analysis. More experienced researchers are invited to attend to provide their input and perspective on all topics discussed in this workshop.
Main Conference Sessions
Wednesday March 25, 2009
Chris Goglia and Alison Strandberg Turner Critical Mix	To Drag-n-Drop or Not? Do Interactive Survey Elements Improve the Respondent Experience and Data Quality?
	Do interactive survey elements affect respondent attention, participation and resulting data quality in an online survey? We examined completion speed, statistical differences between responses, stated satisfaction and verbatim feedback. Interactive questions took longer, provided almost identical responses, and resulted in slightly lower satisfaction. We end with suggestions for making surveys more interesting to respondents.
Deb Ploskonka Cambia Information Group Raji Srinivasan University of Texas at Austin	Design Decisions to Optimize Scale Data for Brand Image Studies
	Questionnaire design requires researchers to navigate a myriad of choices to design an instrument that will capture the information needed. We discuss several scale design decisions that researchers make every day, and how each affects the data that is collected. Detailed case studies will be shown to support design recommendations.
Lynd Bacon Loma Buena Associates and Ashwin Sridhar ZLINQ Solutions	Playing for Fun and Profit: Serious Games for Marketing Decision-Making
	Academics and firms have begun to use games to predict, explain preferences, solve problems, and generate ideas. These games are "purposive;" they have objectives other than entertainment or education. In this presentation we review recent applications, summarize an online platform we built to run group problem-solving games, and discuss game design principles and deployment issues.
Break
Andrew Elder and Terry Pan Illuminas	Survey Quality and MaxDiff: An Assessment of Who Fails, and Why
	Survey response quality has come under renewed scrutiny in response to the industrialization of online recruitment. A battery of quality metrics typically identify “bad” respondents who answer quickly or haphazardly. MaxDiff models provide additional insight into survey quality, contributing to better quality review and further understanding of response characteristics that threaten survey quality.
Jay Magidson Statistical Innovations Inc. Dave Thomas Roche Diagnostics Corp. and Jeroen K. Vermunt Tilburg University	A New Model for the Fusion of MaxDiff Scaling and Ratings Data
	An innovative approach to MaxDiff analysis suggests that the inclusion of ratings in a fused model may result in segments that differ more with respect to factors considered important by lab managers and less with respect to those that are not important. The approach is implemented using commercially available software.
Lunch
John Ashraf, Marco Hoogerbrugge, and Juan Tello SKIM	Benefits of Deviating from Orthogonal Designs
	While orthogonal research designs are the gold standard in CBC, these may actually prevent the researcher from accurately answering the client's questions at hand. Instead, we suggest tailoring the research design to address the specific scenarios the client wishes to explore, at the expense of perfect orthogonality.
Paul Johnson and Bob Fawson Western Wats	Collaborative Panel Management, The Stated and Actual Preference of Incentive Structure
	Keeping panelists engaged and satisfied with the research process is essential to managing panels. We focus on finding effective ways to provide an incentive to those panelists who do not qualify for a study. We show how stated and actual preference may differ when uncertainty of winning rewards is involved.
Break
Joseph Retzer, Sharon Alberg, and Jianping Yuan Maritz Research	Achieving Consensus in Cluster Ensemble Analysis
	This presentation will compare various algorithms for achieving consensus in Cluster Ensemble analysis on the basis Adjusted Rand Index values using synthetic data. Attendees will become familiar with the Cluster Ensemble approach, its benefits and variations in implementation. Guidance in practical application of the latter will also be provided.
Chris Diener and Urszula Jones Lieberman Research Worldwide (LRW)	Having Your Cake and Eating It Too: Approaches for Attitudinally Insightful and Targetable Segmentations
	We focus on the increasingly visible problem of creating segmentation solutions which contain segments that differ meaningfully and significantly on both the “softer” attitudinal measures and the “harder” targeting variables such as demographics and transactions. Reverse Segmentation (Sawtooth 2006) and Nascent Linkage Maximization (ART 2002) will be compared to each other and to traditional approaches.
Thursday March 26, 2009
V. “Seenu” Srinivasan Stanford University Gordon A. Wyner Millward Brown International	An Improved Method for the Quantitative Assessment of Customer Priorities
	Adaptive Self-Explication (ASEMAP) is a new computer-based interactive method for assessing customer priorities. It provides higher predictive validity compared to the more traditional Constant Sum (CSUM) method. The empirical context is that of assessing research priorities among fifteen topics from managers of Marketing Science Institute’s member companies.
Keith Chrzan and Dan Yardley Maritz Research	Tournament-Augmented CBC
	We follow a standard D-efficient CBC choice experiment with a single-elimination tournament among alternatives chosen in the experimental choice sets. We compare basic CBC to a tournament-augmented CBC in terms of a) Hit rates for holdout D-efficient choice sets and choice sets comprised of high-utility alternatives, b) Equivalence of model parameters in D-efficient CBC and high-utility tournament choice sets.
Kevin Lattery Maritz Research	Coupling Stated Preferences with Conjoint Tasks to Better Estimate Individual Level Utilities
	In addition to a conjoint interview, respondents stated preferences about each attribute: whether levels were Completely Unacceptable, Acceptable, or Highly Desired. We describe several ways of including these stated preferences: additional partial profiles, EM CBC constraints, soft cutoffs, and screening rules. These methods improved holdout predictions and impacted the findings.
Break
Bob Goodwin Lifetime Products	Introduction of Quantitative Marketing Research Solutions in a Traditional Manufacturing Firm: Practical Experiences
	A traditionally non-marketing-driven manufacturing company has introduced progressively more sophisticated conjoint and other quantitative marketing research tools over the past three years. Of particular interest to conference attendees is the practical experience gained in helping internal clients to understand and trust these tools for product development and marketing decision-making.
Christopher N. Chapman Microsoft Corporation James L. Alford Volt Information Sciences Chad Johnson Answers Research Michal Lahav Sakson & Taylor Consulting and Ron Weidemann Answers Research	Comparing Results of CBC and ACBC with Real Product Selection
	We compare CBC and ACBC exercises using a within-subjects model and two external validity criteria. Respondents all took CBC and ACBC exercises with identical attributes, and the resulting utilities were used to predict respondents’ actual choice between two products. We also compare the methods’ estimates with external market data.
Lunch
John Hauser MIT Steve Gaskin Applied Marketing Sciences and Min Ding Pennsylvania State University	A Critical Review of Non-Compensatory and Compensatory Models of Consideration-Set Decisions
	We review and contrast recent research on consideration-set decisions. Methods include both “revealed” and “self-explicated” techniques to infer the, potentially non-compensatory, decision rules that consumers use to form consideration sets. We examine variation in performance by data-collection method, estimation method, underlying model of decision-making, and product category.
Robert A. Hart and David G. Bakken Harris Interactive	Using Agent-Based Simulation to Investigate the Robustness of CBC-HB Models
	In this paper we describe the use of agent-based simulation to generate pseudo populations for the investigation of the robustness of CBC-HB models in the face of various consumer choice heuristics. Using a pseudo population with varying mixtures of decision heuristics, we compare estimation using CBC-HB with the conjunctive screening model of Gilbride and Allenby.
Break
Jane Tang and Andrew Grenville Angus Reid Strategies Vicki G. Morwitz and Amitav Chakravarti New York University and Gülden Ülkümen University of Southern California	Influencing Feature Price Tradeoff Decisions in CBC Experiments
	In typical CBC, respondents are shown combinations of product features at different prices and asked for their choices. The prices respondents are willing to tradeoff for features are often unrealistically high. We examine several manipulations and their impact on respondents’ price/feature tradeoff decisions, including narrow vs. broad scale manipulation, a budgeting session, a Van Westendorp pricing exercise and point of sale price comparisons.
Min Ding Pennsylvania State University and Joel Huber Duke University	Motivating Respondents in Preference Measurement Tasks: Methods, Results, and Implications
	The quality of insights from preference measurement methods is limited by the quality of the data they collected. We review recent research on incentive aligning respondents, aimed to improve data quality by ensuring it is in the best interest of a respondent to state truthfully. We also review common practices adopted in the field to encourage respondents to report values that correspond to their actions in the marketplace.
Friday March 27, 2009
Richard McCullough MACRO Consulting, Inc.	Comparing Hierarchical Bayes and Latent Class Choice: Practical Issues for Sparse Datasets
	Latent Class Choice Models (LCC) are an alternative to Hierarchical Bayes (HB) that may offer the practitioner some advantages when dealing with sparse datasets. Using several commercial datasets with a large number of attributes, utilities will be estimated using both HB and LCC and the results compared. Practical advantages and limitations of each technique will be discussed.
Curtis Frazier Probit Research, Inc. Urszula Jones Lieberman Research Worldwide (LRW) and Michael Patterson Probit Research, Inc.	Estimating MaxDiff Utilities: Dealing with Respondent Heterogeneity
	This paper investigates the impact of estimating MaxDiff models (i.e., individual level utilities) at the total sample level vs. estimation by a priori segments or based on other potential levels. The results reveal the estimation approach that maximizes data accuracy without giving up respondent heterogeneity.
Break
Towhidul Islam, Jordan Louviere, and David Pihlens University of Technology, Sydney	Aggregate Choice and Individual Models: A Comparison of Top-Down and Bottom-Up Approaches
	We discuss a new way to estimate models for individuals. We compare it to mixed logit and hierarchical bayes. In- and out-of-sample comparisons strongly favor individual-level models. We show that 40-80% of the variance in individual estimates is due to within-person variability, not preference heterogeneity. We note implications for practice.
Kamel Jedidi Columbia University Sharan Jagpal Rutgers University and Madiha Ferjani University of Tunis	Using Conjoint Analysis for Market-Level Demand Prediction and Brand Valuation
	This paper develops and tests a new, easy-to-use conjoint-based methodology for measuring brand equity. Three novel features of the model are that it explicitly allows for competitive reaction, market expansion and contraction, and uses consideration set theory to translate market-share estimates from the conjoint experiment to the marketplace.
Conference Adjourned