Sawtooth Software: The Survey Software of Choice

Pre-Conference Workshops (Monday, Tuesday)

Monday Morning

(8:00 AM – 12:00 PM) CBC Software Workshop (Part 1 of 4)

Brian McEwan, Sawtooth Software, Inc.

Nathan Bryce, Sawtooth Software, Inc.

If you are relatively new to choice-based conjoint (CBC) or just getting started, join us for two days of hands-on practice with Lighthouse Studio, our flagship survey platform that can include the popular CBC component and market simulator. We'll cover the main aspects of designing, programming, and analyzing CBC studies. You will have an opportunity to program CBC questionnaires individually as well as analyze data from a real CBC study in a team-oriented case study session. We'll provide coverage of counting analysis, logit, latent class, and HB. The instructors will share best practices, pitfalls to avoid, and experiences based on many years of technical support and consulting.

Attendees receive an evaluation copy of the software that they may use for 90 days (for non-commercial studies and evaluation purposes only). Attendees also receive a free copy of the "Getting Started with Conjoint Analysis" book. Limited to 25 participants.

(8:00 AM – 12:00 PM) Introduction to Menu-Based Choice

Bryan Orme, Sawtooth Software, Inc.

Menu-Based Choice (MBC) is a relatively new and flexible choice modeling approach for solving a variety of multi-check (combinatorial) menu-selection problems. Examples include: choosing options to put on an automobile, selections from a restaurant menu, banking options, configuring an insurance policy, purchasing bundled vs. a la carte services including mobile phones, internet, and cable. Using the MBC software requires expertise in terms of experimental design for conjoint exercises, developing MBC surveys using advanced HTML/Javascripting, and some data processing to prepare the files in .csv-format for analysis within MBC. MBC has a built-in simulator and can also automatically export Excel simulators.

Come spend four hours seeing how MBC software can help you tackle menu choice situations, including other interesting extensions such as situational choice (polytomous logit) and perceptual choice experiments.

(8:00 AM – 12:00 PM) Lighthouse Studio Workshop

Justin Luster, Sawtooth Software, Inc.

Gary Baker, Sawtooth Software, Inc.

Lighthouse Studio is a powerful tool that allows you to create general surveys, including those with conjoint analysis and MaxDiff. In this hands-on workshop we will be covering almost everything that Lighthouse Studio can do. Topics covered include:

  • General questions (select, numeric, ranking, grid, etc.)
  • Skip logic, Constructed lists, Randomization, Looping, Quota control
  • Introduction to programming CBC, MaxDiff, and ACBC
  • Testing & publishing the survey, linking to other systems
  • Offline interviewing
  • Introduction to advanced customization: HTML, JavaScript, CSS, Perl
Attendees must bring a laptop PC with Lighthouse Studio installed (a demonstration version will be given to you in advance for the purposes of classroom instruction).

(8:00 AM – 12:00 PM) Introduction to R for Market Researchers

Christopher Chapman, Google

Kenneth Fairchild, Sawtooth Software, Inc.

Introduction to R for Marketing Researchers is a 4-hour, hands-on introduction to the R statistics platform focused on marketing applications. The course presents the basics of the R language, statistical procedures, and graphics. The tutorial begins with the basics of the command-line language and data processing in R. We then examine common statistical analyses such as regression models. Finally we discuss applied topics for marketing research, including a brief overview of R options for clustering and segmentation, choice models, and automated reporting.

A laptop is required for this tutorial, and you should have R installed before coming to the class. All attendees will receive a signed copy of Chapman & Feit, R for Marketing Research and Analytics (Springer, 2015). Those taking both R tutorials will receive a printed and electronic copy of the book.

Monday Afternoon

(1:00 PM – 5:00 PM) CBC Software Workshop (Part 2 of 4)

(1:00 PM – 5:00 PM) How People Choose: Understanding Customer Decision Making

Keith Chrzan, Sawtooth Software, Inc.

This tutorial focuses on the strategies customers use to make decisions and on the statistical models analysts use to understand and quantify those strategies.

The most widely used model in the field of marketing research is the Random Utility Model (RUM), brought to life in the multinomial logit (MNL) choice model. This model supports choice-based conjoint (CBC) analysis, MaxDiff scaling, ACBC and many other techniques marketing researchers use. As convenient and popular as it is, however, the MNL model makes sometimes unrealistic assumptions about human behavior (e.g. the property called Independence from Irrelevant Alternatives or IIA). These assumptions have led to interest in alternative choice models, for example, non-compensatory models like the lexicographic and elimination-by-aspects (EBA) choice models. In addition, augmented versions of the MNL model which enable two-stage models combining a non-compensatory first stage elimination with a compensatory RUM second stage have been developed and tested.

Recent attempts to incorporate additional behavioral effects like loss aversion and extremeness aversion into the MNL model have led to a class of semi-compensatory models that incorporate context effects, like Random Regret Minimization (RRM) and Relative Advantage Maximization (RAM).

The tutorial reviews, illustrates and compares the models listed above and several others. It reviews evidence that survey respondents may not always make choices according to RUM, that they can change their decision strategies from one situation to the next and that they can employ different strategies even within a single choice occasion.

(1:00 PM – 5:00 PM) Intermediate R for Market Researchers

Christopher Chapman, Google

Kenneth Fairchild, Sawtooth Software, Inc.

Intermediate R for Marketing Researchers is a 4 hour, hands-on tutorial that builds on the Beginning R tutorial, and is also suitable for others who already have experience in the core R language. It starts with linear models in R, and then covers hierarchical linear models, Bayesian linear modeling, and an introduction to working with choice models in R. For choice models, we will import and work with estimates from Sawtooth Software and also see how to estimate models entirely in R. Finally the class briefly covers applied topics for marketing research, including an overview of R options for clustering and segmentation, and topics from attendees.

A laptop is required for this tutorial, and you should have R installed before coming to the class. All attendees will receive a signed copy of Chapman & Feit, R for Marketing Research and Analytics (Springer, 2015). Those taking both R tutorials will receive a printed and electronic copy of the book.

(1:00 PM – 5:00 PM) Guided Tour of CBC Design and Analysis,

Bryan Orme, Sawtooth Software, Inc.

Have you ever wished you could "be a fly on the wall" and watch an expert use the CBC software, narrating what to pay attention to, what the numbers in the report mean, how to avoid pitfalls, and what certain advanced settings do? Have you wished you could learn more about what to pay attention to during analysis to obtain good results and make proper conclusions?

Come watch as we step a CBC project through the design and analysis phases using Lighthouse Studio's CBC system. Learn best practices! Ask questions! See what the expert sees and gain new insights into the elegant and powerful methodology that is CBC.

Tuesday Morning

(8:00 AM – 12:00 PM) CBC Software Workshop (Part 3 of 4)

(8:00 AM – 12:00 PM) Advanced Lighthouse Studio Workshop (Part 1 of 2)

Justin Luster, Sawtooth Software, Inc.

Dave Squire, Sawtooth Software, Inc.

Lighthouse Studio is a powerful application that has been designed to be very flexible. Custom code can be added to modify the appearance and functionality of your surveys allowing you to do amazing things. In this workshop we will learn about how to incorporate the following into your Lighthouse surveys:

  • HTML
  • CSS
  • JavaScript
  • jQuery
  • Perl
Learning a little bit about these technologies will greatly enhance your ability to create surveys that your customers will love. This workshop will be very hands on. You will be learning about these scripting languages and then applying them to a Lighthouse Studio survey. We will be on hand to help you every step of the way.

Attendees must bring a laptop PC with Lighthouse Studio installed (a demonstration version will be given to you in advance for the purposes of classroom instruction).

(8:00 AM – 10:00 AM) Random Forests

Joe Retzer, Market Probe

Ewa Nowakowska, GfK

Random Forest (RF) analysis has been gaining in popularity over the past few years as an accessible and robust approach to predictive modeling. RF analysis has been found to be particularly useful in various areas including big data analytics. The RF model indeed offers numerous advantages over other machine learning algorithms i.e.,:

  • Computational efficiency / speed on large data sets
  • The ability to effectively handle numerous variables of mixed type
  • Robustness to noise / outliers; multiple strategies for dealing with missing data
  • Highly accurate predictions at relatively low cost.
RF analysis also provides a variety of valuable outputs without requiring extensions to the model. Details depend on the implementation however typical examples include a transferable classification tool, unbiased classification error estimates and a proximity matrix which may in turn be used to segment the data. Altogether it makes RF analysis a first choice prediction & classification approach for a growing number of analysts and data scientists.

In this tutorial we will focus on explaining the fundamental concepts behind random forests and demonstrate its application using software that does not require extensive programming expertise. We will start with simple tree-based methods, useful for gaining valuable visual insight. Next, we will discuss ways of combining multiple trees in order to improve both model stability as well as predictive accuracy. Our focus will be on random forests however bagging and boosting, which complement and further develop the concept of RF analysis, will also be discussed. Practical examples will be explored using the free and open source R software package Rattle (Williams, 2009). Built on top of R, Rattle provides a means for executing low level R commands via a friendly GUI thus enabling straightforward implementation of the RF model.

(10:10 AM – 12:00 PM) Situational Choice Experiments

Keith Chrzan, Sawtooth Software, Inc.

Kenneth Fairchild, Sawtooth Software, Inc.

In some cases the best way to model choices involves using characteristics of the chooser or of the choice situation as predictors. This is in contrast to choice-based conjoint wherein the characteristics of the choice alternatives are the predictors via a conditional multinomial logit (MNL) model. Situational choice models use an unconditional MNL model, which we will introduce in this session. We supplement the theory of Situational Choice Experiments with practical how-to examples using experimental design and modeling capabilities available in Sawtooth Software products like Lighthouse Studio, CBC/HB, CBC Latent Class and Menu-Based Choice (MBC). Case studies showing the diversity of Situational Choice Experiments will round out the session.

(10:10 AM – 12:00 PM) Random Forests (Repeat)

(8:00 AM – 10:00 AM) Advanced Market Simulator Options

Bryan Orme, Sawtooth Software, Inc.

Walter Williams, Sawtooth Software, Inc.

Lighthouse v9 (previously known as SSI Web) includes a new market simulator that replaces the older SMRT simulator. We'll give you an orientation to the new market simulator with special emphasis on the new advanced features, including: sensitivity analysis, netted shares, ability to export Excel simulators, adjustments for awareness, and the new external effect adjustment to correct for product distribution. We'll also demonstrate the new capabilities for developing client conjoint simulators, including customizing the options the client may use as well as placing your own company's branding in the simulator footer.

Tuesday Afternoon

(1:00 PM – 5:00 PM) CBC Software Workshop (Part 4 of 4)

(1:00 PM – 5:00 PM) Advanced Lighthouse Studio Workshop (Part 2 of 2)

(1:00 PM – 5:00 PM) Advanced MaxDiff Concepts

Keith Chrzan, Sawtooth Software, Inc.

Whether you are relatively new to the use of MaxDiff or already quite experienced, this class provides material that will stimulate your thinking and improve your practice.

This tutorial will take a deep dive into different ways of generating, rescaling, reporting and anchoring utilities. Because there are often multiple ways these can be done, we will compare and contrast options to identify their relative strengths and weaknesses, so that you walk away with a good feel for how and why to make decisions in the design and reporting of your MaxDiff study.

An important practical topic we will cover is how to address the all-too-common situation wherein we need to handle an especially large number of items without fatiguing respondents. Several variations of MaxDiff have been advanced to do this and we will compare them.

Finally, we'll wrap up with some newer MaxDiff topics and intriguing research from the recent Sawtooth Software conferences as well as the broader literature.

(1:00 PM – 5:00 PM) Experimental Conjoint Solutions

Jeroen Hardon, SKIM Group

Sometimes standards are not enough to answer the business question at hand and you have to deviate from the roads most traveled. Jeroen Hardon would like to get you on board for this four-hour tutorial on advanced applications. He will focus on a number of interesting extensions of choice modeling, including:

  • Custom statistical designs techniques (using Lighthouse Studio and Excel), and the implications of these customizations
  • Showing different coding methods for faster processing (using Sawtooth Software CBC/HB)
  • Combining 2 MaxDiff exercises into one Latent Class run
  • Different ways of data augmentation for CBC and MaxDiff
  • Different solutions for the red bus blue bus problem
This tutorial is meant to be very practical. It will show you how to combine the power of Lighthouse Studio and Excel, while still being able to use CBC/HB. As we all know, conjoint is a mixture of art and science.

General Session Presentations (Wednesday—Friday)

Session 1

(7:00 AM - 5:00 PM) Conference Registration

(7:00 AM - 8:25 AM) Breakfast

(8:25 AM) Welcoming Remarks (Bryan Orme, Conference Moderator)

(8:30 AM) The Effects of Incentive Alignment, Realistic Images, Video Instructions, and Ceteris Paribus Instructions on Willingness to Pay and Price Equilibria

Felix Eggers, University of Groningen

John R. Hauser, MIT Sloan School of Management

Matthew Selove, University of Southern California

We compare the effects of incentive alignment, realistic images, video instructions, and ceteris paribus instructions in choice-based conjoint experiments on willingness to pay and price equilibria. We show further which factors influence the predictive validity and that these changes matter both strategically and tactically.

(9:15 AM) How Many Options? Behavioral Responses to Two versus Five Alternatives per Choice

Martin Meissner, University of Southern Denmark

Harmen Oppewal, Monash Business School

Joel Huber, Duke University

What is an appropriate number of alternatives per choice? Why are two or five alternatives so rarely used? We characterize the contexts where CBC on pairs makes sense when projecting to real decisions and use an eye-tracking study to assess the (dis-)advantages of CBC studies with different numbers of alternatives.

(10:00 AM - 10:30 AM) Refreshment Break

Session 2

(10:30 AM) Findings of the Sawtooth Software CBC Modeling Prize Competition

Bryan Orme, Sawtooth Software, Inc.

2016 Sawtooth Software Prize Winner (TBA)

Which CBC models make better predictions? Sawtooth Software organized a CBC modeling competition to enlist the help of multiple teams of researchers to investigate. We summarize what we learned about modeling CBC data and then let the winner of the competition show you how to build a winning CBC predictive model.

(11:15 AM) Using Bayes' Theorem to Adjust Simulated Preference Shares to Market Reality

David Bakken, Foreseeable Futures Group

As every practitioner working with conjoint analysis knows, even with the best possible model, simulated preference shares often do not map to actual market shares. This paper introduces a "new" method of post-estimation adjustment using Bayes' Theorem and compares adjusted results to other post-estimation adjustment methods.

(12:00 PM - 1:30 PM) Lunch

Session 3

(1:30 PM) Mobile MaxDiff: What Are the Optimal Number of Attributes, Screens and Level of Information Complexity?

Michael Patterson, Radius Global Market Research

As research on mobile devices continues to increase, it is important to understand potential limitations associated with various analytic techniques. In this paper, we explore the impact that the number of items, the number of questions and the level of complexity of information have within MaxDiff exercises.

(2:15 PM) Choice Based Conjoint in a Mobile World – How Far Can We Go?

Chris Moore, GfK

Christian Neuerburg, GfK

Given that more than two-thirds of panel respondents have used mobile devices to answer surveys, we conduct the most comprehensive study known using an 18 split-sample design and more than 6,800 (real) respondents to evaluate what effect different conjoint designs have on conjoint data. We derive concrete suggestions for optimizing CBC studies within a mobile environment.

(3:00 PM - 3:30 PM) Refreshment Break

Session 4

(3:30 PM) Can an Adaptive MaxDiff Approach Provide Better Results than Other MaxDiff Approaches?

Howard Firestone, RTI Research

We demonstrate how an Adaptive MaxDiff technique will enable researchers to: a) Report results in a more client friendly manner, b) Shorten the anchoring exercise when using Direct Binary approach, c) Easily, with fewer screens, identify preferred or less preferred items for further questioning, d) Identify respondents who provide inconsistent results during the interview.

(4:00 PM) Comparing Two Methods to Estimate Missing Maximum Difference Utilities

Paul Johnson, SSI

Kelsey White, SSI

Express MaxDiff is used to show a subset of items to each respondent. We examine two methods of estimating the utilities for the items not seen by the respondent: Hierarchical Bayesian and Expectation Maximization. We apply these techniques in an Anchored-Scaled MaxDiff to political statements made by Donald Trump.

(4:30 PM) Researcher's Paradox: A Further Look into the Impact of Large Scale Choice Exercises

Mike Serpetti, Gongos

Claire Gilbert, Gongos

Megan Peitz, Sawtooth Software, Inc.

MaxDiff is ideal for determining preference order, yet becomes problematic for large item-sets. Design techniques including Sparse and Express address these issues, but not enough large item-set research exists. This research explores and validates both methods with real respondents on a set of 100 items to determine which is better.

(5:00 PM) General Session Ends

(5:15 PM - 6:15 PM) 2016 CBC Modeling Prize Panel Discussion (Clinic)

(5:15 PM - 6:15 PM) Teaching Conjoint Analysis at the University (Clinic)

(6:00 PM - 7:30 PM) Reception and Vocal Point Concert

Session 5

(7:00 AM - 5:00 PM) Conference Registration

(7:00 AM - 8:25 AM) Breakfast

(8:30 AM) Naïve Bayes Classifiers, or How to Classify via MaxDiff without Doing MaxDiff

David W. Lyon, Aurora Market Modeling, LLC

Naïve Bayes Classifiers (NBCs) are a general method of classification or segment typing tools, like discriminant analysis or CHAID/CART. NBCs can classify using MaxDiff responses on-the-fly, based on the responses themselves and without requiring MaxDiff utilities. Among other virtues, they also deal well with partial or missing data.

(9:15 AM) Typing Tools in the Context of Choice Experiments

Lech Komendant, IQS

Basing segmentation on choice tasks has many advantages, but probably the main obstacle is creating a useful and valid typing tool for a client. We will deal with segmentations based on MaxDiff, CBC and ACBC and will introduce and compare a few typing task building schemes and basic classifiers used for this purpose.

(10:00 AM - 10:30 AM) Refreshment Break

Session 6

(10:30 AM) Full-Flavoured HB: BYO Data in the Upper Model

Jane Tang, MARU/VCR&C

Rosanna Mau, MARU/VCR&C

Mona Foss, Bootstrap Analytics

BYO exercises are an engaging way to provide respondents the education needed prior to a conjoint exercise. BYO data can also help bring out fully the individual level heterogeneity when included in the upper model of an HB estimation.

(11:15 AM) Simulating from HB Upper Level Model

Peter Kurz, TNS

Stefan Binner, bms marketing research + strategy

Max Pachali, Goethe University Frankfurt

Practitioners mainly rely on "point estimates" or draws from the lower level of the HB model for their simulations. However, theoretically the upper level model provides all information one needs and from a statistical point of view, it is the more coherent way for decision making. Is the upper level model the future of our daily work?

(12:00 PM - 1:30 PM) Lunch

Session 7

(1:30 PM) Mapping the Attribute Non-Attendance Effect

Keith Chrzan, Sawtooth Software, Inc.

Joseph White, MaritzCX

Respondents exhibit attribute non-attendance (ANA) when they ignore one or more attributes in answering a set of designed choice questions. Our paper reviews the current literature with respect to identification and measurement through modeling and inferential consequences of ANA. We extend recent findings to test the effect of pre-task attribute familiarization and experimental design modification on reducing ANA.

(2:15 PM) Using Discrete Choice to Help Individualize Customer Lifetime Value

Michael Smith, The Modellers

Michael Remington, The Modellers

Michael Drago, The Modellers

Given the recent rise in demand for creating customer segments based on customer lifetime value (CLV), we helped a Major League Baseball team to segment and type their database into CLV tiers by incorporating actual customer data alongside a discrete choice model to project the next ten years of spending behavior.

(3:00 PM - 3:30 PM) Refreshment Break

Session 8

(3:30 PM) Amazon Mechanical Turk Survey Deception: Sources, Risks, and Remedies

Kathryn Sharpe, Wharton School of the University of Pennsylvania

Oded Netzer, Columbia University

Joel Huber, Duke University

Analyzing the extent of deception on Amazon Mechanical Turk ("MTurk"), a popular crowdsourcing participant pool, we demonstrate how research results are altered by participants misrepresenting themselves. Given this evidence, we provide detailed recommendations on avoiding participant deception while still benefiting from this crowdsourcing platform.

(4:00 PM) Process Tracing: A New Tool for Modeling Physician Treatment Algorithms

Steve Bell, Bell Falla and Associates

Douglas Willson, Bell Falla and Associates

In healthcare markets, process tracing investigates how healthcare providers make diagnosis and treatment decisions. It collects information about the steps physicians take to acquire information about their patients, as well as their ultimate treatment decisions. The approach can be conveniently implemented in an online survey environment as a relatively simple modification of a traditional choice exercise.

(4:30 PM) Let's Take None Seriously

Ula Jones, TNS

Tomer Ozari, TNS

Peter Kurz, TNS

This paper continues explorations started by Karty and Yu regarding the "none" alternative. We examine four formats of the "none" alternative. We are concerned that current dual response lacks ecological validity and suffers from cognitive dissonance - exaggerating purchase intent. We hypothesize that reversed dual response could be a solution.

(5:00 PM) General Session Ends

(5:15 PM - 6:15 PM) MaxDiff Online Simulator & TURF Optimizer (Clinic)

(5:15 PM - 6:15 PM) Advanced Market Simulator Options (Clinic)

(6:00 PM - 7:30 PM) Reception

Session 9

(7:00 AM - 12:00 PM) Conference Registration

(7:00 AM - 8:25 AM) Breakfast

(8:30 AM) The Art and Science of Nested Logit: Case Studies from Modeling Many SKUs

Kevin Lattery, SKIM Group

Nested logit is helpful for analyzing correlated alternatives. We describe how nested logit works and offer a practical guide. This includes using simple diagnostics to generate an initial nested logit structure, then stepwise building and testing of the nested structures. The case studies focus on SKUs in a shelf set but the process applies to any conjoint with correlated alternatives.

(9:15 AM) Mining and Organizing User-Generated Content to Identify Attributes and Attribute Levels

Artem Timoshenko, MIT Sloan School of Management

John R. Hauser, MIT Sloan School of Management

Using machine-learning methods applied to online user-generated content, we identify, winnow, and organize informative content to produce a hierarchical structure of customer needs. The structured needs identify attributes and attribute levels that are relevant to managerial decisions and consumer choice.

(10:00 AM - 10:30 AM) Refreshment Break

Session 10

(10:30 AM) What a Difference Design Makes! MBC

Karen Buros, Radius Global Market Research

Jeremy Christman, Procter & Gamble

Menu-based choice often leads to greater flexibility (and challenges) in model specification than generally associated with most choice experiments. This paper will explore, illustrated by a case study, the effects of the modeling decisions made on the outcomes obtained. The goal is to offer the audience insight into how to think about the data structure and modeling possibilities available.

(11:05 AM) Explaining Preference Heterogeneity with Mixed Membership Modeling

Marc R. Dotson, Brigham Young University

Joachim Büschken, Catholic University of Eichstätt-Ingolstadt

Greg M. Allenby, The Ohio State University

We demonstrate a way to generate covariates for the upper level of a hierarchical Bayesian choice model that lead to an improvement in explaining part-worth heterogeneity. The covariates are uncovered by augmenting the choice model with a mixed membership model.

(11:35 AM - 11:40 AM) Best Paper Ballot Collection

(11:55 AM - 12:05 PM) Closing Remarks and Best Paper Award, Bryan Orme, Conference Moderator

(12:05 PM) Conference Adjourned

Breakout Sessions (Wednesday—Friday)

Those who have registered for the main conference sessions on September 28-30 may also attend any parallel breakout sessions.

Wednesday | Room 1

(1:30 PM – 6:00 PM) University Student Analytics Mentorship Program

Come see what it's like to work in careers in marketing analytics! We've organized an afternoon of presentations by experienced practitioners who will inspire and inform you about what you can expect. This half-day program has been organized as a free offering for university students. Student attendees will join the Main Conference sessions from 2:15-3:00 so they may also interact with the professional conference attendees. Please join us afterwards for the reception and Vocal Point concert!

Wednesday | Room 2: Analytical Focus

(1:30 PM - 3:00 PM) Introduction to StatWizards' Discrete-Choice Suite

George Boomer, StatWizards

This tutorial covers StatWizards' core set of Excel add-ins. Regardless of what software you use for estimating choice models, the Discrete-Choice Suite can cut the man-hours used to develop designs, data setup and simulators by at least half while improving results and providing advanced features available nowhere else.

DESIGN WIZARD: Starts with a simple list of attributes and levels, then quickly generates D-efficient choice designs.

DATA WIZARD: Given a simple flat file of survey results, organizes input and command files for all major choice-estimation programs.

SIMULATOR WIZARD: Beginning with output from most choice-estimation programs, generates a client-ready simulator in seconds.

(3:30 PM - 5:00 PM) Latent GOLD® 5.1 — New Features & Case Studies: How to Obtain Meaningful Segments with Ratings, Choice and MaxDiff Data

Jay Magidson, Statistical Innovations

The Latent GOLD® program has proven to be a powerful tool for Latent Class Modeling (LC) of Ratings, Rankings (including MaxDiff), and Choice Data, yielding meaningful segments and accurate predictions. In this tutorial, the basic program structure as well as various new features from the latest version of the Latent GOLD® Choice module (releases 5.0 and 5.1) are illustrated using case studies which show how to obtain meaningful LC segments in various applications. Topics include:

  • General Model Fitting Strategies with rating, choice and ranking/MaxDiff data
  • Simplified Analyses of MaxDiff Data with Scale Factors
  • Developing MaxDiff Typing Tools
  • Using the New Paired Comparisons Output to Customize Your Typing Tool
  • How to Specify Random Regret Models

Thursday | Room 1: Survey Research

(8:30 AM - 9:10 AM) Lighthouse Studio for General Surveys & CAPI

Gary Baker, Sawtooth Software, Inc.

Jon Heaton, Sawtooth Software, Inc.

Come see what Sawtooth Software's general survey development platform can do! Although Lighthouse Studio is best known for its CBC and MaxDiff components, there is very much more that you can do in Lighthouse. We'll show the general survey question types, demonstrate skip logic, constructed lists (piping), randomizations, rotations, and looping. If you've wondered if you can use Lighthouse Studio to do all your general survey work, come bring your questions and see what it can do!

(9:15 AM – 10:00 AM) CSS and Lighthouse Studio

Justin Luster, Sawtooth Software, Inc.

CSS allows you to modify the look or style of your surveys. Learning a bit of CSS can really help you to become a more powerful Lighthouse Studio user. Come learn a bit about CSS and instantly have more control over how your surveys look.

(10:30 AM – 12:00 PM) Ingenious Survey Solutions in Sawtooth Software

Saurabh Aggarwal, Knowledge Excel Services

Megan Peitz, Sawtooth Software, Inc.

Many people hear "Sawtooth Software" and think "Conjoint Analysis". But, that's not all! Over the last 12 years, Knowledge Excel has learned to tap into the power of Sawtooth Software's survey solutions. And in this session, we want to share our best programming and design secrets with you.

Join us to learn about common programming challenges and how to create ingenious solutions within the Sawtooth Software platform. From creating an interactive survey environment (virtual shelf, drag & drops, rapid MaxDiff, etc.) to easily adding multiple translations, to building new question types, there will be something for everyone. Have a specific query? Let us know and we'll tailor our presentation to your requests.

Get solutions to your most frequently asked questions and uncover the power of Sawtooth Software.

(1:30 PM - 2:15 PM) Perl and Lighthouse Studio

Lance Adamson, Sawtooth Software, Inc.

You can become a much more powerful Lighthouse Studio user if you understand some Perl programming. Perl allows you to modify and customize your surveys in powerful ways. Come learn a bit about Perl and instantly create more powerful surveys!

(2:20 PM - 3:00 PM) Lighthouse Studio Data Generator

Mike Lodder, Sawtooth Software, Inc.

What originally started as an in-house test tool is now accessible for general use. Available since SSI Web 8.2.0, the Data Generator can help you find errors in skip logic, verify quotas, uncover JavaScript issues, and discover design problems. The tool has also been used to create synthetic datasets for research on research. This session will show how to use the full capabilities of this powerful feature. We'll explain how it works, show test strategies, and teach you how to predefine and customize survey answers to explore various paths that respondents could take.

(3:30 PM - 4:20 PM) JavaScript and Lighthouse Studio

Justin Luster, Sawtooth Software, Inc.

You can become a much more powerful Lighthouse Studio user if you understand some JavaScript. JavaScript allows you to modify and customize your surveys in powerful ways. Come learn a bit of JavaScript and instantly create more powerful surveys!

(4:25 PM - 5:00 PM) An Introduction to Data Visualization with Tableau

Jeffrey Dotson, Brigham Young University

Tableau is a descriptive analytics software package that allows users to easily identify insights by transforming their data into a variety of visually appealing, interactive visualizations. Tableau is easy to use and connects to nearly any data source, be it corporate Data Warehouse, Microsoft Excel or web-based data. In this breakout session, we’ll introduce you to the basics of Tableau, including how to connect to a datasource, how to construct static and dynamic visualizations, and how to build a simple reporting dashboard.

Thursday | Room 2: Analytical Focus

(8:30 AM - 10:00 AM) New Features in SAS® PROC BCHOICE for Bayesian Choice Models

Amy Shi, SAS

Fang Chen, SAS

The BCHOICE procedure is designed for choice data with hierarchical Bayesian methods. The procedure supports three choice models: logit, nested logit, and probit. It fits models with fixed effects (aggregated), or random effects (individual-level), or any combination of fixed and random effects. PROC BCHOICE obtains samples from posterior and produces various summary statistics and convergence diagnostics. It employs optimal sampling algorithms and parallel processing to ensure efficient and fast generation of MCMC samples. For example, the Gamerman metropolis often outperforms the usual random walk metropolis in logit models.

In the 2016 release, MaxDiff and allocation choice models have been implemented in addition to the traditional binary (Yes/No) choice outcome. There are also new functionalities included for checking convergence, calculating willingness-to-pay, simulating market shares, and predicting product profitability. This presentation describes how to use this procedure for estimation, inference, simulation, and prediction through examples, and illustrates some newly added features.

(10:30 AM – 11:10 AM) Introduction to MaxDiff

Nathan Bryce, Sawtooth Software, Inc.

This presentation will review the history, methodology, and motivation for best/worst maximum difference item scaling and demonstrate how to build and analyze a MaxDiff survey using Sawtooth Software products and services. MaxDiff scaling is a discrete-choice trade-off method for measuring the importance or preference for multiple items, such as brands, product features, political platforms, advertising claims, etc. In fact, any time you are considering using a rating scale, ranking scale, or constant sum scale for multiple items, you should consider using MaxDiff because it often has greater between-item and between-respondent discrimination, as well as greater predictive accuracy than either ratings or paired comparisons.

(11:15 AM - 12:00 PM) Intro to ACBC

Aaron Hill, Sawtooth Software, Inc.

If you are new to conjoint analysis, Adaptive Choice Based Conjoint (ACBC) can seem a bit intimidating. ACBC can handle conjoint surveys with lots of attributes, complex pricing, and small samples, and create efficient models without overtaxing respondents. This session will introduce you to the ACBC methodology and explore the features and components that make this tool unique. We will demonstrate the different sections of the ACBC survey, show how to create an ACBC exercise, and discuss some "Best Practices" to make sure your next ACBC project is a success.

(1:30 PM - 2:15 PM) Discover-CBC: A Streamlined Web-Based CBC Software Platform

Justin Luster, Sawtooth Software, Inc.

Discover is a web-based application that makes conjoint analysis easier than ever before. In this session we will show you how to create, field, and analyze a choice-based conjoint survey. We will show you all of the powerful features of Discover.

(2:20 PM - 3:00 PM) Introduction to Segmentation and CCEA

Keith Chrzan, Sawtooth Software, Inc.

Segmentation analysis is a staple of the marketing researcher's toolbox. We introduce segmentation analysis using an intuitive, conceptual approach rather than a heavily mathematical one. After covering basic concepts and illustrating graphically how different segmentation approaches work, we will demonstrate how to use the CCEA (Convergent Cluster & Ensemble Analysis) program and its powerful segmentation algorithms.

(3:30 PM - 4:20 PM) New Lighthouse Choice Simulator

Brian McEwan, Sawtooth Software, Inc.

Walter Williams, Sawtooth Software, Inc.

Join us for a walkthrough of the new conjoint simulator released earlier this year. We'll show how to create a new project and many of the new features. We've revamped the main view to allow for multiple, concurrent simulations with a host of new tricks. Additional improvements include greater control over sensitivity and optimization searches, visual displays of utilities, importances, and simulation results, and better options (than SMRT) for incorporating product availability, awareness, and external effects.

(4:25 PM - 5:00 PM) Make your Conjoint Simulator Shiny

Jake Lee, Quantum Strategy Consulting

Interactive data analysis tools have made major leaps in the last few years. In particular, a package called Shiny made by RStudio is opening up new opportunities for conjoint simulators. And it is free!

This session will illustrate the benefits and challenges of using Shiny for conjoint followed by a step-by-step walkthrough of how to get started.

Friday | Room 1: Survey Research

(8:30 AM - 9:10 AM) How to Win the Panel Sample Battle

Karlan Witt, Cambia Information Group

Reaching your target respondents today has been compared to hand-to-hand combat . . . you are fighting to ensure representative respondents are reached and agree to participate while panel providers are battling eroding response rates and utilizing ever-evolving techniques to recruit participants. This session will provide updates on panel recruiting techniques, tools used by major panel providers to monitor (and ensure) quality respondents, advice on achieving goals for tracking vs. one-time studies, and a view of the impact of global legal environments on panel participation. The session will have a ton of prepared information for participants providing specific guidance on best practices when using panels that have different recruitment methods, recommendations on specific techniques to identify good vs. bad data, as well as time for individual Q&A to explore specific situations of interest.

(9:15 AM - 10:00 AM) What Is R? What Are Its Pros & Cons?

Christopher Chapman, Google

Kenneth Fairchild, Sawtooth Software, Inc.

This 45-minute session is for those who are interested in a high-level introduction to R. We'll address questions such as: what is R? Is it a statistics program or a programming language? How does one learn R? What is it good for? What are some reasons to use it, and not to use it? We'll illustrate these with brief demonstrations of Bayesian regression models and automated reporting in R.

(10:30 AM - 11:00 AM) Best Practices in Client Service for the Market Research Industry

Karlan Witt, Cambia Information Group

Whether your clients are internal to your firm or external in client organizations, we all find ourselves expected to provide some level of client service. Does your organization provide the client service training you need? So few do – and yet skills in this area may mean the next promotion, raise, or client referral. Beyond general client service skills, each of us knows well the challenges that are unique to research, but the solutions are neither uniform nor obvious. This session cuts to the chase to outline both the over-arching philosophies and detailed client service practices you can begin leveraging right away. The session will provide materials that capture these recommendations and set you up to succeed! In the time available, we will also go into individual Q&A to apply the recommendations to specific real-life situations.

(11:05 AM - 11:50 AM) Lighthouse Studio Offline Data Collection – Android and Windows

Mike Lodder, Sawtooth Software, Inc.

Lance Adamson, Sawtooth Software, Inc.

Need to collect data for a survey without an Internet connection? Need to sync survey updates to multiple locations? Come see the new Lighthouse Studio Offline Surveys for mobile survey data collection. Data collection without an Internet connection just got a whole lot easier.

Friday | Room 2: Analytical Focus

(8:30 AM - 9:10 AM) Discover-CBC: A Streamlined Web-Based CBC Software Platform

Justin Luster, Sawtooth Software, Inc.

Discover is a web-based application that makes conjoint analysis easier than ever before. In this session we will show you how to create, field, and analyze a choice-based conjoint survey. We will show you all of the powerful features of Discover.

(9:15 AM- 10:00 AM) Intro to Bayesian Estimation

Jeffrey Dotson, Brigham Young University

This breakout session will provide attendees with a brief introduction to the mechanics of Bayesian estimation and an opportunity to peek under the hood of Sawtooth Software's CBC/HB algorithm. We will discuss what can go wrong (and why), with a goal of diagnosing and resolving problems as they occur.

(10:30 AM - 11:00 AM) A Brief Introduction to Random Forests

Joseph Retzer, Market Probe

Ewa Nowakowska, GfK

Random Forest (RF) analysis has been gaining in popularity over the past few years as an accessible and robust approach to predictive modeling. RF analysis has been found to be particularly useful in various areas including big data analytics. The RF model indeed offers numerous advantages over other machine learning algorithms i.e.,:

  • Computational efficiency / speed on large data sets
  • The ability to effectively handle numerous variables of mixed type
  • Robustness to noise / outliers; multiple strategies for dealing with missing data
  • Highly accurate predictions at relatively low cost.

(11:05 AM – 11:50 AM) Modelling the Customer Decision Journey

Jeff D. Brazell, The Modellers

Christopher Karbo, The Modellers

Matt Madden, The Modellers

Most Customer Decision Journey (CDJ) studies today still tend to be qualitative or descriptive-only. They can be very helpful and provide valuable insight, but a descriptive approach should only be the first stage of insights. Our paper outlines a complete modeling approach that uses different analyses at each stage of the journey beginning with need recognition triggers, and flowing through information gathering, consideration set formation, the purchase decision, and finally post-purchase processes.

This approach allows clients to not just understand the journey, but do something to impact each stage. Early-stage models use driver analysis and clustering, while later-stage models apply discrete choice modeling and SEM customer satisfaction analysis. We reference multiple projects completed over the past two years. This framework and the models should be insightful to conference attendees, as they allow for full use of analytics toolkits to understand the consumer decision journey, an area that is becoming more and more important in industry.