Technical Papers  Downloads  Pricing & Ordering  Your Cart

 

SS Spring 2007

2007 Customer Survey Results

During the month of April, we completed our fifth annual customer feedback and conjoint usage survey. Thank you to all that participated! Your response rate was higher than before, and this is greatly appreciated. We’re using the results to help us improve and design even better products and solutions.

We’re pleased to report that overall satisfaction with Sawtooth Software remains high, with 97% of our customers stating that the interactions they had with us were either good or excellent.

A portion of the survey focuses on tracking the use of conjoint-related methods among our users. Here were some of the main findings:

1. Among the three main flavors of conjoint, CBC continues to be used more often than ACA and traditional, full-profile conjoint (CVA). Among our users, the relative application of the methodologies as a percent of conjoint projects was as follows: CBC (78%), ACA (14%), and CVA (8%). The results for previous years are shown in the chart below:

2. Though CBC is used most often, one should not conclude that it is best for all types of applications. Researchers continue to employ multiple conjoint methods, depending on the needs of the project. Among our users’ firms conducting preference modeling in the last 12 months, 80% used CBC for projects, 42% used ACA, and 20% used CVA. About one-third of users (33%) relied on one of these three conjoint tools exclusively.

3. Among those who used CBC last year, HB estimation was used in 69% of final models.

4. MaxDiff was used by 24% of respondents’ firms in the previous year. This compares to 18% of firms in the 2006 survey.

Go back to Index

SSI Web v6.2 Released

Recently, we released SSI Web v6.2 as a free update to all v6 users. With this update, our capable programmers have greatly enhanced some nuts-and-bolts aspects of the software that affect most every SSI Web project. We have updated the following three major areas:

• Respondent Passwords
• Quota Control
• Online Administrative Module

We have paid close attention to user requests in making improvements, and we appreciate those of you who have sent suggestions or reviewed our initial planning documents.

Respondent Passwords Update:

We have dramatically improved how SSI Web manages and uses respondent passwords and “passed-in” data fields.

• We are now managing the passwords data as a single Microsoft Jet file. Working with very large passwords files is easier and quicker than before.
• The number of characters per password and the total number of password fields have been greatly increased.
• Cookies may be used instead of (or in addition to) passwords.
• Respondents may specify their own passwords.
• Data already collected on respondents may be merged into the passwords database and becomes part of the final data set for respondents who visit the survey.

Quota Control Update:

Multiple independent quotas are now supported. For example, quotas may be established for Gender and Age separately. In the previous versions of SSI Web, you needed to develop quotas based on the joint combinations of Gender and Age.

Admin Module Update:

We’ve simplified the interface (shaving many clicks) and updated the Admin Module for a more modern look. In addition, you can customize the interface to control exactly which menu items are available to given administrative passwords.

Upgrade Today!

If you are a licensed v6 user, you should have already received your notification of the free update. If you have not, please contact us. And, of course, if you are using a version prior to v6, call today for upgrade pricing and to talk to our helpful staff.

Go back to Index

Intermediate CBC Workshop
November 8-9, New York

Already know the basics for CBC modeling? Want some help getting to the next level? Come to the Intermediate CBC Workshop in New York for an interactive, hands-on training taught by the experts at Sawtooth Software.

Attendees will use the CBC/Web software to perform a variety of programming and analytical tasks to extend their knowledge. The class is structured as part lecture, part software exercises. Topics include:

• Choosing among the four questionnaire design strategies: Complete Enumeration, Shortcut, Balanced Overlap, and Random.
• Prohibitions: are they universally bad? Testing the impact of modest to severe prohibitions.
• Design Testing: Quick Test vs. more advanced test using simulated respondents. How the advanced test can help with sample size decisions.
• Conditional Pricing: customizing price ranges without the use of prohibitions.
• Partial Profile, Constant Sum, Alternative-Specific Designs.
• Advanced Topics: .CHO file modification, custom layouts for CBC questions, more details on HB analysis. Plus, a look at the latest research on cutting-edge Adaptive CBC questionnaires!

The workshop will be held in the Sheraton Tarrytown, 600 White Plains Road, Tarrytown, NY, about a 40-minute drive from La Guardia Airport. Attendees will use free demo software to perform all in-class exercises (must bring laptop PC running Windows 2000 or later). Attendees should already be familiar with the CBC/Web software. Knowledge of basic HTML strongly encouraged.

To register, visit www.sawtoothsoftware.com/workshops.shtml. The cost is $950 for the 2-day training. Continental breakfasts and two full lunches included.

Go back to Index

Don’t Miss Conference 2007
October 17-19, Santa Rosa, CA

If you attend just one market research event this year, make it the Sawtooth Software Conference. This event is held once every 18 months and nearly 200 people attended the previous meeting. This is a practical conference, where experienced consultants and academics discuss issues that matter to our everyday research practice.

The conference is not a sales forum for our software (in fact, some attendees have complained that we don’t provide more of a software sales focus!).

Two-dozen people representing both practitioners and academics will present papers on a variety of subjects related to quantitative market research, including data collection, conjoint/choice analysis, segmentation, mapping, and hierarchical Bayes. Both favorable and contrasting viewpoints to our approach and software systems will be presented.

Read what previous attendees have said about the Sawtooth Software Conference:

	"It's the only practitioner-oriented conference for marketing science, and Sawtooth always runs a first-class operation."
	"Increasingly, the Sawtooth Software conference has become the vehicle for bringing academic research into an accessible format that is open to validation and critique. This conference has done more to change the research industry and the tools available to researchers than has any other forum."
	"Very well-organized, well-planned. Worth every dollar and every minute."
	"This conference is by far the best in the industry - it is down to earth and practical."

And don’t forget the optional 4-hour tutorials on the day leading into the conference. They are a great value and consistently receive great reviews.

To view abstracts or to register for the program, visit: www.sawtoothsoftware.com.

Go back to Index

Sawtooth Software Website Updated

We recently updated our website’s look and functionality. The changes are based on user feedback, our own preferences, and a study of other successful websites. Please visit us at www.sawtoothsoftware.com to experience our new site.

Layout and Navigation:

• The new homepage is 800 pixels wide (our previous page was 750 pixels), with more white space than before. We’ve “themed” different functional areas on the website using nature photographs from our surrounding area on the Olympic Peninsula of Washington State.
• We’ve included clickable links that expand and hide supporting material (such as viewing an abstract for a technical paper).
• We’ve developed a new major link called Solutions, which emphasizes how our products can help solve common business problems.
• We’ve added a Shopping Cart. Our previous purchasing form was restrictive in the sense that it involved a confined, linear process for specifying software products to buy.
• A new interactive project advisor is available to help researchers decide which conjoint methods are appropriate for different project situations (see featured article in this newsletter).

Go back to Index

New Interactive Conjoint Advisor

We spend a great deal of time on the phone and via email coaching customers regarding which conjoint method and software they should use for different research situations. Indeed, this decision can be a source of frustration when people new to conjoint analysis are compelled to act quickly to purchase the right software for an immediate project. We’ve written white papers on the subject, we’ve posted recommendations on our Online Forum, and over the years, we’ve come to the opinion that the choice is largely driven by a few key factors: interviewing mode (i.e. computer, paper), number of attributes, maximum number of levels in the largest attribute, sample size, and whether price is an attribute. Given that conclusion, we investigated whether an interactive tool, posted on our website, could make good, real-time recommendations. To build such a tool, we naturally turned to conjoint analysis.

We started by creating a traditional full-profile conjoint plan (using our CVA software), reflecting different combinations of the five project-related factors. For each research situation (conjoint profile), we rated the likelihood that we’d recommend each of the conjoint methods. We estimated a set of “utilities” associated with the project factors for each of the conjoint methods so that we could predict the likelihood that we’d recommend each conjoint method under all possible project specifications on the five factors. We then evaluated the model, to see if we agreed with its recommendations. The first version was generally good, but not good enough. To improve the recommendations, we made some modest manual adjustments to the utilities and added some interaction effects to account for issues not captured in our original main-effects plan. This second version was definitely better.

Our new website features the resulting interactive project advisor. The interface uses five drop-down menus (based on the five factors outlined above), and produces bar charts showing a “usability” score for each of the main conjoint methods. The higher the value, the more confidence you may have in employing that technique. Some study specifications lead to more than one method receiving a high usability score; while others lead to all methods receiving relatively low recommendations.

Of course, choosing a conjoint method for a project involves other factors than just the five that compose this decision tool. Therefore, a tool such as this cannot guarantee an optimal recommendation in all situations. Even so, we think the benefits to customers just becoming acquainted with the multiple flavors of conjoint methods outweigh the drawbacks. It will point customers in the right direction so that they can more efficiently research and evaluate promising conjoint approaches.

To find the interactive tool on our website, click Solutions + Conjoint Analysis and near the bottom of the page click interactive advisor.

Go back to Index

Reporting Preferences for Attribute Levels in Conjoint Analysis

There are myriad ways to present the basic results of conjoint/choice analysis. This short article discusses the basics and offers a few examples. We’ll assume the reader already understands the essentials of estimating part-worth utilities and the basic rules for interpreting results. Please see the following two introductory articles (available on our website at www.sawtoothsoftware.com) for additional background information:

• “Interpreting Conjoint Analysis Data”
• “Analysis of Traditional Conjoint Using Excel: An Introductory Example”

Part-Worth Utility Scores

Sawtooth Software’s market simulation tool (SMRT) includes an example dataset (TV dataset) that we often refer to in our documentation. The data were collected in 1997 regarding features for then-available midrange televisions. When using Sawtooth Software’s SMRT market simulation tool, the software automatically reports average “zero-centered part-worth utilities” in the report window. The default display looks something like:

Within each attribute, the preference (utility) values sum to 0. This has been a long-time convention among academics and practitioners. It reflects the fact that the origin of the utility scale for each attribute is unknown and the utility of one level of one attribute cannot be directly compared to the utility of one level from another attribute. Given the relatively large scaling of the values, the default two decimal places of precision are really unnecessary if presenting these data to an audience.

Researchers find this display useful, but zero-centered utilities are often challenging to present to non-researchers, who often are disturbed by negative utility values. To avoid this issue, some researchers simply shift the utilities by a constant within each attribute so that the worst level of each is equal to zero. Rescaled in this way, the average part-worth utilities would look like this:

And, it’s quite common to portray the results graphically, such as the following display:

If performing segmentation analysis, multiple series could be represented on the chart, reflecting the relative utilities for different segments of the population.

While it is quite common to present such a display, the main problem is that non-technical people may start to draw inappropriate conclusions. Since the origin for each attribute is zero, it is tempting to conclude that Sony (59 points) is more than twice as preferred as RCA (27 points). Part-worth utility data are interval scaled (rather than ratio scaled) and therefore do not support such ratio comparisons. If using the original zero-centered data, it is obvious that Sony (utility of 30.03) isn’t twice as preferred as RCA (utility of -1.47). But, after shifting the data so that each attribute’s worst level has a utility of zero, we’ve invited inappropriate ratio comparisons.

Some researchers point out that preventing managers from interpreting utility scores in a ratio sense is a minor victory, overshadowed by the fact that they will tend to do it anyway and the consequences are not all that terrible. There is probably some truth to that fatalistic viewpoint, but fortunately there is yet another approach that allows ratio comparisons.

Generic Sensitivity Analysis (no specific assumed competition)

Market simulations are the favored method for communicating strategic findings to managers from conjoint analysis. They are easy to interpret, since the results are scaled from 0 to 100. And, unlike part-worth utilities, simulation results (shares of preference) are assumed to have ratio scale properties (it’s legitimate to claim that a 40 is twice as much as a 20, etc.). Market simulations offer a way to report preference scores for each level by way of sensitivity analysis.

The sensitivity analysis approach is based on the notion of how much we can improve (or make worse) a product’s overall preference by changing its attribute levels one-at-a-time, while holding all other attributes at constant base case levels. We prefer to conduct sensitivity analysis for a test product versus relevant competition (as shown in the final section of this article), but if you cannot come up with a reasonable definition of competitive products for your study, you may assume no specific competition.

Generic Sensitivity Analysis Steps:

• Use “Purchase Likelihood” model in the market simulator. (When using CBC, where you haven’t asked a purchase likelihood question, the math is equivalent to a Share of Preference simulation that assumes the product concept is being compared to a single competitor of average preference.)
• Specify an “average” base case product in the market simulator. We suggest using the “middle” level of preference for each attribute. For non-ordered attributes (like brand or color), choose the level closest to the average score for that attribute. For binary attributes (two levels, such as “channel blockout/no channel blockout”) choose level 1.5 (the interpolated value half-way between the two levels). For quantitative attributes (like price), choose the middle price. If there is an even number of levels, choose the interpolated level half-way between the two middle levels.
• Run the simulation in Sensitivity Mode. Each level of each attribute will be systematically tested, where all other levels are held at the base case levels.
• Chart the results in Excel, such as shown below.

The base case product for this sensitivity run was:

• Brand: RCA (the brand closest to “average” within this attribute)
• Screen Size: 26” screen (middle level)
• Sound: Stereo sound
• Channel Blockout: Level 1.5 (interpolated value half-way between the 2 levels)
• Picture-in-Picture: Level 1.5 (interpolated value half-way between the 2 levels)
• Price: $375 (level 2.5, the interpolated value between $350 and $400)

The first simulation within the sensitivity run computes the purchase likelihood of the JVC brand when combined with all other levels in the base case specification. We record that purchase likelihood result, then repeat this process for all levels in the study.

Sensitivity Analysis (given specific competition)

Rather than consider the relative preference for the attribute levels when compared to a generic product, we suggest you consider their strengths when associated with a specific product concept facing a given set of existing competitive products. How would this be different from the generic case? As an example, if no competitors currently offer “picture-in-picture” capability, the benefit of offering that feature is greater than if multiple competitors currently are offering that feature. Also, if the same people who like Sony also tend to desire picture-in-picture, Sony will get an incremental benefit when including this feature. Rather than use the Purchase Likelihood model, we can use the default Randomized First Choice simulation method (which has similarities to the First Choice rule and Share of Preference). The key here is to have a base case product (typically, your client’s current product specifications) along with competitive products (typically, your client’s main current competitors).

Let’s assume your client is Sony and the current base case competitive landscape is as follows:

Sony	25” Screen	Surround Sound	No Blockout	Picture-in-Picture	$400
RCA	27” Screen	Stereo Sound	No Blockout	Picture-in-Picture	$350
JVC	25” Screen	Stereo Sound	No Blockout	No Picture-in-Picture	$300

If we repeat the sensitivity analysis, this time modifying Sony’s features one-at-a-time (holding RCA and JVC constant) the results are as follows:

At the base case (Sony, 25” screen, surround sound, no channel blockout, picture-in-picture, $400), Sony captures 33% relative share of preference. The chart above shows the new share of preference if Sony were to modify its existing product to have other specific levels.

Obviously, Sony cannot change its brand to RCA or JVC, so the first attribute is irrelevant. The potential improvements to Sony’s product can be ranked:

• Add Channel Blockout (48 relative preference)
• Reduce price to $350 (45 relative preference)
• Increase screen size to 26” (39 relative preference)

Although it is unlikely that Sony would want to reduce its features and capabilities, we can also observe the loss in relative preference by including levels of inferior preference. One of those is Price. Increasing the price to $450 results in a new relative preference of 25.

Of course, you will eventually want to do more sophisticated what-if analyses than varying each attribute one-at-a-time. But, this simple approach provides a good way to summarize the relative preferences for the levels within your study. Also, we caution the reader regarding the common practice of converting utility values to monetary equivalents (also known as “willingness to pay analysis”). You can read more on this subject in “Assessing the Monetary Value of Attribute Levels with Conjoint Analysis: Warnings and Suggestions” in our Technical Papers library at www.sawtoothsoftware.com.

Go back to Index

© 2008 Sawtooth Software, Inc. All rights reserved.