
	Technical Papers Pricing & Ordering Your Cart

Sawtooth Software » Education » Sawtooth Solutions

Education

Support

SS Fall 1996

The CBC Paper-And-Pencil Module (SS Fall 96)

Over the past year we have spoken to many researchers wanting to use CBC for paper-and-pencil studies. Unfortunately, using CBC with paper and pencil has been labor-intensive and methodologically limiting.
In response, we have written an add-on to our CBC System called "The CBC Paper-And-Pencil Module" which provides access to the full range of CBC's capabilities without having to use computers to run the questionnaire and collect the data.
Fixed paper-and-pencil designs employing only a few versions usually allow for estimation of main-effects, but not interactions. This has been particularly limiting for pricing studies where it is important to measure price sensitivity for each brand. The Paper Module permits up to 60 versions of the questionnaire and up to 20 tasks per version, providing more than enough product combinations to measure main-effects and two-way interactions.
The CBC Paper-And-Pencil Module costs $1,000 for a Category I license. As with all our products, it comes with a 60-day money-back guarantee. Please contact us at 360/681-2300 if you have questions or would like us to fax you an order form.
Go back to Index

CBC Latent Class Segmentation Module (SS Fall 96)

Because choices seldom provide enough information to estimate utilities for each individual, CBC estimates average utilities for groups of respondents. However, if there are distinct segments, a model that recognizes them can produce more accurate results. Although one can conduct separate analyses for subgroups differing by demography or product-usage, it has been difficult to do segmentation based on choices themselves.
The CBC Latent Class Segmentation Module uses choice data for the simultaneous development of segments and estimation of utilities. For example, one segment might be composed of price-sensitive shoppers, and another might be composed of those who usually select premium brands. Each respondent has some probability of belonging to each segment, but can be classified into the most likely segment for subsequent tabulation.
This module has other features that may be of interest to CBC users: it permits weighting of respondents, and you can constrain utilities to be monotonic, such as for levels of price or quality. Choosing the number of segments is handled by specifying a range to investigate, such as from 1 through 10 segments. Statistics are provided for assessing goodness of fit for each solution, and alternative solutions are tabulated with one another.
The CBC Latent Class Segmentation Module will be available in November. It costs $1,000. As with all our products, it comes with a 60-day money-back guarantee.
Go back to Index

CVA v2.0 Now Available (SS Fall 96)

We are pleased to announce the release of CVA v2.0. CVA is Sawtooth Software's full-profile conjoint package, and is particularly useful for conducting paper-and-pencil conjoint studies.
Version 2 includes an entirely new interface for defining attributes and levels, managing multiple studies, calculating utilities, and running simulations. The new interface means the end of creating ASCII files to input attributes, file layouts and control parameters. The interface features a menu bar with pull-down menus, dialog boxes and on-screen help. Those familiar with ACA 4.0 will recognize that we've borrowed generously from ACA's easy-to-use interface. If you fall into this category, we expect this sense of deja vu will make your transition to CVA v2.0 even easier.
In addition to improving the interface, we have enhanced CVA's questionnaire designer to create more efficient designs and to create either single-profile or pair-wise comparison interviews automatically. Instead of developing a single design and stopping as in Version 1, Version 2 reserves a larger pool of potential product profiles to draw upon. The profile which contributes least to the efficiency of the design is discarded, and the process repeated, until a smaller set of highly efficient profiles remains. More efficient designs lead to more precise utility measurements.
For current CVA users, the price for upgrading to Version 2 is $500. For ACA users who do not own CVA yet, the price is $1,000. Otherwise, CVA v2.0 costs $1,500. These prices are for Category I licenses.
Go back to Index

Ci3 v2.0 Windows Interviews to Run under Windows 3.x (SS Fall 96)

Ci3 v2.0 Windows interviews will soon work under Windows 3.x without relying on Win32s. This enhancement will be particularly valuable to researchers who use Disk-By-Mail surveys. All of Ci3 v2.0's advanced capabilities, including font control, graphics and sound will be available within the 3.x interview. (The Ci3 v2.0 System will still require either Windows 95 or Windows NT for questionnaire authoring and analysis.)
This will be a free enhancement for Ci3 v2.0 users. As soon as the 3.x interviewing module is ready, we will send a general update.
Go back to Index

Getting The Most From CBC--Part 1 (SS Fall 96)

When designing CBC, we tried to make many of its features automatic, so users could accomplish their goals by accepting built-in defaults. However, CBC users have had questions about several issues, and these are our recommendations about them.
Using Prohibitions: CBC lets you specify that certain combinations of levels should not appear in the questionnaire, such as a luxury product at an economy price. Prohibitions should be used only rarely, to avoid showing hypothetical products that would be seen as completely absurd. There are several reasons to avoid using prohibitions:
(1) CBC lets you specify unique price ranges for different combinations of levels, such as brand and package size, so prohibitions are not required to ensure reasonable prices.
(2) Respondents are usually able to deal with product concepts more "extreme" than currently available in the market, and permitting such combinations not only increases the efficiency of estimation of their utilities, but also let you estimate "what might be" as well as "what is."
(3) Too many prohibitions, or a particular pattern of them, may make it impossible to create a good design, and may undermine your ability to analyze the data. Although all prohibitions decrease statistical efficiency, CBC provides a way to test a design before you go to the field to ensure that prohibitions will not have catastrophic effects. If you do use prohibitions, test your design!
Numbers of attributes and levels: CBC permits a maximum of 6 attributes, and a maximum of 9 levels per attribute. But, as with other conjoint methods, you get the best results if you keep things simple. Don't include unnecessary attributes or levels just because there's room for them. Most CBC studies use only three or 4 attributes. You may have to use as many as nine levels for "categorical" attributes like Brand or Package Type, but there's seldom any reason to have more than 5 levels for quantitative attributes like Price. It's usually better to have more data at each price point than to have thinner measurements at more price points, particularly if you're interested in interactions. The interaction between two 9-level attributes involves 64 logit parameters, but the interaction between two 5-level attributes requires involves only 16.
Determining sample size: In a paper presented at the 1997 ART forum (which you can download from our home page) we showed that the statistical gain from increasing the number of choice tasks per respondent was similar to the gain from a proportional increase in the number of respondents. Therefore, one way to determine sample size is as follows:
1. Count the number of "cells" in the largest interaction you want to measure. For example, with the interaction of two 9-level attributes, there are 81 cells. One way to think about CBC analyses is that you want to estimate the proportion of times that concepts defining each cell are chosen.
2. Determine how many concepts will be shown altogether, which is the number of respondents times the number of choice tasks per respondent times the number of concepts per task.
3. The approximate number of concept occurrences per cell will be equal to the total number of concepts shown, divided by the number of cells. Call this quotient n.
4. Ignoring choices of "None," the average probability of a concept being chosen is 1 over the number of concepts per task. Call this p.
5. The standard error of a proportion is sqrt [p (1-p) / n] .
For example, consider a "typical" CBC study, with 300 respondents, each with 10 choice tasks, and with 5 concepts in each task. The total number of concepts shown will be 15,000. If the largest interaction we want to measure is 5 x 5, the number of concept occurrences per cell will be 15,000 / 25 = 600. The average probability of a concept's being chosen will be 1/5 = .2, so the average standard error will be sqrt [.2 * .8 / 600] = .016, and the 95% confidence interval will be about +/- 1.96 * .016, or +/- .03.
If we were interested in a 9x9 interaction, the number of cells would be 81 rather than 25, and the number of respondents required for equivalent precision would be about three times as large. If we were only interested in main effects, the maximum number of cells might be 5 rather than 25, and a sample size only a fifth as large could yield equivalent precision for those estimates.
The "typical" CBC study is like our example, in which the total number of concepts divided by the number of cells of interest is about 600. If separate estimates are desired for several segments, then adequate samples should be included for each of them. Finally, remember that you can get about the same increase in precision from proportional increases in the number respondents, or the number of tasks per respondent, for questionnaires with up to 20 tasks.
Reporting results: Counts vs. Simulations: If you don't use prohibitions, CBC produces designs in which each attribute varies independently of the others. This means that you can measure the effect of an attribute simply by observing the proportion of times concepts are chosen when they have each level. Similarly, two-way interactions can be evaluated by seeing how often concepts with each pair of levels are chosen. We call this the "counting" approach. You can also do logit analyses to estimate average utilities, which you can then use in simulations. If you have not used prohibitions, these two approaches will produce similar results.
For simple questions, such as obtaining price sensitivity curves for specific brands, the counting approach is often adequate. Since no complicated analysis is required, results are easy to communicate to others who are not market researchers. However, other objectives may require logit analysis and simulations. For example, if you want to simulate a particular product's showing in a competitive market, the logit/simulation approach is more appropriate.
Still to come: In subsequent issues we'll have some suggestions about use of the "None" option and how to calibrate CBC results to external market share data. We'll also discuss the "IIA Problem" (Independence from Irrelevant Alternatives), when to use CBC's Correction for Product Similarity, and we'll have some suggestions for dealing with respondent heterogeneity.
Go back to Index

Which Conjoint Method Should I Use? (SS Fall 96)

Conjoint analysis has become one of the most widely used quantitative tools in marketing research. When used properly, it provides reliable and useful results. We hope you have had successful experiences with conjoint analysis. If you have been involved with many conjoint studies, you've probably discovered that each is unique. Just as the golfer doesn't rely on a single club, the conjoint researcher should weigh each research situation and pick the right combination of tools.
Conjoint analysis comes in a variety of forms. Sawtooth Software offers three different conjoint software packages: Adaptive Conjoint Analysis (ACA), Choice-based Conjoint (CBC) and Conjoint Value Analysis (CVA). It makes little sense to argue which of these is the overall best approach. We have designed each package to bring unique advantages to different research situations.
We discuss each of our conjoint packages below, give guidelines for deciding which to use, and provide a grid to summarize the information.
Adaptive Conjoint Analysis (ACA)
The first version of ACA was released in 1985 and was Sawtooth Software's first conjoint product. Since then, ACA has been reported to be the most popular conjoint software tool in Europe, and we believe it shares the same status elsewhere. ACA is user-friendly for the analyst and respondent alike. But ACA is not the best approach for every situation.
ACA's main advantage is its ability to measure more attributes than is possible with traditional full-profile conjoint. In ACA, respondents do not evaluate all attributes at the same time, which helps solve the problem of "information overload" that plagues many full-profile studies. We believe respondents cannot effectively process more than about 6 attributes at a time in full-profile context. ACA can include up to 30 attributes, although typical ACA projects involve about 8 to 15 attributes. Even with six or fewer attributes, ACA has been demonstrated to provide results at least as good as the full-profile approach.
In terms of restrictions and limitations, the foremost is that ACA must be computer-administered. The interview adapts to respondents' previous answers, which cannot be done via paper-and-pencil. Like most traditional conjoint approaches, ACA is a main-effects model. This means that utilities for attributes are measured in an "all else equal" context, without the inclusion of attribute interactions. This can be limiting for pricing studies where it is frequently important to estimate price sensitivity for each brand in the study. ACA also exhibits another limitation with respect to pricing studies: when price is included as just one of many variables, its importance is likely to be underestimated.
Choice-Based Conjoint (CBC)
One of the most exciting recent innovations in conjoint research is the introduction of Choice-Based Conjoint. CBC interviews closely mimic the purchase process. Instead of rating or ranking product concepts, respondents are shown a set of products on the screen (in full-profiles) and asked to indicate which one they would purchase. As in the real world, respondents can decline to purchase in a CBC interview by choosing "None." If the aim of conjoint research is to predict product or service choices, it is natural to use data resulting from choices.
CBC can measure up to six attributes with nine levels each. CBC can be administered by PC or via paper-and-pencil using the CBC Paper-And-Pencil Module. In contrast to either ACA or CVA, CBC results are analyzed at the aggregate, or group level. Results are analyzed in aggregate since choices provide less statistical information per respondent than traditional approaches. Not surprisingly, CBC projects require larger sample sizes to achieve the same precision of estimates as traditional conjoint. For sample size decisions with CBC, see an accompanying article, "Getting the Most out of CBC."
Academics and practitioners alike have argued that consumers have unique preferences and idiosyncracies, and that aggregate-level models which assume an average buyer cannot be as accurate as individual-level models. It is true that desirable qualities are lost in aggregate models. However, aggregate models have an important advantage. By analyzing group-level data, more information can be leveraged to measure two-way interactions. Interactions can become critical in many applications, such as pricing research, where it is desirable to fit separate price functions for each brand. For most commercial applications, individual respondents cannot provide enough information with even ratings- or sorting-based approaches to measure interactions at the individual level.
Recent advances have been demonstrated for calculating individual-level utilities from choice data. To date, many of these new methods require enormous amounts of computing time and are not accessible to most researchers. Other methods use standard approaches such as Multinomial Logit, but can only support limited main-effects designs. At Sawtooth Software, we are working on ways to improve CBC in light of these advances. Methods for segmenting respondents into homogenous groups based on choice data have shown great promise. Choice models for segments of like-individuals which are aggregated to represent the market generally out-perform a single, aggregate model. We've tested this approach using a commercial CBC data set and significantly improved predictability of hold-out concepts versus the single aggregate-level model. A Latent Class segmentation method is included as an add-on to CBC and will be available starting in November. For more information, see the article, "CBC Latent Class Segmentation Module."
Conjoint Value Analysis (CVA)
CVA brings full-profile conjoint to the arsenal of Sawtooth Software's conjoint tools. Full-profile conjoint has been a mainstay of the conjoint community for decades now. We believe the full-profile approach is useful for measuring up to six attributes. CVA is designed for paper-and-pencil studies, whereas ACA must be administered via computer. CVA can also be used for computerized interviews when combined with the Ci3 System for Computer Interviewing.
CVA calculates a set of utilities for each individual, using traditional full-profile card-sort (either ratings or ranked), pair-wise ratings, or trade-off matrices. Up to 10 attributes with 15 levels can be measured, as long as the total does not exceed 100 parameters.
Through the use of compound attributes, CVA can measure interactions between attributes such as brand and price. Compound attributes are created by including all combinations of levels. For example, two attributes each with two levels can be combined into a single four-level attribute. However, interactions can only be measured in a limited sense in CVA. Interactions between attributes with more than 2 or 3 levels each are better measured using CBC.
In addition to traditional full-profile designs, CVA offers a unique way for measuring price sensitivities for individual features. This can be useful for research which seeks to determine price sensitivity of individually-priced components of a larger product or service bundle.
So Which Should I Use?
If you need to study many attributes, ACA is the preferred approach. If you need to include attribute interactions in your models, you should probably use CBC. In many cases, survey populations don't have access to PCS, and it may be too expensive to bring PCS to them, or vice-versa. For pricing research which involves measuring interactions, CBC is preferred. If your study must be administered paper-and-pencil, consider using CVA or CBC with its paper-and-pencil module.
Many researchers include more than one conjoint method in their surveys. For example, some studies need to measure a dozen or more attributes, and also require brand-specific demand curves. ACA followed by CBC can solve this problem within a single questionnaire. ACA would include all the attributes, while brand, price, and perhaps another key performance variable would be studied using CBC. ACA provides the product design and feature importance model, while CBC provides price sensitivity estimates for each brand and a powerful pricing simulator.
Many of the criteria that govern choice of method are summarized in the table below. We have placed Xs under the product(s) that satisfy each criterion.

ACA CBC CVA

Six or fewer attributes X X X

More than six attributes X - X(a)

More than nine levels per attribute - - X

Computerized questionnaire X X X(b)

Paper questionnaire - X(c) X

Interactions - X -

Small sample size X - X

Individual-level utilities X - X

(a) CVA can measure up to 10 attributes, but for most conjoint projects, respondents may not be able to process more than 6 attributes effectively.
(b) When used with Ci3.
(c) When used with the CBC Paper-And-Pencil Module.