|
||||||
Technical Papers| Home |
Solutions |
Products |
Services |
Education |
Support |
| | ||
|   SS Summer 2004Sawtooth Software Conference 2004 Program SetWe are pleased to announce the program for the 2004 Sawtooth Software Conference, to be held in San Diego, CA, October 6-8, 2004.Our conferences are well-known for their practical focus, friendly environment, accessible presentations, and excellent food. They are not sales events for our software, but forums for discussing a variety of issues related to conjoint/choice analysis, computer/web interviewing, and other quantitative methods. Typically about one-quarter of the attendees are not Sawtooth Software users. To register for the conference, or to view more details (including abstracts), please visit www.sawtoothsoftware.com. The conference registration is only $750 for the 2.5-day event ($900 after August 16). Preliminary Program Outline: Wednesday, Oct 6, 2004 Data Collection Issues
Conjoint Analysis Case Studies
MCMC Methods
These four-hour tutorials provide an opportunity for more in-depth training. The cost is $195 for one tutorial, or $320 for two tutorials (until August 16). Morning:
 Go back to IndexWorkshop for Newcomers to Conjoint AnalysisWe often hear from people very new to conjoint analysis who want to receive instruction and training. An excellent opportunity is the 1.5-day introductory class on conjoint analysis held just prior to our regular research conference in San Diego, CA, on October 4-5 (Monday and Tuesday). Tom Pilon, Ph.D. will be conducting the sessions. Tom has extensive experience in applying conjoint methods to business problems.Tom will introduce the three main methods of conjoint analysis, including traditional full-profile conjoint, Adaptive Conjoint Analysis (ACA) and Choice-Based Conjoint (CBC). The cost is $700. To register for this event, please visit www.sawtoothsoftware.com and click the Education + Conference link. Update on Relative Conjoint Analysis UsageIn April, 2004 we completed a second wave of our annual customer feedback survey. For those who participated, thank you! Aside from asking users how we might improve, we also collected some information that may be of interest to you.Customers specified what percent of projects over the last 12 months employed which specific conjoint methods.
 The results (Exhibit 1) show that the momentum continues to shift (from 2003 to 2004) in favor of CBC (Choice-Based Conjoint). Considering only the projects that used CBC, ACA or CVA, the relative use of CBC increased from 50% to 61%. (Data are weighted by the number of projects completed.) In the 1990s, ACA was the most widely used conjoint method (according to two industry surveys). Although we cannot be certain exactly when, in about the year 2000 CBC began to be used more often than ACA. One of the main reasons for this shift was the availability of HB methods (starting in the late 1990s) to estimate individual-level part worths from CBC data. Previously, only group-level estimation was available.
 Users often test multiple methods to estimate part worth models, often selecting the one that best meets some criterion, such as predictive accuracy. Exhibit 2 shows that 62% of CBC users are using HB for their final models. Using HB to analyze CBC data importantly leads to individual-level part worth estimates. ACA (Adaptive Conjoint Analysis) and CVA (Traditional Full-Profile Conjoint Analysis) have always supported individual-level estimation. Still, HB estimation can improve estimates for ACA and CVA beyond the classical OLS approaches. The use of HB in these more traditional contexts is growing, with 33% of ACA users and 25% of CVA users now relying on HB estimation. SSL (Secure Socket Layer) Available with Sawtooth Software Hosting ServicesMany of our SSI Web users have been taking advantage of our web survey hosting services. Even though our software enables users to host their own projects on either Windows or Unix servers, some users find it more convenient to ask us to host surveys for them. Our basic hosting package for a small project starts at $250.
Recently, we have added SSL (Secure Socket Layer) as a security option in our hosting
packages (base price starts at $500 for SSL). SSL encryption protects your data as it
is transmitted across the internet and may give your respondents an added measure of
confidence and trust. When respondents view your survey, the "Secure Connection" icon Testing for Significant Differences in Conjoint AnalysisWhen analyzing conjoint analysis data, we may wish to conduct statistical tests to detect significant differences. Common situations include:
Tests between Two Products
When running market simulations using individual-level utilities, a share and standard error are reported for each product in the simulation scenario. To test for a significant difference between a product’s share for two unique respondent groups (such as males vs. females), we first compute a t-statistic:
 Where the subscripts 1 and 2 refer to the respondent groups 1 and 2, and SE refers to the standard error of the shares, as reported in the market simulator. A t-value of absolute magnitude greater than 1.96 indicates a significant difference at the 95% confidence level. Testing Differences for Importances/Part Worth Utilities
In conjoint analysis, the standard errors reflect the uncertainty in the preference estimates due to sampling and the uncertainty regarding the estimated part worths. They do not completely characterize the accuracy of the model. There are other sources of inaccuracy such as bias or misspecification that are not captured in the standard error. Standard errors decrease as the sample sizes increases. Quadrupling the sample size cuts the standard error in half. User Experiences: Using SSI Web CAPI Module to Gain the Advantages of Online Interviewing OfflineEditorial Note: Sawtooth Software has an ongoing, open invitation for users to submit stories regarding innovative ways they are using our tools. The following represents one such story, submitted by Cory Schwartz, President of ConsumerQuest.The Situation A leading developer and marketer of innovative, technology-based educational software was considering 60 new titles for a limited number of openings in its 2005 product line. A screening test was needed among 300 mothers of target-age children. Using SSI Web 3.5’s CAPI research module, ConsumerQuest quantitatively assessed the relative appeal of the 60 new product candidates. The most appealing titles were identified and the combination of titles that reached the most moms was discovered. The study also provided insight into what moms liked about the titles they’d most like to buy and what they disliked about those they’d least like to buy. Methodological Challenge Our client’s business – technology-based educational software – is in an extremely competitive category where premium value is placed on design innovation and ideas. The need to determine consumer preferences for new products had to be weighed against security concerns. Exposing consumers online to the new concepts was a risky proposition because of a lack of control over who saw the concepts and over the a bility to prevent new ideas from being captured in screen shots and circulated over the Internet. Traditional central location testing via pre-recruit was desirable because 1) respondents could be screened for security from research facility databases, 2) the research could be conducted in less sensitive markets where the client’s direct competitors did not exist, and 3) respondents could be monitored so they could not “capture” the images of the new products. On the other hand, central location testing via pre-recruit interviewing posed other limitations. First, each mother had to be exposed to a pre-determined, select number of new concepts according to a randomized rotation based on her child’s age. Results were needed within ten days of when electronic versions the concepts were available, necessitating the need for interviewing small groups of moms simultaneously, which in turn would mean printing and controlling several hundred color concepts. The need for rapid turnaround also posed challenges on the back end in terms of a lengthy keypunch process and the need to de-rotate the data file. Why We Chose To Use SSI Web’s CAPI Module Fortunately, SSI Web 3.5’s CAPI module offered a solution that met all of our needs. First, our staff is well acquainted with the software, having used it to program and conduct a plethora of studies online. Custom JavaScript routines could easily be incorporated, such as one that instructed moms with more than one child to focus on a particular child by name of randomly selected age. The software also provided the flexibility to have respondents read questions while simultaneously seeing the concept on screen. Thus, respondents did not have to scroll back and forth, up and down, between concept and questions: The respondent could contemplate the question while simultaneously seeing the concept. SSI Web allowed the order of exposure to the concepts to be randomly rotated. After seeing and evaluating 25 of the 60 concepts, each mom was asked to build a library by selecting her first, second, and third choices for her child from among the 25 concepts. Each mom also had to see all 25 concepts at once via thumbnail-sized images. Happily, we found that SSI Web could allow moms the option of clicking on any thumbnail to see the concept again in its full-sized form. Writing field instructions for installing the CAPI module and activating our survey on a local server (without any access to the Internet) was straightforward. Outcome We were able to complete interviewing over a three-day weekend. And since the data was already “keypunched and de-rotated” by SSI Web, we generated data tabulations and a subsequent TURF (total unduplicated reach and frequency) analysis the following week. The technology-based educational software company successfully identified the most appealing titles and the combination of titles that would attract the most moms to its product line. User Experiences: How to Marry People, Prototypes, and Adaptive ConjointEditorial Note: Sawtooth Software has an ongoing, open invitation for users to submit stories regarding innovative ways they are using our tools. The following represents one such story, submitted by Thomas Klobucar, Director of Research, Vernon Research Group.People. Market research would be considerably easier if people were not required to make it work—particularly when testing prototypes with dozens of new characteristics and clients waiting to make critical design and pricing decisions based on the results of consumer testing. Faced with the problem of designing a study to gather quantitative data from a large sample of respondents for such a test, we mulled over a variety of methods for gathering such data. One of the most difficult issues we faced, and often face when designing studies for manufacturers of durable goods is in finding a method that allows us to show the actual item to respondents before asking them to express their opinions about the items they have seen. Traditional focus groups of eight to ten people are fine for gathering ideas, but their inherently eclectic and unscientific nature cannot yield the utility and pricing data manufacturers crave. No one wants to base business decisions about entire model lines on focus group outcomes. Another method under consideration was to use the internet to graphically display the object at hand and then use sophisticated interviewing techniques allowed by Sawtooth Software’s SSI Web ACA Module that are only possible using a computer. The problem, of course, with conventional internet interviewing is that respondents cannot see the actual item being tested. For this recent project, which called for the nuanced understanding only possible using computer-assisted interviewing while exposing participants to the actual prototypes, we hit on a design that incorporated broadly-recruited Voice of the Consumer panels of 30 participants each in a number of cities across the country and combined that design with computer assisted interviewing CAI on the internet Only the largest market research firms may own sufficient computer resources to maintain a squadron of laptop computers for CAI. Even for those that do own laptops in those numbers, the cost of maintaining each with the software necessary to conduct 30 simultaneous interviews may be prohibitive. Since we needed to conduct a computer-dependent survey design (using Sawtooth Software’s ACA), our solution was to rent laptop computers and arrange for high-speed internet access at each of our interviewing facilities—because the prototypes we needed to demonstrate were so large, our interviews ended up being conducted in hotel ballrooms and convention centers. The interviews were conducted in a single room and the interviews were conducted via the internet, at the moderator’s direction. The outcome of this project was extraordinary. Not only were we able to gather the information you can only get through ACA, we were also able to capture responses to “normal” survey questions using the same SSI Web interview, all without having to fiddle with a single interview computer beyond setting the browser’s homepage to the study URL. The data were ready for download when we returned to the office the next day without having to enter a single response manually. We had our data, and analysis could begin immediately. The results? Using Peter Williams’ suggestions for including holdout tasks in the study* to ensure pricing data were correctly estimated (and weighting the data accordingly), we were able to duplicate quite precisely the current market and build a series of models that showed what consumers wanted and how much they were willing to pay for those options. The non-ACA data we also captured using SSI Web integrated well, allowing us to look at market segments very closely and determine the attributes that appealed most to different populations. Sawtooth Software’s SSI Web and ACA module and a little creative logistics management allowed us to build a research model that got the people, the products, and the computers in the same place at the same time without major headaches. We were left with a high degree of confidence in the answers we gave to our clients, who in turn walked away secure in the knowledge that we had gotten things right. *Williams, Peter (2000). Calibrating Price in ACA: The ACA Price Effect and How to Manage It. Available in Technical Papers Library at: sawtoothsoftware.com CPM Perceptual Mapping Software Upgrade ComingPerceptual maps help managers understand how products are perceived with respect to their strengths, weaknesses, and similarities relative to one another. By way of a two-dimensional picture, perceptual maps can summarize a great deal of information about brands and attributes. This time-proven tool helps managers picture and develop effective strategies for product positioning.Sawtooth Software first offered perceptual mapping in the 1980s with the APM (Adaptive Perceptual Mapping) System. APM used multiple discriminant analysis, and had the additional benefit that respondents could rate a subset of the brands (e.g. familiar brands) on a subset of the attributes (e.g. most important attributes). We released a new perceptual mapping product in the 1990s that in addition to the standard discriminant-based maps could produce “composite” maps that leveraged both perceptual and preference information. The preference information could directly come from either paired comparisons of brands/products or from conjoint part worths. With preference information in place, the composite mapping technique creates a mapping space that discriminates among the products in terms of perceptions and preferences. Therefore, an attribute that discriminates strongly among brands in terms of perceptions but is not a driver of preference will have little effect on the map. This makes mapping more actionable, as the dimensions are better tied to preferences rather than just perceptions. Areas of the map representing greater demand can be shaded with darker colors (called a density of demand display), giving managers a better idea of desirable directions for repositioning.
 An upgrade for CPM (Composite Product Mapping) will soon be available. This new system is a fully Windows-based program that offers more plotting options, much better graphics rendering (see the graphic on this page as an example) and better data handling capabilities than the previous version of CPM. The attribute vectors are much cleaner (sharper) than before, including the ability to place arrow-heads at the ends of the vectors. The density of demand contours have greater resolution, and there are more options for formatting the individual elements of the maps. Data for CPM can come from just about any source, whether self-administered computer-based interviewing, CATI, or paper-and-pencil. CPM can read data from standard text-only, delimited formats. CPM v2 is based on Microsoft’s .NET platform, which requires the .NET framework. This may already exist on your system. If it does not, you can download it for free from Microsoft’s site. The software is undergoing beta testing, so it is being put through its paces by existing APM and CPM users. The response so far has been positive. If you have any questions regarding this new upgrade, please call us at 360/681-2300. If you have fundamental questions about CPM or the methodology, please download the CPM Technical Paper from our Technical Papers library at sawtoothsoftware.com.
|
|
appears in their browser window.
If you would like more information about our hosting services, please contact Aaron Hill, at 360/681-2300. We may also be able to give you suggestions if you want to implement SSL within your own survey hosting environment.|
About Us | Contact Us | Privacy | FAQ's | Forum |
|
© 2008 Sawtooth Software, Inc. All rights reserved. |