Sawtooth Software: The Survey Software of Choice

Internet Software Available for ACA and General Surveys

We are pleased to announce the completion of two new integrated software systems for collecting market research data over the Internet: ACA/Web (Adaptive Conjoint Analysis for the Web) and CiW (a general survey administration tool for the Web). These two programs are components within a developing suite of Web-based software products that we are calling SSI Web.

These products have undergone one of the most intensive beta test programs ever completed for Sawtooth Software products. A few select firms began testing the software for commercial applications back in November, 1999. Since then, over 50 organizations have used the software. The demand for those desiring to become beta test sites far exceeded our expectations. It is a clear indication of how popular Web-based interviewing has become. We estimate that over 75 projects have been completed for over 100,000 separate interviews using CiW and ACA/Web since last November.

Tom Pilon (TomPilon.com), one of the beta testers, commented to us:

Very few other web interviewing packages have capabilities such as advanced skip logic and randomization (random number generator, randomize questions within a page, randomize responses within a question) which are critical to serious researchers. Most other web interviewing packages that I have looked at are toys and not appropriate for serious researchers.

Tom Pilon (TomPilon.com)

The Big Picture

The Internet is a fast-growing environment involving many diverse technologies. Internet servers which store the pages and files accessed over the Web come in different flavors, such as Unix, Apache, Windows NT, and Linux. People access the Web using different browser technologies, the most common being Internet Explorer and Netscape. We decided to develop software that could be run on just about any server and would display surveys that could be viewed consistently by just about any browser.

The process of creating a survey for the Web using our software is as follows:

  1. The researcher uses a point-and-click Windows-based tool operating on his/her local PC to develop a questionnaire. No programming is involved, and the resulting Web pages are viewed using a Web browser running on the researcher's local PC. Standard survey questions (like numerics, open ends, select questions) and conjoint questions (ACA) can be included. To this point, no part of the survey development has involved the Internet.
  2. When the researcher is pleased with the look of the questionnaire, he/she uploads (copies) the questionnaire file and the programs that run the questionnaire (Perl scripts) to an Internet server. This is done using a FTP (File Transfer Protocol) software, which also lets you specify the needed permissions on each file and directory on your server. (Note: In our users' experience, Microsoft FrontPage is not a good choice when doing this process, since it doesn't permit the full control that is needed.) The Internet server can be at the researcher's company, or can be at the ISP (Internet Service Provider).
  3. Once the questionnaire file and programs are copied to the Internet server, the researcher tests the questionnaire by accessing it with his/her browser. If changes need to be made, the researcher makes the changes using the Windows-based interface operating locally on his/her computer, and then uploads the modified questionnaire to the Web server.
  4. After the researcher has finished modifying the survey, respondents are invited to access and take the survey. There are many ways to do this, including email invitations, a clickable link from a corporate site, clickable banner ads, by phone interview, or even snail mail. Probably the most popular method among our beta testers has been email invitations. Email messages are sent to respondents, with a hyperlink (Web address) embedded in the email message. (You should use your own bulk email software to generate these email invitations.)

To: W. E. Coyote
From: Acme Company
Subject: Tell us what you think

Dear Mr. Coyote,

Thank you for your recent purchase of the X-13 rocket powered skates. We hope you are finding hours of enjoyment with this innovative new product. Will you please fill out a five-minute questionnaire regarding your experience with the X-13? Please click the following link: http://www.acme/survey/cgi-bin/ciwweb.pl?resp_name=1455&password=tr8h3 to provide your feedback.

Respondents click on the hyperlink and their browsers access the first page of the survey over the Web. (Note also that we included the respondent number and password information within the hyperlink, so that the respondent does not have to type the passwords manually.)

As respondents complete the survey, their data are stored in ASCII format on the Web server. The researcher can view the results (simple tabulations of select survey questions) in real-time. The data can also be downloaded to the researcher's hard drive for further analysis.

Password Protection

Posting your questionnaire on the Internet gives anyone connected to the Internet potential access to your survey. The imposter/repeater problem is one of the most oft-cited criticisms of on-line survey research. You can control the imposter/repeater problem by assigning passwords and quotas with the Internet Passwords Module.

Respondents must specify a User Name and/or a Password to start your survey. You can assign different password combinations to different sample sources to track the source of sample or limit the number of responses from different sample sources.

Assigning a quota of 1 completed survey per password lets respondents quit a survey part-way through and restart the interview at a later time at the point they left off. You can assign tens of thousands of passwords if needed. When using so many passwords, it is important that the software be able to rapidly search the list of passwords to verify that a respondent is qualified to take the survey. We have designed our software to process respondent passwords rapidly, to reduce the wait time between submitting a password and receiving the next page in the survey.

General Survey Capabilities

The CiW system includes many (but not all) of the capabilities of our popular Ci3 system.

  • Numeric question types (with response verification to accept values within a range)
  • Select question types (select one, all that apply, and drop-down combo boxes)
  • Open-end question type (with response verification to require a certain number of characters)
  • At user's control, all questions can require responses or be left blank
  • Ability to show graphics
  • Skip patterns, question rotation, and randomization of items within questions
  • Random number generator can be used to randomly assign respondents into groups that receive customized sets of questions.
  • Questions can be organized onto different pages
  • Previous responses can be displayed later in the questionnaire
  • Respondents can resume questionnaires that they prematurely terminate

On-Line Data Management Module

Having your questionnaire on the Internet is not only convenient for respondents, it is convenient for you (or your client) for checking the progress of the survey.

You can calculate marginals, download data, delete incomplete records, or change the quota controls for passwords to your survey. There are two classes of passwords you can assign to control access to the Data Management Module: Read and Read/Write.

You may wish to give your client Read Only capability, which would let the user tabulate marginals and view (but not download) the data file. Read/Write access lets you perform additional functions, such as download data, or adjust password quotas.

Example Internet Survey

We invite you to view an example ACA/Web and CiW survey by visiting http://www.sawtoothsoftware.com/samplesurveys.

Technical Support

We provide free technical support for the software, with the exception of implementing the server-side aspects of your survey. Many Internet servers follow general conventions. But because each server can be configured differently, we require that the support for using that server come from your ISP or from your internal IS support staff.

Given the complexities of Internet servers, bandwidth issues, security, and bulk emailing, SSI Web is a more complicated software system to use than our typical market research software (e.g. CBC, ACA, Ci3). You should make sure that you have the appopriate resources available at your company to manage these challenging issues.

Prices

CiW comes in different sizes, depending on the number of questions in the survey. The prices are as follows:

CiW/50 questions    $3,000
CiW/100 questions   $5,000
CiW/250 questions   $7,000
CiW/500 questions   $9,000

ACA/Web includes CiW capabilities for 10 questions. ACA/Web is sold depending on the number of attributes in your study.

ACA/Web 10 attributes    $3,000
ACA/Web 30 attributes    $6,000

There is a small discount for current ACA users (v4.5).

Future Directions

We expect that Web-based market research will continue to grow in popularity. Sawtooth Software is committed to an aggressive program of enhancing CiW to manage more complex questionnaires, more in-depth realtime reporting, and to offer a greater variety of question types and formatting flexibility than version 1 of CiW supports. Future versions of SSI Web will accommodate not only ACA and CVA conjoint questionnaires, but CBC (Choice-Based Conjoint) and perhaps even CPM (perceptual mapping) surveys as well.