CBC Paper-and-Pencil Studies
Top  Previous  Next

SSI Web lets you run your CBC studies via Web, over PCs (CAPI installation), or as paper-and-pencil questionnaires (sometimes a "paper-based questionnaire" is one that you are implementing via a 3rd party web-interviewing platform). The process of running paper-based CBC studies is very similar to computer-based studies. You should set up your attributes, levels, and compose your choice tasks in the same manner as if you were going to run a computerized study. However, you should probably choose plain black (text) on white (background) for questionnaire colors, unless you plan to print the questionnaires in color.

For CBC, even though you will not have a typical "randomized" design with paper-based questionnaires (with, say, 500 questionnaire versions), you should still use CBC's random choice tasks. Rather than having so many possible versions of the questionnaire (perhaps a unique version for each respondent), you will create just a few versions (design blocks) of the questionnaire and assign respondents (randomly) to take one of the different questionnaire versions. It is not necessary that an equal number of respondents complete each block of the questionnaire to obtain satisfactory utility estimates.

We generally recommend that you include enough versions of the questionnaire so that the number of random choice tasks times the number of questionnaire versions is greater than or equal to 80 (assuming no prohibitions, and typical attribute level specifications). However, we urge you to spend extra time looking at the design efficiency report that was created when you generated your paper-and-pencil design plan. With very few questionnaire versions, the design seed can have an effect on design efficiency. Also, you may want to take further steps to generate "dummy" response data (holding the number of respondents constant) under different design plans (e.g. three versions vs. four versions, etc.). Please see more information on testing your design in this documentation.


Saving the Questionnaire

When you click the Field | Create Paper & Pencil Interviews… button, each version of your questionnaire is saved to a separate file within the Paper-and-Pencil folder within your study directory. The files are named STUDYNAME_EXERCISENAME_V#.htm, where STUDYNAME is the four-character or less study name for your project, and # is the version number of the questionnaire (always beginning with version #1). These files are in HTML format and may be opened with most any recent word processing software.

You should do any additional formatting you like to make the questionnaire appear as you wish prior to printing.

Note: It is helpful to print the version# and task# with each task in a paper-and-pencil study. This gives an additional reference while performing data entry, helping ensure that you do not introduce errors when specifying the version numbers seen by respondents. An easy way to automatically label your choice tasks is to add a function [%CBCVersion( )%] in the header or footer of the choice task. When you do this, each task includes a label such as D - 1, T - CBCRAN1, meaning "Design #1, Task CBCRAN1."


Fielding the Questionnaire

Make sure when fielding the questionnaire that you are able to match the different questionnaire versions correctly with respondent answers. You should also try to randomly distribute the questionnaire versions among respondents, so that roughly equal numbers of respondents complete each version. It is not necessary that an equal number of respondents complete each block of the questionnaire to obtain satisfactory utility estimates. Even somewhat large deviations from equal distribution will not affect the quality of your results very much.


Data Entry

After you have collected the data, you prepare a .csv file containing the respondent answers. This file follows a required .csv (comma separate values) format, with specific labels in the first row of the file. To see that format, prepare a template for yourself by clicking Field | Create Accumulated Data Template File.... A file named Accumulated Data.csv is saved to your study folder. Use this template to arrange your data, and make sure to use the prescribed header row within your data file.  
 
The layout is:  
 
Column 1: Respondent#  
Column 2: Version#  
Column 3: CBC answer #1, etc.  
 
Any missing data are coded as blank (meaning, if you viewed the data file using Excel, the cell would be empty; if you viewed the file with a text editor, the missing value would look like two consecutive commas with no text in between).  
 
Constant-Sum Data: multiple responses are provided for each task, one per concept. During analysis, responses are normalized to 100% within task.  
 
Dual-Response None: the Dual-response None answer (coded as 1 or 2) is indicated after each choice task response.  
 
Accumulate Respondent Answers:

Click Field | Accumulate Paper & Pencil Data... and browse to the .csv file containing your respondent data.