(Design Tab)
|
CBC Exercise Settings
(Design Tab) |
|
| Number of Random Choice Tasks
|
| The Random Choice Tasks are the questions/scenarios generated by CBC's design algorithms for the purposes of estimating part-worth utilities. The attribute levels making up the product concepts are formulated to have good statistical properties (such as level balance and independence of attributes). It is typical to include about 8 to 20 of these types of choice tasks in your study.
|
| Number of Fixed Choice Tasks
|
| If you want to specify specific choice scenarios for CBC to ask of every respondent, these are called Fixed Choice Tasks. If you add these to your questionnaire, you must specify the attribute level combinations that define the tasks (by clicking the Fixed Task Designs... button). Fixed Tasks are often used to include scenarios of particular interest to the client, and for collecting additional choice tasks that may be used to assess internal validity. If you are not inclined to include any fixed tasks in your study, you may set the number of fixed tasks to 0.
|
| Number of Concepts per Choice Task
|
| Here you specify how many product concepts (alternatives) to include in each choice task. Although it is tempting to want to make it easier on respondents by displaying fewer concepts per task, we urge you to ask respondents to consider many product concepts per task. For example, if each of your attributes had 3 levels, it would be a good idea to show 4 or 5 product concepts on the screen, so that some levels repeat within the choice task. This not only means each choice task provides more information, but including level overlap encourages respondents to provide more thoughtful responses and leads to stronger estimation of potential interaction effects.
|
| Random Task Generation Method
|
| This control lets you select which design method CBC should employ in determining the combinations of attribute levels to be shown in the Random Choice Tasks for your survey. We encourage you to read the section on CBC Questionnaires and Design Strategy to learn more about CBC's design strategies. For standard CBC studies, we generally suggest using either the Balanced Overlap or Complete Enumeration design strategies. (Note: these methods are computationally intensive, and for very large numbers of attributes and levels may require a long time to generate a design file.)
|
| Number of Questionnaire Versions
|
| CBC automatically creates a design file that is a) uploaded to the server if using web-based data collection, b) is copied to other PCs if using CAPI-based data collection, or c) is used to generate a questionnaire file if using paper-based data collection. During the process, you are asked to specify how many versions of the questionnaire should be written to that design file. For computer-based interviewing, we generally recommend that you include at least 100 versions, as there are benefits to fielding many versions of the questionnaire. Optimally, each respondent would receive his/her own version of the questionnaire. However, there are diminishing returns and including more than a few dozen versions often is of very little to no additional practical benefit. For that reason, we limit the number of versions that can be placed in the design file to 999. If you have more than 999 respondents, this means that some respondents will by necessity receive the same designs. For all practical purposes, once there are so many unique versions of the questionnaire in the total design pool, this poses no harm.
|
|
|
| Design Seed
|
|
|
| This is the seed value used in the random number generator for the design algorithm. By using a different starting seed, a different design will result. With very many versions of the questionnaire, the overall design efficiency should be nearly identical no matter the random seed. With only a few versions of the questionnaire, using a different starting seed can have a meaningful effect on the overall efficiency.
|
|
|
|
|
|
|
|
|
| Randomize Attribute Order
|
| Randomize Attribute Order specifies whether to present the attributes in random presentation order within a concept. If this is selected, the attribute list is randomized once per respondent, and all tasks within a respondent's interview will display the attributes in that given order. This can be useful to control order effects across respondents.
|
| First Randomized Attribute
|
| The first attribute in the range of attributes to be shown in random presentation order. Specify a "1" if you wish all attributes to be randomized. If, for example, you want the first and second attributes always to appear as the first two attributes in a product concept, specify a "2."
|
| Last Randomized Attribute
|
| The last attribute in the range of attributes to be shown in random presentation order. Specify the last attribute number if you wish all attributes to be randomized. If, for example, you had a total of five attributes in your study and you wanted the fourth and fifth attributes always to appear as the last two attributes in a product concept, specify a "3."
|
|
|
| Concept sorting refers to the order in which the product concepts are displayed in a task. For example, consider a choice task with three product alternatives. The default behavior in CBC software is to randomize the order in which these product concepts are displayed. Thus, a given brand may appear in the first concept in one task, but in the last concept position in the next task. This is Full Randomization.
|
|
|
| However, researchers do not always want the levels appearing in random concept positions within the task. There are two options for researchers to control concept ordering:
|
|
|
| Sort Concepts Using a Randomized Level Order. With this option, the researcher selects a specific attribute that is used to control the concept presentation order. For example, the researcher may choose the "brand" attribute. The levels for brand are randomized once per respondent, and then the product concepts are displayed in that new order for all choice tasks for that respondent. For example, respondent #1 may see brands in the order 2, 3, 1 for the three concepts in the task (left, middle, right), for all tasks. Then, respondent #2 may see the brands in the order 3, 2, 1 for the concepts in his questionnaire.
|
|
|
| This strategy helps control order bias, without making respondents reorient themselves to new positions for, say, brands in each new task.
|
| Sort Concepts Using the Natural Level Order. With this option, the concepts are shown in the natural level order for a selected attribute, such as "brand." Brand 1 will always appear in the first concept, brand 2 in the second concept, etc.--assuming that the researcher uses a Complete Enumeration or Shortcut design strategy (which enforces minimal overlap), has no prohibitions, and shows as many concepts on the screen as there are brand levels in the study.
|
|
|
| This strategy is often used in "shelf facing" displays (Advanced Design Module only), or in cases in which the researcher doesn't want to make respondents reorient themselves to new positions for, say, brands in each new task. However, using a fixed concept order may introduce order biases.
|
|
|
|
|
| Show Levels of First Attribute no more than Once per Task
|
|
|
| Sometimes with attributes like brand, the researcher wants each brand to appear exactly once per choice task. If you have 4 brands and are showing 4 concepts per task, this happens naturally with Complete Enumeration or Shortcut strategies--unless you have included prohibitions in your study. When prohibitions are in place, it is possible for a brand to be repeated within the same choice task. To avoid this possibility, you can check this box, and the designer will ensure that levels of the first attribute appear just once per task. (Of course, this isn't feasible if you are trying to show more concepts per task than there are levels in the first attribute.)
|
|
|
| Include Two-Way Frequencies in Test Design Report
|
| Check this to include an additional analysis within the Test Design Report of joint frequencies of levels. These joint frequencies reflect the number of times each attribute level appeared with each other attribute level within the same product concept across all concepts and tasks of your design.
|
| When you click the Generate Design button, CBC automatically creates a design file using your current CBC settings that you upload to the server (for web-based data collection), or install on PCs (for CAPI-based collection), or that is used in generating questionnaire files for paper-and-pencil collection. It also produces a test design report, letting you know whether the design can efficiently estimate the main effect utilities pertinent to your study. You should pay close attention to the design report to make sure the design efficiencies for the attribute levels in your study are acceptable and that the report does not state "Design Deficient" or have any asterisks in place of the estimated standard errors.
|
| Please note that in some cases it may take a long time to generate a design if using the Complete Enumeration or Balanced Overlap methods. A typical study with few attributes and levels will face little delay (a few seconds) to generate hundreds of designs. CBC gives an estimate of the time remaining to generate the design. For testing purposes you may wish to generate few designs for the sake of speed, but once you are ready to post the survey to the Web (or use CAPI data collection) you may request that hundreds of designs be generated and saved to the design file.
|
|
|
|
|