Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Show level only once per choice set

Hi guys,

one of the attributes in my CBC contains four levels and each choice set contains four alternative choices. It is important that all four possible levels are shown in each choice set. None of the levels is supposed to show in two of the four alternatives per choice set. I would prefer the Balanced Overlap algorithm because the study focuses on interactions. So I would need no overlap for this one attribute but I want overlap for the other ones (one other attribute has four levels too). Under "addtional settings" in the "design settings" I can tick the box at the bottom to prohibit overlap for one attribute only when I select "Complete Enumeration" or "Shortcut".

Does anyone have an idea if what I intend to do is possible?

Thank you very much in advance!
asked Dec 15, 2017 by Leo (270 points)
edited Dec 15, 2017 by Leo

1 Answer

0 votes
Best answer
If you select the Complete Enumeration experimental design method (on the Design tab) rather than Balanced Overlap, this will happen.

This will lead to minimal overlap for all attributes, not just the first attribute.  Meaning, the design algorithm will not repeat a level within the same task unless forced to do so (e.g., there are fewer than 4 levels in the attribute, but you are showing 4 concepts per task).

Some level overlap on the other attributes may be desirable in some cases.  If so, there are power tricks that could be done to massage the design such that the first attribute had minimal overlap and other attributes had some level overlap.
answered Dec 15, 2017 by Bryan Orme Platinum Sawtooth Software, Inc. (142,515 points)
selected Dec 15, 2017 by Leo
Thank you very much for your answer! What you describe in your last paragraph is exactly what I am trying to do: No overlap in one attribute but overlap in another attribute that has four levels too. Could you tell me about these power tricks, please?
Well, if your second attribute has 3 levels (but you are showing 4 concepts on the screen), then this naturally will lead to some level overlap on the second attribute.  Is this solution going to work for you?  Or, do you have 4 or more levels on the other attributes?
I have one more attribute with 4 levels. Thats why the natural overlap doesnt work here. Do you know how to make sure that attribute gets more overlap? Thanks again.
OK, it's not hard:

First, generate your design using the default Balanced Overlap method.  I'm assuming you'll be generating 300 versions of the design, which is the default.  Make sure to Test your study using the Test Design button and save that text report somewhere so you can refer to it below.

Next, on the Design tab, click Import/Export Design...

Click the Export Design button, and a .CSV file with your design (called yourCBCexerciseName_CBC_Design.csv) is written to your study folder.  Click Save to save it there.

Browse to your study's folder and open the _CBC_Design.csv file with Excel.

The fourth column of that file is associated with your first attribute.  There are four rows for each choice task (because you are showing 4 concepts per choice task).  You will find integers of 1-4 in that column, per the original design that Balanced Overlap created.   You should insert new values of 1-4 (only one 1, one 2, one 3, and one 4) so that there is no level repeating for attribute 1 for each of those choice tasks across all 300 versions.  For example, if you want the levels for attribute 1 to always appear in the order 1, 2, 3, 4, then that would be easy to do.  If you wanted to make sure they (levels for attribute 1) were scrambled for respondents, then put them into the .CSV file always in the order 1, 2, 3, 4; but back on the Design tab, Additional Settings... button (I'm assuming you're using v9), click Concept Sorting + Randomized level order; Settings... Attribute to Sort Concepts by=Attribute1.  Save the file in Excel as a .CSV file, overwriting the previous .CSV file that Lighthouse Studio had created.

Now, back on the Design Tab, click Import/Export Design..., then click Import Design button and browse to the .CSV file you just modified.  Click Open to import that file, overwriting the original design Lighthouse Studio had created.

Now, Go back to the Design tab, and click Test Design.  Compare the counts and standard errors to the design you previously had.  You will find that the 1-way and 2-way counts balance have degraded (compared to the first report you saved based on Lighthouse Studio's default Balanced Overlap design), but only very slightly.  You should find that the standard errors have not changed much at all compared to the original design.

Test your questionnaire by clicking through the CBC questions to make sure it looks as you expect it to.
Thank you very much for your detailed answer. It worked out. I did it for 15 versions und 2000 respondents to test it.

As you said, looking at the 2-way counts they vary a bit more for the first attribute. Between the two attributes with 4 levels each they varied between 43 and 46 counts in the original design and after the adjustments they vary between 39 and 54.

The standard errors indeed barely changed. That’s awesome.

Thank you so much for the great support!