Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to randomize items in maxdiff section?

Hi,

I have  a problem regarding the items in the maxdiff section. 5 items out of 20 items overall are displayed in 20 tables. The problem is that they are not shown in random order or based on participants previous answers. Do you have any ideas how to solve this problem?

Many thanks,

Maike
asked Feb 2 by Maike
Could you give us a bit more information?  I think you are saying that in each of 20 tables (we call these MaxDiff "sets") 5 items are shown, and that there are 20 items overall in your MaxDiff exercise.  So, each item is shown 5 times to each respondent across the 20 sets.   If you are using our MaxDiff software, then by default 300 different versions of the sets will be shown (assuming you have 300 respondents or more in your study).  So, there are 300 unique versions (randomizations) that will be used.  The first respondent is assigned to the first version of the 20 sets, the second respondent is assigned the next, etc.

So, you can see that across the 20 versions, the items appear in a somewhat random order, but in a controlled fashion such that each item appears and equal number of item and each item appears in sets with other items an equal number of times.  

Now, to your last point, it is possible to show items to respondents based on previous answers, so long as the number of items to show each respondent is a constant (such as always 16 out of the 20 items).  This is an advanced feature of our MaxDiff software and it is described in the documentation within Lighthouse Studio under Help + MaxDiff Help + Creating a MaxDiff Experiment + Developing a List of Items.  Scroll down in that section until you see the paragraph titled, "Using Constructed Lists in MaxDiff Experiments".

1 Answer

0 votes
Maike,

How many versions (blocks) of the MaxDiff experiment do you have?  We recommend having many blocks in which case this issue goes away, the order of items varying from one block to the rest.  

If you have just one block or very few blocks, then our designer tries to balance the order of items as best as possible so as to prevent position bias.  

I believe you only have a problem if (1) you've imported your own design, (2) it has one or only a very few versions and (3) your design doesn't control for position effects.  If that's the case I don't have a good solution for you, but perhaps someone else can offer one.
answered Feb 2 by Keith Chrzan Platinum Sawtooth Software, Inc. (64,350 points)
Dear Keith,

thanks for your help but unfortunately I do not quite understand what you mean. I have created one list including 20 items. As recommended 5 items are shown in each table. Overall there are 20 tables or maxdiff exercises in the experiment. Do you mean this by blocks?

Thanks

Maike
Maike, ordinarily we would create the design so that there were many (scores or hundreds) of sets of 20 MaxDiff exercises.  Each set of 20 would be a "Version" or "Block" in the design and different respondents would see different versions.  Making your design with many blocks neatly solves the problem you are seeing with order effects.  Of course if you used something like a balanced incomplete block design (BIBD) to make your 20 sets of items, then that design should also be controlling order/position effects.  How did you make your design?  Please send it to me at keith@sawtoothsoftware.com and I can take a look at it for you.
...