Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Assigning a version in CBC

Hi all

Our client has provided us with a spreadsheet detailing a CBC they want programmed.  There are 8 versions.  10 tasks, with 3 choices and 15 attributes.

There are two types of respondent.  The client wants an equal (or close to it) number of versions shown  and across both types (so n = 800 overall, 400 per type, and 50 per version per type).

Can we use quotas and assign the version number?
If the answer is no ... would another approach be to set it up twice, and have type 1 go through one and type 2 go through the other?  How are the version numbers assigned?  Would we get an even number?

Cheers
Marion
asked Sep 3, 2014 by Marion Bronze (3,435 points)

4 Answers

+1 vote
What Mike is talking about is that if your client has given you the experimental design (the combinations of attribute levels to show in each task), then there is a way to ask our software to import and use that design.

To do that:

1. On CBC's Design tab (after specifying your attributes and levels) generate an 8-version plan using our software that mimics your client's setup (to create a dummy design that you'll end up replacing with what the client gave you).

2.  Export the dummy design using the Export button from the Design tab.  This creates a .csv file that you can open with Excel.

3.  Open the design file CBC created, and following that same format, overwrite it with the design your client gave you.  Save the .csv file.

4.  From the Design tab, click the import button and browse to the .csv file (that you modified) and import it.

5.  Finally, run Test Design to make sure the design you have imported seems efficient to estimate the effects you are interested in.

Please note that now SSI Web will automatically hand out the design to subsequent respondents (respondent #1 to design #1, respondent #2 to design #2, etc).  But, that also means that a respondent who is incomplete will get assigned a version of the plan...the next respondent coming to the survey gets the next version, etc.  So, incomplete respondents will create small deficits in the versioning.  Usually, those holes tend to balance out, though.

Experimental designs with many versions in CBC can handle quite well what would appear to be quite bad balance in actually completed surveys.  This often freaks clients out, because they think they need near-perfect balance to get good results.  But, if the versioning plan is initially created well (wherein each version has a balanced presentation of levels and is nearly orthogonal within the version), then even pretty poor balance in handing out the designs causes very little problem for unbiased and precise utility estimation.

I think there are more sophisticated ways to try to use quota control to hand out specific versions...to obtain better balance in the presence of incompletes...but I don't know the details of how to do that.  And, it's most likely overkill to do anyway.  If you really MUST have near-equal counts on your versions, then you should call our technical support line and see if the wizards there (who know more than I do on these mechanical issues) can help.
answered Sep 3, 2014 by Bryan Orme Platinum Sawtooth Software, Inc. (146,540 points)
Thanks, Bryan ... yeah that's what we thought would be the case (re incompletes).  We were living in hope.  I'll send off an email to the tech guys/gals and hopefully we'll have an answer in our morning tomorrow.  (BTW, you and Mike on the forum late ... thanks for your help)
Just to follow-up with this on what I e-mailed Marion, the only thing I can think of is to make eight, 1-version CBC exercises in a study.  Then use quota control with 8 cells where each cell is set to Always Qualify and check for cell membership randomly.  Then skip people around so that they see the appropriate CBC exercise.  The quota would make is so that once a cell fills up, nobody should see that exercise anymore.

You would need to do a little data manipulation when you are done to make a design file with the 8 versions and recode the design ID and version # of the respondent's data, then probably use the standalone CBC/HB software with the dual CSV import option.

You probably would also want to randomize the order of the CBC tasks and (ideally) turn on randomized concept sorting
0 votes
SSI web does this for automatically. So in your case every 8 respondents will see the same as the previous 8th.
answered Sep 3, 2014 by Mike Lodder Gold (23,410 points)
So how does the software handle incompletes?
0 votes
I had same type of request from my past client. I used to store whole design in array using perl and assign CBC version using least fill approach. That way we can get equal balance on completes.
answered Sep 3, 2014 by Rajesh Rana Gold (23,530 points)
0 votes
I would again advocate provision of selecting version through a constructed list. Default option can be obviously current method. However, if someone want to select version at runtime, they should be able to do by specifiying his custom list. You can pop-up a warning that this may effect the equal distribution etc.
answered Sep 8, 2014 by Saurabh Aggarwal Gold (29,515 points)
...