Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How can I detect, in the data, possible biass caused by Burden Respondents?

In a general CVA and CBC; in which part of the data is reflected the possible burden from respondents? or through which statistical technique do I figure this out?

What implications in the data can result if  this burden actually exists?

Would it be ok to said that the burden can lead a poor R-squared (and RLH for CBC)? or  that no significant differences could be found within attribute levels, meaning none attribute level can be differentiated in their contribution to  respondents preferences?

asked Nov 14, 2014 by Andrés P.

1 Answer

0 votes

I would not rely on a single statistic to tell me which respondents are overwhelmed by the task.  While a low fit measure like R-squared or RLH MIGHT be evidence of a respondent who is overwhelmed by the task, it's also possible that such a respondent could focus on just a single attribute, like price, and answer the survey very consistently (thus having a high R-squared or a high RLH in a choice-based model).   At our recent Turbo Choice Modeling event in Amsterdam Joseph White from Maritz Research presented results that showed this very clearly.  We just don't know from a high RLH whether the respondent is consistent because he was engaged and made very definite choices or whether he was overwhelmed and simplified his choices.

I know some analysts use a combination of measures to identify respondents who might have questionable data quality - for example did the respondent speed through the survey and did she straightline her responses?  I think a combination of measures is probably better than any single measure and that  the fit statistic is a problematic one to use to decide which respondents are suspect.
answered Nov 14, 2014 by Keith Chrzan Platinum Sawtooth Software, Inc. (50,900 points)

I agree with you, the r-squared and RLH alone can't tell about only this bias, therefore my question of how is reflected, in the data, the possible burden (noisy) data.  Which are these combination of mesures, that you mentioned?

In Getting Started with Conjoint Analysis (Orme, 2010) and other papers, mentioned that too many questions can burn the respondents and get noisy data, so do you know how did they notice this effect? or how was presented this noisy data?