Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Checking for "straight-lining" data

Is anyone aware of a software/program or a way to check data for straight-lining responses?  By that, I'm referring to people who just quickly key in they same answer over and over (for instance on a series of rating questions everything has the same rating).  This discussion has come up and the key thing I look for currently would be unusually short interview lengths but I'm interested in how others look for and deal with this.  Any input would be welcomed.
asked Apr 10, 2014 by Jay Rutherford Gold (24,385 points)

3 Answers

0 votes
We call these people "lazy raters". One additional method we use above and beyond quick response times is to create a "lazy rater" variable (true or false) and apply conditions that may look for all low, medium or high ratings on grid questions.

We sometimes identify questions of interest and then setup variables that count the different ratings. If any of the counts hit a certain limit, then the respondent can be deemed to be a "lazy rater". The beauty of this is you decide the rules.

You can do this either in Sawtooth during data capture or in your analysis software.

In saying all of that, you do need to be careful with your approach as you would not want to lose legitimate surveys.
answered Apr 10, 2014 by Paul Moon Platinum (60,230 points)
0 votes
For maxdiff, you can do an export and one of the files is the .position which is a record of the position of the chosen items. You can look for respondents then that have answers like 1,2,1,2,1,2. Cbc this is easy as all answers are already position based. In 8.3, Acbc is now easier to spot this. Screeners would look like 1,1,1,1 or 0,0,0,0. Choice tasks and calibrations are just like Cbc.

Other than that I don't know of any automated way to perform this. If you want the entire survey checked for straight liners, you would have to do this manually for now.
answered Apr 10, 2014 by Mike Lodder Gold (23,410 points)
0 votes
I don't typically do this on-the-fly, but computing the standard deviation of items in a grid is a quick way to identify straight-liners. If the SD is zero, I raise a red flag. This would be an easy JavaScript task.
answered Apr 10, 2014 by Paul Riedesel