False Positives in the Test for Identifying Random Respondents
In an earlier post, Bryan Orme described how to use the Lighthouse Studio data generator to identify “bad” (i.e. random) respondents in a choice experiment: https://www.linkedin.com/pulse/identifying-consistency-cutoffs-identify-bad-respondents-orme/. Using the method Bryan describes, you can generate random respondents and then measure how well their choice data fits their utility models, using HB estimation and a fit statistic called root likelihood (RLH). Using the RLH higher than that of 95% of the random respondents as a cutoff, you can identify 95% of random respondents and potentially remove them from your data set. The remaining 5% of random responders pass the test, but they are false negatives (i.e. the test has, by design, a 5% false negative rate).