False Positives in the Test for Identifying Random Respondents

In an earlier post, Bryan Orme described how to use the Lighthouse Studio data generator to identify “bad” (i.e. random) respondents in a choice experiment: https://www.linkedin.com/pulse/identifying-consistency-cutoffs-identify-bad-respondents-orme/. Using the method Bryan describes, you can generate random respondents and then measure how well their choice data fits their utility models, using HB estimation and a fit statistic called root likelihood (RLH). Using the RLH higher than that of 95% of the random respondents as a cutoff, you can identify 95% of random respondents and potentially remove them from your data set. The remaining 5% of random responders pass the test, but they are false negatives (i.e. the test has, by design, a 5% false negative rate).

Read More

Consolidate Your Survey Programming Business on Sawtooth Software’s Platform

Sure, we’re known for conjoint/MaxDiff. But, did you know that 38% of our customers use Lighthouse Studio exclusively for all their survey research projects? Featuring SPSS file export, gorgeous survey style, free hosting, and an offline mobile Android app, it’s the powerful and economical tool for all your survey research needs.

Read More

Rybbon Incentive Management Integration

We are excited to announce a new integration with the incentive management platform, Rybbon!

Read More

Webinar: Avoiding Common Pitfalls in Conjoint Analysis

Take advantage of decades of experience working with Sawtooth Software customers from our technical support team to learn about common pitfalls and how to avoid them. This webinar is geared towards beginners and those with a few studies under their belts. We will cover topics from attributes and levels, experimental designs, to fielding your survey and running analysis.

Read More

New Technical Paper: Consistency Cutoffs to Identify "Bad" Respondents in CBC, ACBC, and MaxDiff

Over the last few years, the incidence of bad respondents is increasing. Conjoint analysis and MaxDiff have a fit statistic called RLH when using HB estimation that helps identify bad respondents. As long as the conjoint or MaxDiff questionnaire has enough questions relative to the number of levels or items in the study, random responders can be identified with a high degree of accuracy. This paper, authored by Bryan Orme of Sawtooth Software, gives instructions for generating random data to identify the RLH cutoff that has a high probability of identifying random respondents. It is available in the middle of the General Conjoint section of the website's Technical Papers library.

Read More