Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Holdout tasks

Hi,

Is this understanding correct?
In case we use a fixed design(send the same set of tasks to all respondents),  all those tasks have the potential to be chosen as holdout task.

Moreover, Is it necessary to exclude holdout task from utilities calculation?

Thanks for support
asked Jun 27, 2017 by Robin59 Bronze (565 points)

1 Answer

0 votes
Robin,

This is correct, you could use that set as a holdout.  You don't learn much from a single holdout (unless you predict it terribly, in which case it helps you identify potential errors in your analysis) but it may serve as useful evidence that you can reproduce a base case through simulation, if you designed it to represent a base case.  

I would check utilities with and without the holdout case included in estimation - including it can change the utilities, because the particular combination of attributes and levels in the holdout set is greatly over-represented.
answered Jul 2, 2017 by Keith Chrzan Platinum Sawtooth Software, Inc. (62,500 points)
...