Sawtooth Software: The Survey Software of Choice

Each year we field a customer feedback survey. This tool allows us to track our achievement in customer service and also track the relative use of different Sawtooth Software conjoint methods.

We fielded the study during March 2015. Among those users whose firms do conjoint analysis, we asked how many conjoint analysis studies they had conducted over the previous 12 months and what percent of the projects used different conjoint analysis approaches. If we weight the results by number of projects completed and normalize them to sum to 100% across the Sawtooth Software conjoint methods, we see the results for 2015 as shown at the far-right within the figure below.

Relative Use of Different Sawtooth Software Conjoint Methods

We've conducted this usage survey since 2003, so previous years are also displayed. Of note, the choice methods (CBC, ACBC, MBC) most recently accounted for 95% of the projects completed, whereas ratings-based methods (ACA and CVA) accounted for 5%. CBC continues to be the most commonly used approach, accounting for 79% of projects.

Project Methods

Percent of projects using different methods: (Weighted data, by number of projects conducted)

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
CBC (Choice-Based Conjoint) 35 46 54 56 59 62 68 65 68 60 65 69 69
ACBC (Adaptive Choice) 6 9 10 10 11 11
MBC (Menu-Based Choice) 2 2 2 2
ACA (Adaptive Conjoint Analysis) 25 20 15 12 11 10 6 6 5 4 4 2 3
CVA (traditional conjoint analysis) 10 9 8 7 6 4 4 2 3 4 2 3 1
Proprietary version of conjoint analysis 12 10 10 11 12 9 9 11 6 6 9 3 7
Self-explicated approach 5 4 3 4 3 3 2 2 3 2 3 3 2
Other software system/approach not listed 10 6 3 7 6 10 9 6 5 11 4 5 3
BPTO (Brand Price TradeOff) 3 5 3 3 3 2 2 2 2 1 2 3 1

*Among respondents who reported that their company conducts tradeoff/conjoint/choice or some other preference modeling. The figures for 2005 sum to only 96% because we included MaxDiff as an option and it captured the remaining 4%. In 2006 and later, recognizing the MaxDiff is often used in addition to conjoint analysis within the same study, we asked the usage question about MaxDiff as a separate question.

Types of Conjoint

Percent of total users reporting that their firms used each conjoint-related software system within last 12 months:

2013 2014 2015
CBC 66% 68% 67%
ACBC 29% 32% 28%
ACA 13% 9% 9%
CVA 6% 6% 4%
MBC 10% 10% 10%
SMRT 46% 42% 41%
Online Simulator 9% 14% 14%
Advanced Simulation Module (ASM) 13% 12% 13%

CBC Project Types

Among those that used CBC in last year, what percent of CBC projects utilized:

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Constrained estimation 20% 19% 22% 19% 29% 29% 23% 24% 29% 26%
Alternative-Specific Designs * * 22% 18% 26% 31% 26% 23% 27% 26%
Dual-Response None * 6% 18% 15% 18% 22% 20% 23% 22% 25%
Free Format Layout * * 12% 10% 23% 24% 15% 23% 20% 23%
Interaction Effects 26% 24% 26% 18% 20% 24% 19% 17% 17% 18%
Shelf Display * * 6% 5% 9% 16% 9% 11% 9% 11%
Constant Sum * 7% 12% 7% 11% 14% 9% 10% 10% 10%
Best-Worst CBC * * * * * * * 9% 6% 13%
*Either not available in the software or not asked about in that year's survey

MaxDiff (Maximum Difference Scaling)

The table below shows the percent of our total customer base (which involves many non-conjoint users) who reported that their firm had used MaxDiff during the previous 12 months.

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
8% 18% 24% 31% 37% 47% 52% 54% 57% 67% 68%

Sample/Online Providers

We also asked our users which Online Web Sample/Online Panels they primarily used over the last 12 months. Up to six mentions were allowed, asked in open-end format with no pre-coded responses. Sample providers who were mentioned by at least 3% of the sample in 2015 are reported below. (If a firm acquires another firm, the results for years prior to the acquisition are not netted.)

  2008 2009 2010 2011 2012 2013 2014 2015
e-Rewards/Research Now 23 30 46 47 47 48 50 46
SSI/Opinionology 21 22 23 27 28 28 31 38
Toluna/Greenfield/Ciao 17 23 23 17 15 13 19 18
Lightspeed Research/TNS/GMI 6 9 8 10 8 17 17 18
United Sample (Usamp)/Instantly 0 0 3 8 8 8 11 9
Critical Mix 6
Norstat 0 3 2 2 4 3 3 5
respondi 2 2 2 2 4 4
Bilendi/M3 0 1 1 0 2 4
Qualtrics 2 5 3