Sawtooth Software: The Survey Software of Choice

Results of Sawtooth Software User Survey

In March of each year we field a customer feedback survey. This tool allows us to track our achievement in customer service and also track the relative use of different Sawtooth Software conjoint methods.

Among those users whose firms do conjoint analysis, we asked how many conjoint analysis studies they had conducted over the previous 12 months and what percent of the projects used different conjoint analysis approaches. If we weight the results by number of projects completed and normalize them to sum to 100% across the Sawtooth Software conjoint methods, we see the results as shown at the far-right within the figure below.

Relative Use of Different Sawtooth Software Conjoint Methods

We've conducted this usage survey since 2003, so previous years are also displayed. Of note, the choice methods (CBC, ACBC, MBC) most recently accounted for 96% of the projects completed, whereas ratings-based methods (ACA and CVA) accounted for 4%. CBC continues to be the most commonly used approach, accounting for 78% of projects.

Project Methods

Percent of projects using different methods: (Weighted data, by number of projects conducted)

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019
CBC (Choice-Based Conjoint) 35% 46% 54% 56% 59% 62% 68% 65% 68% 60% 65% 69% 69% 72% 73% 72% 71%
ACBC (Adaptive Choice) 6% 9% 10% 10% 11% 11% 11% 12% 14% 13%
MBC (Menu-Based Choice) 2% 2% 2% 2% 2% 2% 3% 3%
ACA (Adaptive Conjoint Analysis) 25% 20% 15% 12% 11% 10% 6% 6% 5% 4% 4% 2% 3% 2% 2% 2% 2%
CVA (traditional conjoint analysis) 10% 9% 8% 7% 6% 4% 4% 2% 3% 4% 2% 3% 1% 2% 2% 2% 1%
Proprietary version of conjoint analysis 12% 10% 10% 11% 12% 9% 9% 11% 6% 6% 9% 3% 7% 6% 3% 2% 5%
Self-explicated approach 5% 4% 3% 4% 3% 3% 2% 2% 3% 2% 3% 3% 2% 1% 1% 2% 1%
Other software system/approach not listed 10% 6% 3% 7% 6% 10% 9% 6% 5% 11% 4% 5% 3% 3% 4% 2% 3%
BPTO (Brand Price TradeOff) 3% 5% 3% 3% 3% 2% 2% 2% 2% 1% 2% 3% 1% 1% 1% 1% 1%

*Among respondents who reported that their company conducts tradeoff/conjoint/choice or some other preference modeling. The figures for 2005 sum to only 96% because we included MaxDiff as an option and it captured the remaining 4%. In 2006 and later, recognizing the MaxDiff is often used in addition to conjoint analysis within the same study, we asked the usage question about MaxDiff as a separate question.

Types of Conjoint

Percent of total users reporting that their firms used each conjoint-related software system within last 12 months:

2013 2014 2015 2016 2017 2018 2019
CBC 66% 68% 67% 66% 68% 70% 72%
ACBC 29% 32% 28% 28% 32% 34% 37%
ACA 13% 9% 9% 8% 8% 11% 7%
CVA 6% 6% 4% 5% 4% 5% 5%
MBC 10% 10% 10% 11% 9% 13% 10%

MaxDiff (Maximum Difference Scaling)

The table below shows the percent of our total customer base (which involves many non-conjoint users) who reported that their firm had used MaxDiff during the previous 12 months.

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019
8% 18% 24% 31% 37% 47% 52% 54% 57% 67% 68% 70% 70% 72% 73%

Sample/Online Providers

We also asked our users which Online Web Sample/Online Panels they primarily used over the last 12 months. Up to six mentions were allowed, asked in open-end format with no pre-coded responses. Sample providers who were mentioned by at least 3 of the sample in the last year are reported below. (If a firm acquires another firm, the results for years prior to the acquisition are not netted.)

  2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019
Dynata/Research Now/Critical Mix/SSI 23 30 46 47 47 48 50 46 45 48 66 63
Lucid 2 2 2 2 5 8 13 16
Cint 1 2 5 4 3 3 4 4 8 12
Lightspeed Research/TNS/GMI 6 9 8 10 8 17 17 18 17 16 18 11
Bilendi/M3 0 1 1 0 2 4 4 7 8 8
Qualtrics 2 5 3 6 8 5 8
Toluna/Greenfield/Ciao 17 23 23 17 15 13 19 18 10 10 8 5
Market Cube 1 1 3 4
EMI Research Solutions 0 2 2 3 2 3 2 4 3 2 4
Norstat 3 2 2 4 3 3 5 2 5 5 3
ROI Rocket 1 2 3
Prodege 3