Sawtooth Software: The Survey Software of Choice

Server Requirements : Frequently Asked Questions

How do we set up our servers to run SSI Web surveys?

Here is a link to documentation containin the steps needed to get SSI Web to work on Linux or Windows servers: www.sawtoothsoftware.com/serversetup

How much hardware do we need to throw at the problem?

Here is an article that gives some general recommendations: http://www.sawtoothsoftware.com/1379

If time is pressing, an 8 Core / 8 GB of RAM server with an SSD with 50+ GB free is more than good enough for most of our users, and adequate even for some of our heavier users.

Why do we need a server of that size?

There are lots of details as to why we would suggest a server of that size, which mainly has to do with accounting for peak usage, since respondents won't be evenly distributed. Even distribution would make most any server (1 CPU, 1 GB 50 GB of disk) capable of handling many respondents spread over time:

  1. Parallel participations: How many simultaneous participations of a survey as specified above can SSI Web handle?
  2. The number of possible simultaneous respondents depends mostly on the hardware available, and most strongly is limited by database write-through performance.
  3. For example, in our load balanced system (recently released to production) we used 256 simulated respondents (equivalent to 2-6x the performance requirements of a real human respondent, depending on average page viewing time). We simulated against the example Golf CBC survey included in SSI Web, and ended up finding that the database was the bottleneck. Our database maxed out at allowing ~60 respondent page submissions per second, using Rackspace's Managed Database Instance.
  4. Knowing the actual performance of the database server will be crucial to understanding how many respondents can comfortably take the survey at once. We can easily provide our standalone Data Generator to help you measure what to expect. 
    1. http://sawtoothsoftware.com/download/SSIWeb_Standalone_Data_Generator.zip 
    2. Just unzip, and point it at the survey in your test environment. Each virtual respondent answers the survey as fast as it is able to load the page and respond again, unlike a human respondent, which has to read and choose responses. A response is usually chosen and submitted within a few hundred milliseconds of completing a page load.
    3. Multiple instances can be opened on the same computer, with each Data Generator spawning 16 virtual respondents to simulate many respondents and test the maximum throughput of the hosting system you've created.
    4. Not all of the features work as expected, so if you're using passwords, you may need to provide a large number of respondents for a test survey and include it in the one-click link.
    5. ACBC doesn't currently work in the Standalone Data Generator.
    6. We've used the CBC sample survey in Help » Sample Studies » CBC Sample for our tests.
  5. The Database's write-through performance does not involve a huge number of megabytes being written per second, but creates a lot of small writes. Spinning hard disks have a hard time keeping up, so an SSD is recommended for the database server, since they tend to excel at many small reads and writes. Disk technology matters much less for the Web server/survey files.
  6. The key metric for determining the number of respondents that can be taking an SSI Web survey at once is determined by the database's write-through performance per second. Using the maximum number of responses recorded per second, one can estimate the number of respondents that can be handled by the system. Of course, CPU and RAM requirements for the webserver factor into this. See the attached Excel document for help getting a ball-park estimate.

What requirements are there for hosting surveys?

  1. How many CPU cores, GB of RAM and GB of disk space are needed is a tricky thing to estimate, but here is a link to a spreadsheet that should give you an idea of how much load is being generated by a given survey.
  2. The specifications given in the spreadsheet come from benchmarking Linux machines running as VM's in Rackspace's cloud infrastructure using SSD's, with the Database AND Webserver running on the same machine. Expect better performance if the Database is on a separate machine from the Web Server with either Windows OR Linux.
  3. The performance given for the 1 CPU/1 GB and 2 CPU/2 GB statements is well tested, but again, this is with the MySQL server on the same box. We'd expect more throughput if the database is split out. 
  4. The database server needn't be particularly performant from a CPU perspective if it's separate, as SSI Web doesn't do very many complicated SQL statements. Mainly it's storing data. A 2 CPU/2 GB box should do very nicely with an SSD. Data in a MySQL database even for thousands of respondents tends not to take too much space. We'd allocate 50-100 GB, but expect less than 10 GB to be used by the database. Probably even less.
  5. The study files are very small, so the webservers will only need ~50 Megabytes or less for all the study files, likely as little as ~10 MB, unless you've got lots of rich media in the survey (videos, audio files and large image files such as .jpg, .gif, or .png's).
  6. If you plan on using Microsoft SQL Server for the Database, then you'll be best served by using Windows as the Web Server.
  7. If you plan on using MySQL as the Database application, then the database may run on either a Windows OR Linux box, and you can use either Windows or Linux as the webserver machine.
  8. We'd expect machines running Windows Server to perform about the same or a little bit worse if they are running as Virtual Machines, and a little bit better if they are running on the "bare metal". Same goes for Linux machines. Virtual machines are very convenient, but have some overhead.
  9. Take a look at the number of pages in your surveys, and the average page viewing time for your surveys and use this information in the Load Estimator to give you a guess. Again, it doesn't matter how many people complete the survey, just how many pages they view and submit on the survey.

How would these requirements change if we used ACBC?

ACBC does require more CPU and RAM than a CBC survey, since it's dynamic. You may need to allocate more CPU and RAM resources for the survey.

How do I know everything is working properly?

Please see the SSI Web Context Diagram to understand how to make sure everything is working as needed.

  1. The SSI Web desktop application or its users need to be able to transfer the survey files to the web server.
  2. The web server needs to have access to a database server and a database schema to store respondent data. The database server application can be hosted on the same server, or on a separate server as needed.
  3. Respondents and Survey administrators need to be able to access the web server from wherever they will be doing so, over the Internet or on a local network if needed.

Sawtooth Software

6:30 AM to 5:30 PM Mountain Time
(GMT-6; GMT-7 Autumn/Winter)
Monday through Friday

Phone: +1 801 477 4700
Fax: +1 801 434 5493
Email: support@sawtoothsoftware.com

We are open!

SKIM Software

9:00 AM to 5:30 PM Central European
(GMT+2; GMT+1 Autumn/Winter)
Monday through Friday

Phone: +31 10 282 3500
Fax: +31 10 282 3560
Email: software@skimgroup.com
Online: www.skimgroup.com

Lighthouse Studio

Lighthouse Studio is our flagship software for producing and analyzing online and offline surveys. It contains modules for general interviewing, choice-based conjoint, adaptive choice-based conjoint, adaptive choice analysis, choice-value analysis, and maxdiff exercises.

Try Lighthouse Studio

  Buy Lighthouse Studio