As many of you know by now, we're working hard on a web-delivered (Software as a Service—SaaS) version of CBC targeted to teaching students at the university level. It’s been an exciting project so far: how to take the best and key aspects of our (SSI-Web version of) CBC and produce a clean, intuitive, attractive interface for student learning and projects.
We think we've now cracked the nut of two challenging algorithmic aspects of the work. The first involves experimental design and the second involves utility estimation.
With experimental design, we wanted to be able to generate high-quality, customized designs for each respondent without placing too much demand on the server (since this application is fully-managed on the server, rather than running locally on the user’s device). Furthermore, we wanted to do something new by avoiding dominated concepts (a product concept that is logically inferior to another within the same task.) Initially, we thought that our current Complete Enumeration and Balanced Overlap designers would be very hard to beat. But, we've implemented an ultra-fast level-relabeling/swapping routine that produces individual-level designs with slightly higher D-efficiency than these current routines…all within about half a second per respondent.
For utility estimation, we want to avoid running full HB on the server for now for the student audience, due to the computational intensity involved. (User’s may export the data for analysis locally using CBC/HB if they prefer.) So, we’re planning to do individual-level estimation using empirical Bayes, essentially a shortcut (poor-man’s) HB that runs very rapidly (in seconds for hundreds of respondents). Past research presented at the Sawtooth Software conference and at the most recent ART/Forum by Jordan Louviere and Bart Frischknecht (et al.) suggests that other methods of individual-level estimation, including empirical Bayes, can do either just as well or nearly as well as the full HB. Just this last week, we verified that empirical Bayes can perform at nearly the same level as HB for a well-behaved CBC dataset. For this data set, it took three seconds to compute utilities for the 352 respondents.
Once utilities are computed, they are seamlessly loaded into our attractive and streamlined Online Market Simulator (which many of you are already using and sharing with clients).
We plan to conduct a split-sample study soon to compare this new, streamlined, SaaS version of CBC to the standard SSI Web version of CBC.