Use of conjoint for a chart audit

Question

We're looking at using conjoint to back into a chart audit study.  Have physicians pull 5 to 10 of their patient charts and fill in the information on the patient and which drug they prescribed.  Looking back at Keith's Choice Experiments for Physicians 2015 paper, I see this approach matches to Figure 7: Patient Type Experiment.  Which maybe then isn't an out of the box conjoint at all based on his description.  Am trying to figure out how viable this approach to a chart audit study is?  Using his example of 5 patient attributes each with 3 levels - and then let's say 4 drug options, what might a minimum sample size look like to even get rudimentary utilities?  Of course, we also have to consider the number of charts we can reasonably ask them to pull.  Maybe 5 to 10 would be the top which represents the number of exercises so that would also limit the amount of data we have.  I imagine this would have to be a custom analysis?  Sound like a viable approach?  Even if we only have n=200?  Thanks!

Answer 1

Steve,

While this wouldn't be an experiment (the patient factors in the charts are an observation you make, not something you design) this is a perfectly reasonable analysis.  Just like folks run regression models on cross-sectional data all the time that isn't experimentally designed, so too you can run logit models on data from a chart audit.  In fact, my first experience with what's not called an unconditional logit model was with just such data - case file data from a state prison system that I used to predict some particular inmate choices.

Comments

Keith, very helpful, thank you!  Since we can't test the design described above, do you have any guidance for sample size that you think would generate at least somewhat reliable utilities?  These are physicians so not a dime a dozen unfortunately.  Is 200 OK?  How much leeway downward could this go and still generate something useful?  Trying to get some sense of sensitivity to how the parameters could change and still avoid things breaking down.  Thanks!

---

It's hard to say without knowing how many levels you'll build for each attribute.  A rule of thumb for logit models would be something like this, adopted from Peduzzi et al (1996):  your sample size should be greater or equal to 10kp/t where k is the number of parameters in your  model, p is the number of drug options a respondent might choose and t is the number of observations (charts) you pull per respondent.

k is a tricky one.  In this kind of model, an unconditional logit model, the number of parameters is (total # of levels - # of attributes)*(p-1).

Our new book (http://www.sawtoothsoftware.com/81-products/conjoint-analysis-software/1852-new-advanced-conjoint-analysis-book) has a chapter on models and experiments like this one and discusses issues like sample size, design and analysis of the unconditional logit model.