Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Validity of a Conjoint Study

Apologies in advance - this question is not directly about Sawtooth Software, but about the validity of a design.

My research focuses on the design of a standardised set of cyber-attack graph visual semantics. An attack graph is a visual (data flow-like) representation of a cyber-attack. Although attack graphs are a very popular method of describing attacks in the academic literature, there is no standard in terms of the visual semantics. Consequently, there are at least 50 structures published in the academic literature. I am trying to understand which configuration in terms of the visual semantics in an cyber-attack graph, are preferred by practitioners. I hope to do this through a conjoint analysis study.

I have read Bryan’s paper: “Sample Size Issues for Conjoint Analysis” and have adopted the formula which proposes the number of questions to be asked. The total number of levels across all attributes is 16, the total number of attributes is 5, therefore 3(K−k+1) = 36.

I have generated 36 products using SPSS and aim for each participant (sample is around 100) to be provided with 2 products each and to do a pairwise ratings based comparison, in other words, each participant will have 2 products – both with different visual semantic designs. The participant will rate each component of the design i.e. compare attribute 1 (design 1 vs design 2) and select a preference. The participant will do this for each of the 5 attributes (possibly multiple times) and then provide an overall product design (they will now see the whole graph where previously they only saw sections of it).

It seems like a very complex design and hence why I want to determine the validity of it before I move ahead. Is this something someone may be able to advise on?

Look forward to hearing from you!
asked Aug 6, 2017 by HarjinderLallie (120 points)
edited Aug 6, 2017 by Walter Williams

1 Answer

0 votes

Your design seems not only complicated, but confusing.  Typically conjoint experiments focus on evaluations (choices among or ratings of) of the profiles (combinations of attributes).  

It sounds like you are maybe doing something akin to the pairs section of the ACA (adaptive conjoint analysis) but even that is only an approximation to the research design you are describing.  

I suggest you call someone our Support group and discuss your design, because from my understanding of your description it is unusual.  I do not want to promise from what I've read that your design will give you what you want.
answered Aug 6, 2017 by Keith Chrzan Platinum Sawtooth Software, Inc. (52,450 points)