Conjoint analysis is one of the most potent instruments in market research. It can indicate the exact level of importance a customer assigns to a particular feature, how they balance the price-quality ratio, and the product configuration that will dominate the market.
Nevertheless, teams regularly end up with results they are unable to utilize, findings that seem theoretical, are at odds with reality, or remain unused in a slide deck.
Product managers and researchers seem to express similar concerns on social media: “Our conjoint results were very promising, but the product ended up being a failure.” “It seemed like the survey was totally disconnected from people’s real shopping behavior.”
A fundamental insight is as follows: Conjoint analysis does not fail because of the method itself, but rather due to poor study design, targeting the wrong audience, and weak interpretation.
Address those issues, and conjoint will turn into one of the most dependable decision-making tools that your team has at its disposal.
Why Conjoint Studies Fail: The Real Reasons
1. Picking the Wrong Attributes
The most frequent error with conjoint study design occurs even before the first respondent is presented with the survey.
Teams tend to add attributes that they themselves feel are important, rather than the ones that really influence customers’ purchase decisions. For example, a software firm conducts a test of “ISO 27001 certified” features with small business clients.
The salespeople consider it a critical factor. The buyer however has no idea what it is. The attribute simply increases cognitive load and leads to meaningless utility scores.
Attributes have to be derived from real customer dialogues, support tickets, sales conversations, and product reviews. If your client does not mention a particular term or attribute when describing his/her requirements, it should not be in your conjoint design.
2. Too Many Features, Too Much Fatigue
Survey fatigue is one of the most referenced issues with conjoint analysis. When the research features eight, ten, or even twelve attributes, respondents gradually become uninterested, and instead of evaluating thoroughly, they click through to the end.
The outcome is data full of noise, pretending to be insight. The typically recommended workable limit is four to six attributes. Above that, attention can drop – people say that they literally “gazed at a wall of text” and “guessing was what the choices felt like.”
Fortunately, when you use the right approach and techniques you can do more complex conjoint and get good results. We have long experience with conjoint tasks that routinely have 10 or more attributes. We at The Analytics Team have the experience, expertise, and confidence in doing complex conjoint to deliver high quality results even with longer attribute lists.
If a task is even more complex we have approaches like partial profile, adaptive and hybrid methods that we can apply.
Sometimes, If your product is feature-heavy, a MaxDiff test would be good for the attributes selection that will warrant a conjoint study.
3. Unrealistic Scenarios Nobody Would Ever See
One more frequent issue with conjoint analysis is the creation of product or service configurations that do not exist in the real market.
If this situation arises, the data immediately becomes ineffective if one tries to make a decision based on it. For instance, an airline study pairs premium seating with a budget fare.
Naturally, in this case, the flight attendants are producing data that no pricing strategy can accommodate.
Prohibited pairs that exclude unrealistic attribute combinations are a fundamental protection that most practitioners ignore. Employ them.
4. Surveying the Wrong People
Even a properly structured conjoint study can’t succeed if it targets the wrong audience. In B2B research usually the person who completes the online survey does not have signing authority.
In consumer research, general population samples are often used when the real target should be category buyers, not to mention that loyal customers’ preferences differ from those of new ones trying the product for the first time.
First of all, when recruiting, prepare a well-defined profile of the real decision-maker in the purchase scenario that you want to model.
And after that, be very selective. The quality of respondents is the basis of everything else.
5. Weak Design and Black-Box Outputs
Two technical mistakes tend to go hand in hand. One of them is a poorly planned experiment where you don’t use the full orthogonal design, so you can’t even separate the effect of one attribute from the other, and therefore, your part-worth utilities are unreliable.
The other is the treatment of those utilities as the last product. Part-worth utilities are only the means to an end. Do not make decisions based on raw utilities. These need to be translated back into a predictive, probabilistic model to be correctly applied.
The actual thing you deliver is the market simulation: what happens to the share of the market if you increase the price, add a feature, or remove an attribute? That’s where conjoint data really works its magic, but only if someone knows how to create and interpret the simulation.
Link every conjoint research to a particular decision before you do it. If you are unable to identify the compromise the research is supposed to address, it is not ready for the field.
How to Get Actionable Conjoint Results
Start With a Decision, Not a Research Question
Frame the study around a specific choice, bundle this feature, or price it separately? That question shapes every design decision that follows.
Validate Attributes With Real Customers First
Run ten to fifteen interviews before building the survey. Use the language customers use. Attributes that never surface in interviews need a strong justification to include.
Keep the Design Tight and the Audience Specific
Tie each attribute and levels to a decision and potential market scenario. Be as parsimonious as possible. Minimize to only the most critical prohibited pairs. Ensure your screener is built around the real buyer’s job title, purchase authority, and category usage. No shortcuts on respondent quality.
Plan the Simulation Before You Field
Know which market scenarios you will model and how you will present findings to whoever needs to act on them. Research without a clear downstream use is just overhead.
Choosing the right research method is just as important as designing it correctly. Conjoint analysis works best when aligned with the right use case and decision context. Read our detailed guide on MaxDiff vs Conjoint for a clearer understanding of when to use each method.
The Bottom Line
Conjoint analysis is not doomed to fail just because it is an incorrect technique.
It fails when the study design is done too quickly, the audience is only approximated, and the results are given without any explanation.
Once those three things are carried out correctly, an appropriate design, proper respondents, and decision-related outputs will not only come up with fascinating results.
It will revolutionize how your team sets prices, packages, and markets products.
When designed correctly, conjoint analysis becomes a powerful tool for pricing, product strategy, and decision-making. The Analytics Team provides world-class advanced analytics consulting that turns complex research into clear, actionable insights. Connect with our team to move your research forward with confidence.
