MaxDiff

MaxDiff helps you find out which options matter most and least to your customers. Instead of asking people to rate every option or rank a long list, you show a few at a time and ask them to choose one most important and one least important.

It's useful for prioritizing product features, comparing messages, testing benefits, or deciding which products to offer. You get a simple summary of the choices people made, with deeper statistical analysis available through Research Assistant.

A MaxDiff question takes one place in your survey, but includes several rounds. Each round shows a different group of options. The person chooses one positive and one negative option, then clicks Next to continue. They can go back to an earlier round and change their choices.

If the question is Required, every round needs two different selections. For an optional question, people can leave rounds blank.

For example, you might ask:

When choosing a delivery service, which of these is most important and least important to you?

Your options could include delivery speed, price, tracking updates, flexible delivery times, packaging, pickup locations, customer support, and returns.

With eight options, four options per round, and three minimum appearances, each person completes six rounds. They compare a manageable part of the list each time, rather than sorting all eight at once.

MaxDiff measures preference relative to the options you include. Something can finish last and still be important. The results tell you how people prioritize this list, not whether they like or dislike every option in absolute terms.

Keep each option short, distinct, and easy to understand. Option labels must be unique and can contain up to 160 characters. A list of 8 to 20 options is a useful starting point for many studies, but a shorter list may be enough for your question.

Match the labels to your question. For a message test, “Most convincing” and “Least convincing” may make more sense than “Most important” and “Least important”.

The number of rounds is calculated for you. We multiply the number of options by minimum appearances, divide by options per round, and round up. For nine options shown four at a time with three minimum appearances, that's seven rounds. Rounding up means some options may appear one additional time.

A question can include up to 20 rounds. If your setup needs more, show more options per round or reduce minimum appearances.

You don't need to create the groups yourself. Iterate generates them when you save, balancing how often options appear, spreading comparisons across different pairs, and varying where options sit on the screen. Different people can receive different sequences of rounds.

Once someone starts the question, their sequence stays the same as they move forward and back.

After saving, the options and round settings are locked. You can still edit the question text, instructions, prompt image, Required setting, and positive and negative labels.

To change the options or round settings, Duplicate the question. The duplicate opens for editing before you save it. Responses to the original stay with the original question.

If you translate your survey, the options and positive and negative labels are available in Edit translation copy. Keep translations consistent with the original meaning so everyone is answering the same study.

Open View responses to see the MaxDiff summary. It updates as responses come in and shows:

Preference score = (Best − Worst) / Times shown × 100

For example, an option shown 100 times, chosen as Best 40 times, and chosen as Worst 10 times has a score of +30.

Scores range from −100 to +100. Positive scores mean an option was chosen as Best more often than Worst. Negative scores mean the reverse. Zero means the counts are equal. Dividing by Times shown lets you compare options even when they appeared slightly different numbers of times.

This is a summary of observed choices, rather than a fitted statistical model. Use it for a quick read on the results. Your usual response filters apply to this summary. Blank rounds don't contribute to the counts.

You can also open individual responses to see each person's Best and Worst choices for every round.

Modeled preferences is part of Research Assistant. If you don't see it, contact our support team to learn more about adding it to your plan.

The analysis uses hierarchical Bayesian modeling. In plain terms, it estimates each person's relative preferences from their best and worst choices, while also using patterns across respondents to help make those estimates. It considers the options shown together in each round, rather than only the total counts.

The model checks that its estimates have settled sufficiently before making results available. If those checks don't pass, the analysis can fail rather than show unreliable estimates. More data may help, but isn't a guarantee. Contact us if you need help with a failed run.

Each run uses the available MaxDiff responses for that question. Skipped rounds are left out, and dashboard filters don't change the modeled results. Modeled analysis currently supports questions with up to 10,000 respondents who answered at least one round. Questions above that limit can't be analyzed.

Modeled results are a snapshot. To update them after more responses arrive, use Run again. Another run becomes available 24 hours after the previous analysis completes, once you have at least 10 new responses or 10% of the previous respondent count, whichever is larger. The button's tooltip shows what you're waiting for.

The Scores tab has two views of the same modeled preferences.

Preference share expresses estimated relative preference as percentages that add to 100% across your options.

For each person, the model converts their estimated utilities into shares totaling 100%, then averages those shares across respondents. Higher shares indicate stronger relative preference.

A share of 30% does not mean 30% of people selected the option in the survey. It also isn't a prediction of sales or market share. It's the option's share of estimated preference within this particular list.

Utility is the underlying estimate of preference strength. Each person's utilities are centered so they sum to zero, then averaged across respondents.

A negative utility doesn't mean someone dislikes an option. The differences between options carry the meaning. A utility of 2 isn't “twice as preferred” as a utility of 1.

Utility and the observed preference score can look similar because both show positive and negative values around zero. They are different calculations. The preference score comes directly from counts, while utility is estimated by the model.

The whiskers show uncertainty in each estimate. Given the responses and the model's assumptions, the displayed range contains 95% of the probability for that estimate.

A narrow range means a more precise estimate. A wide range means there's more uncertainty, so small differences between options deserve a cautious read. The range describes uncertainty in the overall estimate, not how widely individual people's preferences vary.

Use the Compare section below the chart to check the difference between two options. Comparisons always use preference share, including when the chart is set to Utility.

The result is the first option's share minus the second option's share, in percentage points. If the shares are 25% and 20%, the difference is +5 percentage points.

The comparison also has its own 95% credible interval. If the range includes zero, the responses don't clearly establish which option has a higher share. “No clear ordering” doesn't mean they're identical. It means the available evidence doesn't clearly separate them.

The Counts tab shows the observed choices in a table, including Best %, Worst %, and preference score.

Best % is the number of Best selections divided by Times shown, multiplied by 100. Worst % uses the same calculation for Worst.

These are live counts, so they can include responses received after the model ran. Modeled results stay unchanged until you run the analysis again.

The highest-scoring options don't always make the strongest combination. Several popular options may appeal to the same people, while a less popular option appeals to people the others don't cover.

For example, two product features might both be favorites among experienced users. Including one of them alongside a feature preferred by newer users could cover a broader audience than including both.

TURF stands for Total Unduplicated Reach and Frequency. It helps you compare combinations using the individual preference estimates from your completed model run. Someone covered by several options isn't counted multiple times.

Reach means something slightly different under each method, so compare combinations using the same method. A weighted reach of 70% isn't a count of 70% of people saying they would buy one of the options.

Use Include to let an option be considered, Always include if it must appear in every combination, or Exclude to leave it out. Excluding an option doesn't move another option into that person's first or second place.

You'll see a recommended combination, alternatives, and a chart showing how reach changes as combination size increases. Use Try your own combination to calculate reach for a set you choose.

Small searches check every possible combination. Larger searches use a faster search, so the recommendation isn't a guarantee that every possible combination has been checked. TURF uses fitted preference estimates and doesn't show credible intervals for reach.

Use the results alongside practical considerations such as cost, availability, or implementation effort. They describe relative preferences among the tested options, not future sales.

Your survey CSV has separate Best and Worst columns for each MaxDiff round, with the selected option labels as values. Unanswered selections and skipped questions are blank.

The export keeps one row per survey submission. It contains people's answers, rather than modeled preference shares, utilities, or TURF results.

If you have questions about setting up MaxDiff or interpreting your results, reach out to our support team.