What is a Tradeoff Ranking Question and how to set it up?

Main use:

Tradeoff Ranking is mainly used to test the attractiveness and competitiveness of different ideas, including concept, messages, logos, features and other stimulus materials. It explores further why the customers like or dislike the ideas so that users could understand from their customers’ perspective and iterate their ideas accordingly.

Tradeoff Ranking consists of three parts.

The first part asks the participants to rate if they like or dislike the idea presented in a series of popup cards.

• Ideas will be shown one by one in a randomized order
• Here users can define the scales based on their needs - For example, they can choose a scale with three points for measuring liking, such as like, not sure, and dislike; Or they can use a scale with five points for gauging interest level, ranging from not at all interested to very interested.

The second part asks the participant to make head-to-head comparisons among the ideas that they liked in the first part, until we identify the one that the person like the most (i.e. the Top Choice). 

• Head-to-head comparisons are made only among the ideas liked in the first part. If it’s a 5 scale points, then all the ideas with a rating of 4 and 5 would go to the head-to-head comparison.
• If there’s only one idea they liked in the first part, then this one will automatically become the Top Choice without head-to-head comparison.
• If they don’t like any of the ideas, then this part will be skipped and they will be asked why they dislike one of the ideas.

The third part includes two follow up open-ended questions as to why they like or dislike the idea.

• “Why they like” will be asked for the Top choice
• “Why they dislike” will be asked for one of the ideas they dislike (We would randomly select one idea if there are multiple ones that they don’t like)

A quick video to show how Tradeoff Ranking works on the inca chatbot

How to set up in the platform:

1. In the Questionnaire Builder page, add new question OR insert new question by hovering over an existing question and click the green (+) button (either above or below the existing question).

2. Select Tradeoff Ranking from the popup menu that appears. 

3. Write your question number or label in the upper box 

4. Confirm or edit the Liking Question

(Note: The Liking Question's objective is to ask the participants to rate if they like or dislike the idea/concept presented in a series of popup cards)

5. Confirm or edit the Trade-off Question

(Note: The Trade-off Question's objective is ask the participants to make head-to-head comparison (which one they like better) among the ideas/concepts that they liked in the first part)

6. Add the concept stimuli by clicking Add New Concept. 

(Note: For the trade-off ranking question you must upload at 5 least materials for comparison. The default concept format is image only but you change the image label as your reference in the dashboard)

7. Select the Option types:

• 2-pt scale: Likes, Dislike
• 3-pt scale: Likes, Not sure, Dislike
• 5-pt scale: Dislike very much, Dislike, Not sure, Like, Like very much

8. Toggle Follow-up question to enable the two open-ended questions Reason for Favorite and Reason for Dislike. Probing is already enabled for these two follow-up questions.

• Reason for Favorite: This question is asked only for the most preferred concept/idea.
  
  
• Reason for Dislike: If more than 1 concept is disliked, this follow-up question will be only asked randomly for one of the disliked concept.

As default, there will be no follow-up questions for concepts/ideas with Not sure answers or Neutral response. 

(Note: You can use the sample questions or edit the wording for the two questions. The objective is to ask why they like the idea/concept (above the rest) for Reason for Favorite; and ask why they dislike or less prefer the idea/concept from others for Reason for Dislike)

Other Useful Question Settings

Allow User Corrections

To allow participants to update or change answer after submitting, toggle on  Allow User Corrections. 

Trade Off Types

In the dropdown, users can select an Image or Text type of idea/concept to be used as stimuli

*Skip First Stage

To skip the first part wherein participants rate if they like or dislike each of the idea/concept presented in a series of popup cards, toggle on Skip First Stage. This assumes that the participant likes all the idea/concept to be tested and allows participants to skip to head-to-head comparison

In this option, you may delete the default Liking Question by clicking the X button or modify the wording that introduces the Trade-off question, or customize the wording that would suit your question objective. 

*Comprehensive Mode

This allows head-to-head comparison between each concept. 

Enabling this feature skips the first part where the participants are asked to rate if they like or dislike the idea/concept presented in a series of popup cards. This assumes that the participants likes all the idea/concept and they are directly asked about the head-to-head comparison for all concepts. 

For example, you have 5 concepts, then participants will be asked 10 times for the head-to-head comparison with all the possible combinations of two concepts being compared. 

*Please note when enabling Skip First Stage and Comprehensive Mode, the Tradeoff question will not be displayed on the dashboard and the data accessible in your data export will be modified. 

Other Option Settings

The user can edit the following by clicking the "<" icon:

1. Option label (reference to data/dashboard)

2. Option text (how the participants view the selection)

3. Piped Options - this allows to PIPE or CARRY FORWARD options or specific answer choices from another question. Note that for this option to work, the question where the option is piped from should be set up before the Trade-off Question. To learn more about how to pipe or carry options, please refer to the article here.