Task Scenarios: How to Avoid User Testing Bias

You’ve found the right candidates. You’ve set aside the time and space for your usability tests. Now what?

It’s time to create specific task scenarios to guide your users during the sessions.

How Task Scenarios Prevent Useless Testing Results

Consider a website containing that allows users find flight deals based on current location and desired destination. The user goal is crystal clear: “Find the best flight deal for my trip”.

Photo credit: Kayak

The interface (and overall experience) is successful if users can launch it, locate themselves and find a good deal – all without friction. When conducting usability testing, your goal is to reveal any of the following potential issues:

Misplaced interface elements
Buttons that don’t look like buttons
Unclear copy
Navigation inconsistent with user expectations

It doesn’t sound very complicated, but it’s fairly easy to get it wrong.

For example, if you ask users during the test to: “Pick a destination and a departure location, select dates and look for a flight using the box on the right-corner,” the test won’t provide any useful insights. You’ve already lead them down a single path. Your goal is to understand if they’ll follow the right path without guidance.

This is an exaggerated example to show my point, but bias is still incredibly common in usability task instructions.



Instead, a more effective instruction would be: “You’re planning to travel to Berlin next week. Use the website flightdeals.com to find the best flight deal for your that trip”. The phrasing doesn’t give away any bits of the interface and or experience – such as “Using the box on the right-corner”.

These clear instructions are Task Scenarios – i.e. activities aimed to replicate natural user scenarios.

The 5-Step Process to Creating Useful Task Scenarios
1. Describe your user.

Refer to your user research results: revisit user interviews, check the personas created, and dive into any other results that you or your team might have gotten. You need to understand user motivations, pain points, comfort with technology and their background.

2. Focus on the user’s goals.

What must happen for users to leave satisfied with your product?

3. Write down the tasks

Now that you’ve created a complete user profile, write down a list of specific tasks that lead to success in your product. Tasks must be clear and concise. Unless you’re designing an extremely complex product (like in an enterprise setting), less than 10 will usually work.

Remember the flight deals website example above? Here’s a few example tasks to write down:

Finding the best flight deal
Booking a flight
Receiving alerts for future deals

Don’t give instructions or lead in any way. Write simple binary tasks so that you can clearly measure success or failure.

4. Write scenarios for the tasks

The task is incomplete without realistic scenarios: the participant needs to know in which context of the tasks. Background scenarios allow users to relate to the testing more easily.

Create a scenario that mentions what illustrates the “what” and the “why” behind the tasks. Generally, 3-5 scenarios per task will suffice.

Pay close attention to details when creating scenarios. For example, the ones you create for a retired lawyer living in a US city will most likely be different from the ones you’d write for a 20-year-old actor living in India.

5. Combine tasks with scenarios

Now you must combine tasks and their scenarios to create Task Scenarios.

For example, let’s create a task scenario for a selected task from the flight deals example:

Task: “Booking a Flight”


Scenario: “Josh remembered that he promised his wife to take her on a trip to Hawaii for their wedding anniversary, which happens next week and now he needs to book their flight.”


Task Scenario: “You promised to take your wife to a trip to Hawaii next week for your wedding anniversary. Use flightdeals.com to book a flight for that trip.”

As shown in the below usability test created in UXPin for a prototype, make sure you only need to show users the task scenario.

The 6-Point Checklist for Useful Task Scenarios

At TestLodge, we use the below checklist to keep task scenarios on point:

1. Are they realistic?

The scenarios you create need to be realistic for that specific participant’s background. Don’t ask users to do something they usually wouldn’t do.

For example, let’s say Josh’s wife is responsible for organizing trips (and as part of it, booking flights). In this case, creating a task where you ask him to organize a trip won’t be relatable at all.

To avoid straying from reality, compare your scenarios against your user research (especially interviews).

2. Are you proving any clues?

Don’t provide any clues on how to complete that task. The Task Scenarios shouldn’t have indications of where to go, what buttons to push, what actions to perform or anything similar. Any explicit guidance will bias results.

3. Is it specific?

Your task should be specific enough so that the user won’t ask the moderator for clarification.

“Book a flight” has a lot of room for interpretation. That’s why we specified:

The what: Book a flight to Hawaii
The why: For a wedding anniversary

4. Does your task have a correct solution?

You need a way to measure whether or not users accomplished the task.

“Open flighdeals.com” doesn’t have a solution. The user will open the website, but then what? Instead, “Visit flightdeals.com and book a flight from SF to NYC next week” indicates a crystal clear solution.

5. Are you asking the participant’s opinion?

Don’t ask “What do you think of the feature X?”. The user will just look for a way to say the “right thing” and will provide mostly visual feedback. Asking him/her to use the feature, for example, is a better option.

6. Is it as concise as possible?

After writing, ask colleagues for input or review them yourself and try to trim out all unnecessary words.

Thinking about Real World Conditions

Consider the difference between these two situations:

A paid participant using a food delivery app in a well-lit usability lab supported by a reliable connection and a calm state of mind


A stressed person trying to grab some quick food before their next meeting.

Here lies one of the biggest challenges of usability tests: recreating real world conditions that account for physical and/or emotional states, location, time and other factors.

It’s not always easy to separate the edge cases from the core use casese.

As explained in The Guide to Usability Testing, here’s two solutions:

If possible, visit the user’s natural environment. In the case of the food delivery app, you could previously recruit a user and arrange a time that the person usually is under the stresses mentioned.


If not possible, then gather as much insights into these cases when conducting user research. Ask users for feedback on their difficulties experienced in those scenarios. While the results aren’t totally empirical, they’re certainly better than designing off pure assumption.

Next Steps

Usability tests are only as effective as the task scenarios assigned to participants.

Remember to craft your scenarios based on prior user research, leave out any biased phrasing, and write the scenario as clearly and specifically as possible.

If you found this guide useful, check out the in-depth Guide to Usability Testing. You’ll learn best practices for 20+ different usability tests.

The post Task Scenarios: How to Avoid User Testing Bias appeared first on Studio by UXPin.