How To Be A Games User Researcher

by Steve Bromley

As development continues, teams are happier to test - but are less able to make changes

Yeah, this is a huge issue

Neutrality is, therefore, a key part of being a games user researcher - being evidence-led and avoiding subjective opinions about design decisions will help create trust in the truth of their research findings.

The usefulness of a games user researcher

The goal of a user researcher is to reduce the undesigned difficulty so that only the designed difficulty is experienced by the player.

Love this, Undesigned vs Designed Difficulty

efficiency is less relevant for many games - people presumably have a better time spending thirty minutes slaying the dragon than if it was over in five seconds.

The multiplayer experience is also often an important part of games, and requiring multiple people to use the software simultaneously is more common than in other sectors - leading to different methodological choices.

This can be exacerbated in studios which currently run playtests as informal feedback sessions, who may not understand the difference between those tests, and a robust and reliable research study.

As a user researcher, I’ve never run a study without finding issues - no matter how small the scope - and it is my belief that something useful can always be learned from studies, regardless of whether the game is ‘ready’.

For each objective, think about how it could be measured. Some objectives are focused on performance or behaviour, and could be measured by observing what players are doing - are they able to complete the section? Do they know where to go? Do they get stuck for a long time? Observations can be performed either by the researcher, or from in-game analytics. Other objectives are focused around a player’s thoughts, such as do they enjoy the experience, does the player believe it’s too easy or too hard, etc. This data can be captured by getting players to articulate their thoughts or opinions - either through asking them questions verbally, or asking for ratings or comments on a survey.

Observation describes watching players play the game and looking out for parts where the player doesn’t do what the designer expected them to do.

Probing questions are usually asked while observing,

researcher’s skillset is how to ask those questions without revealing information to the player

player’s understanding of how the game works (their mental model of the game),

Questions such as ‘will you buy this game?’ do not correlate well with the behaviour of buying the game, and so can be extremely misleading and unhelpful.

The data from surveys is most useful when compared to other results

This is one reason that the multi-seat playtests can be common in games since this allows surveys to be run on-site to answer research objectives.

One of the most common statistical tests a user researcher performs is to calculate the confidence intervals between different results,

Jakob Nielsen’s 10 Usability Heuristics for User Interface Design[6]

be either much bigger fans, or much more vocal critics, than an average player. Recruiting only unpaid participants will introduce a sampling bias in the kind of users taking part in your research sessions.

In order to reduce the chance that these issues disrupt the real study, it’s very important to run a pilot study.

mind maps can speed up the process of capturing notes, in a suitable format for speedy analysis. More on this is in the previous book Building User Research Teams.