More on this book
Kindle Notes & Highlights
Started reading
March 8, 2020
In most organisations, the degree that people are interested in user research will start out quite low.
help make the best possible use of all the very expensive engineering time. Or perhaps they are looking to professionalise the work that product teams are doing already to understand their users, raising the quality of that work, and ensuring that the information everyone is acting upon is robust and sensible.
- Explaining what user researchers do and framing the benefits of running studies to inform business decisions around the factors that are important to the company. - Avoiding many of the compromises that will severely limit the effectiveness of user researchers. - Finding allies and building relationships with other teams across the organisation. - Creating and justifying an appropriate budget to run effective user research. - Setting expectations for how much research is enough.
Those decisions are informed by a whole variety of sources.
analytic data
CEO reading an article about blockchain
The goal of a user research team is to improve the quality of those decisions by providing accurate information about user behaviour to inform and evaluate the product and help everyone do a good job.
Building the case for how user research can help improve development is essential groundwork required before research can be truly embedded in how things are made and prevents the findings from research studies being dismissed later.
GDS’s ‘Discovery’->’Alpha’->’Beta’->’Live’, to the Design Council’s Double Diamond which has the steps ‘Discover’, ’Define’, ’Develop’ and ‘Deliver’.
At the core of every model of how design works is a process that involves: - Defining and refining the problem, from a vague idea of what the domain for the problem is, to a clear ‘this is what we want to fix’ decision. - Creating and evaluating potential solutions that fix that problem. - Refining one (or more) solution until it’s ready to be put into the world.
These models seem very sensible, and when followed lead to the development of appropriate solutions to problems that exist in the world - design is second only to luck (and possibly marketing) as a factor that will lead to creating something successful. They are applicable to broad topics, such as the creation of an overall product or service, and on a smaller scale for the development of individual features for an existing product.
However, one of the most critical things when assessing the suitability of a potential solution is understanding its impact on users
Very true!
example, while evaluative research is looking at the implementation of an idea for a product, new spikes of activity to add features may require additional generative research about the audience and their behaviour to occur.
Evaluative research vs Generative research
The challenge for organisations that don’t already have a research team is helping them recognise their existing design process, and how research fits into how decisions get made.
these organisations all require decisions about ‘what to do’ and ‘how to do it’ in order to deliver the product or service.
Each of those decisions impact the experience that their users have, and ultimately the success of the product.
Although these are design decisions, they are not just made by designers - everyone
The decisions required go beyond just the digital part of a product.
Each decision made introduces an element of risk, and a poor decision will impact the final product.
Companies who make poor products or services tend not to last long.
Good decision making is not just important to private companies trying to make a profit.
In the public sector, in-house teams are also under pressure to demonstrate the value of their work by making useful things - with the threat of being picked apart, and the work being outsourced to the many big conglomerates that hover around public sector contracts if they fail to deliver.
Imagine if there was some way of finding that out before it was built - the team could have built something else instead.
The team making a product are very rarely the same as the people who will be using it
Without outside input until launch, the quality of the product doesn’t improve before launch
Every organisation is interested in its budget - and promising to reduce development costs while reaching a higher quality product earlier is a powerful message that will help get senior leadership enthusiastic about the potential for user research.
Helping people make good stuff and look good
It is not only bad ideas that impact reputation, but also bad implementation - a product that does the thing it is meant to do but is difficult to use, will also reflect poorly on the company.
There’s a reason why all the most prominent tech companies have matured and developed UX practises that include researchers working closely with their product teams - it is because they recognise that understanding users helps avoid these kind of embarrassing mis-steps and helps create great things.
Surveys are quick and cheap to run and feel trustworthy because they contain numbers.
However, they are usually a poor method for finding information that is valuable for informing decisions.
It is definitely not the same as what people will do.
And behaviour is what is important to understand when making successful product decisions.
Most questions the team will want to know will be based around what people are actually going to do.
Surveys are also limited by the imagination of the person writing the survey.
Because surveys do not give information relevant to most of the thing’s product teams need to know, they are not very useful, and using them instead of an appropriate research method will impact the product teams team’s perceptions about how useful user research is.
However, like surveys, on closer inspection, this method is not as useful as it might seem. Because their participants are all people who test multiple websites regularly, they become increasingly unlike other users.
This is a sampling bias
If when watching the session videos an interesting behaviour occurs, there is no ability to probe deeper and understand why.
Analytics are great, and an important part of building a complete understanding of the behaviour of users.
Understanding why the behaviour occurs is essential when making appropriate decisions about what to change.
It is also important to be aware that analytics are only seeing a slice of a user’s experience, and their whole end-to-end journey also included a lot of things that happened off the website, or in the real world.
Building a strategy that combines insight from analytics with understanding of motivations from qualitative studies is a powerful c...
This highlight has been truncated due to consecutive passage length restrictions.
Additionally, analytics can only show behaviour on launched products.
Market research is often an established discipline in many organisations by the time a user research team joins, running focus groups and surveys to help understand people’s opinions about the things being built.
- the method is defined by the research objective, and a mature user research team should be confident at running quantitative studies when required.
The goal of a market research team is different to what a user research team is trying to achieve. A user research team will be trying to learn about the audience so that the product can be crafted in an appropriate way. A market research team will be attempting to learn about the audience so that the product can be described in a way that sounds appealing.
Although a market research team also runs studies, their research objectives and standards are often very different - using methods like focus groups that wouldn’t be appropriate for user researchers for learning deep information about people’s motivations, or creating personas focused around demographics or attitudes rather than behavioural insights.
Although it is definitely true that they will learn after launch if the product is useful for its users, and if users understand how to use it, it is a very expensive way of learning and requires throwing away a lot of work when changes inevitably need to be made. Instead, user research offers a variety of methods to discover whether the concept of the product is useful, and if the implementation is going to be successful before launch, which is much cheaper, and quicker, than waiting until after launch.
To incorporate information learned from research studies into decision making can be considered quite radical at some organisations, particularly to those with a history of deferring to client wishes or listening to the highest paid person in the room. A lot of hard work is needed to bring around change in how people work.
