BY: MOTIVATE DESIGN
Bias is an unavoidable part of doing any kind of research. Users will always be biased to some degree in their feedback and UX designers will always receive and filter that feedback through their own set of biases. This is an important point to understand because it’s easy for researchers to fall into the trap of believing they can construct methods for gathering purely objective feedback. The more they recognize how bias affects their research, the better positioned they are to account for and minimize its impact.
While psychologists have categorized many different types of bias, confirmation bias poses particular problems for UX designers. Confirmation bias is the tendency to adopt a specific view or interpretation and then place outsized value on information that confirms that belief while downplaying or ignoring data that contradicts or doesn’t support it. Fundamentally, confirmation bias is driven by a flawed selectivity that not only dictates which data should be paid attention to, but also how it should be interpreted or processed.
The danger of confirmation bias is that it can very easily distort a person’s perspective by eliminating alternative possibilities and delegitimizing disagreement. For example, confirmation bias is one of the biggest contributing factors to political polarization in the US. Political partisans are increasingly consuming media that reinforces their preexisting beliefs, preventing them from understanding or sometimes even seeing opposing viewpoints.
Over time, confirmation bias can create an echo chamber that makes change or growth increasingly difficult because it’s no longer possible to identify where problems exist. For UX designers, it can lead to stagnation and leave them unable to adapt to changes they never saw coming or address critiques they never knew existed.
Tips For Avoiding Confirmation Bias in UX Design
1: Ask Better Questions
One of the biggest sources of confirmation bias in UX research is the use of leading questions in the feedback process. Designers rarely ask questions without a reason. Their questions are based on internal assumptions about an issue or problem, many of which may have nothing to do with the actual user’s experience with a product.
Take, for instance, an app that isn’t performing as well as designers anticipated. When they (sensibly) decide to collect collect user feedback on the program, they want to find out why people are having difficulty using the app. Because the team spent a substantial amount of time debating the navigation design, the survey may ask the following question:
“Did the navigation menu make it difficult for you to find the specific page you wanted to visit?”
There are a couple of problems with this question. In the first place, it’s narrowing the scope of the problem by making a suggestion to the respondent rather than allowing them to articulate the problem in their own terms. Secondly, it imposes two assumptions upon the reader, that they’re having trouble finding pages and that the navigation menu must be responsible. Even if that’s true, it may not be the user’s biggest problem with the app.
A non-leading question might look like this:
“Did you have difficulty finding a specific page you wanted to visit? If so, why?”
This question contains no pre-existing biases. It recognized that navigation may be a contributing factor in the app’s poor performance, but it doesn’t make any assumptions about the exact nature of that problem and allows the respondent the space to provide a more informative answer. Remember, collecting feedback should be about understanding the user’s experience, not confirming the designers’ assumptions about that experience.
2: Diversify Your Feedback
If UX designers collect feedback from users who are similar them (be it in terms of race, class, education level, sexual orientation, etc), they’re likely to receive responses that reinforce their existing biases. Any good UX research should collect data from a diverse pool of respondents. While it’s not always possible to enlarge that pool, special care should always be taken ensure that it features as many unique perspectives as possible. Traditional feedback mechanisms, such as surveys and focus groups, are so familiar to many users that they often provide predictable results even when they are designed with diversity in mind. Newer approaches based more on organic conversation rather than neutral interviews may help some designers to greatly diversify feedback.
3: Seek Out Disagreement
The greatest danger of confirmation bias is that it discounts or ignores contrary views. If a survey finds that 75% of users find a product intuitive, for example, there is real value to understanding why a quarter of users think otherwise. User feedback is not intended to reinforce what designers think they already know; it should identify problems they didn’t anticipate or expose them to issues they hadn’t considered. This is not not say that all disagreement is created equal. It could well be that some contrarian feedback isn’t useful for improving a product or service, but designers will never know if they don’t actively seek out disagreements and try to understand them.
4: Look at Quantitative Data
While there are all sorts of ways that data can be selectively interpreted to reinforce confirmation bias, quantitative data points also provide a good way to test out those assumptions and see if they’re supported by hard evidence. If user feedback suggests that people like a product and are willing to use it but the data clearly demonstrates that they’re not actually using it, that should be a sign that there’s a problem somewhere in the feedback process. Perhaps designers are asking the wrong questions and merely reinforcing their own design assumptions rather than catering to what users want. This approach to design can easily result in elegant, well-designed products that nobody actually uses. By tracking actual usage data with quantitative metrics, designers can get a sense of what questions they need to be asking to get the qualitative data that explains these trends.
Bias presents serious challenges for UX designers, but with a little forethought, there are many ways to avoid its effects. By understanding how bias on the part of both designers and users can impact the feedback process, it’s possible to design better methodologies to collect the kind of actionable data that leads to better products and services.
