One of my coworkers did a presentation in our team meeting on a topic he’s passionate about, usability testing. It is also something I’m fascinated by as well.
If you’ve ever done user testing you’ve definitely observed all or most of these issues. I’m very familiar with them but until now didn’t know the terms to describe them.
So here are my notes from the short presentation by Simon Whatley on 9 Usability Testing Biases (which was adapted from Jeff Sauro’s 9 Biases in Usability Testing):
- The hawthorn effect – people act differently because they know you’re watching them
- Task selection bias – if you’ve asked me to do it, it must be able to be done
- Social desirability – users generally tell you what they think they want you to say or avoid saying discouraging things
- Availability – may be rushed or thinking about where they have to go next
- Honorarium if someone gets paid to do a task, they will try to answer questions as they think you want them to, some may cheat to get the money (i.e. fib about their credentials)
- Task importance – if you’re asking me to do it, it must be important, i.e. they don’t really have an opinion on what they’re testing on but will make one up because they think you think it’s important
- Note taking – if I see you write something down then what I did must have been wrong or important, users are aware of the moderator taking notes and it affects their decision making
- Tech-savvy – some people say they are more tech savvy than they are
- Regency and primacy effects, tendency to weight recent events more heavily than older events or vice versa
It’s tempting to think that because of these biases that the results of usability testing are not valid, but that’s not true. Every time I watch a user test software I gain insights into ways it can be improved. Be aware of the biases and try to mitigate them, but above all else, test and do it often!
