min read

Keeping Bias at Bay During User Research

by Yvette Simon , 14.06.2012

This article is re-posted from User Centric’s blog.

Having been in the moderator’s seat as well as having witnessed many different and equally good moderation styles, I have come to understand that consistency is paramount when conducting an unbiased interview.  As UX professionals, we understand that variations between sessions could lead to questionable conclusions but what about consistency within a session?

As a moderator, it’s easy to unknowingly communicate expectations during a usability test session.  In doing so, there is a risk of biasing a participant’s experience with the test stimulus.  A bias can be introduced by something as simple as leaning forward when the participant makes an interesting mistake, jumping in a moment too soon with a follow-up question, or choosing to feverishly take notes at certain times and not others.  Participants are aware of these verbal and non-verbal signs and—although they aren’t being paid to do it—often conduct a “within-subject analysis” in the test room.

Keep it consistent

It’s true that the goal of each study dictates its tone.  For example, a focus group session with six participants exploring conceptual web designs calls for a more informal atmosphere and flexibility to go wherever the conversation may lead.  A stark contrast may be a one-on-one medical device validation session which requires little to no moderator interruption, and very strict objectives in need of definitive answers.  Regardless of the study, it’s important to maintain a uniform method of delivery within a session.  This is one way to ensure that a moderator’s own preferences and opinions do not contaminate those of the participant.

"A consistent delivery requires a moderator to maintain the same levels of energy and perceptible engagement throughout a session.”

The test room is exciting because it’s where so many discoveries take place; the ability to capture the true user experience is what makes for meaningful results.  I must admit that I often find myself holding back a grin when a participant is “successful,” and we have all experienced that test session where we fantasize about leaping up, pointing to that large yellow button, and saying, “it’s right there!” A consistent delivery requires a moderator to maintain the same levels of energy and perceptible engagement throughout a session. This doesn’t mean that we should lose the human component and become complete robots in the test room or refrain from making the participant feel at ease.  The key consideration is for moderators to make a conscious effort to temper their reactions and responses each time something comes up, regardless of whether it is new, expected, good, bad, or life-changing.

Getting to the point where a moderator has truly achieved a neutral balance is more of a journey than a switch in state.  Although I’m seeing progress, I can’t quite say I’m there yet; it was just yesterday when I had to stifle a whoop of joy when the yellow button was eventually found.