Researcher’s Serenity Prayer

Grant me the serenity to accept the things I cannot test; the option to use qualitative or quantitative methods, and the wisdom to know the difference.

I originally posted an abridged twitter version found here. I meant it more as a joke but as with most jokes, they spark from truth. This came from a recent usability test I have been engaged with and the discussion ‘would our goals benefit more from AB Testing than qualitative research?’ In this particular case, we were asked to perform qualitative research. I am not naturally a fan of AB testing, and yet in this case I wholeheartedly agreed it was the more valuable method.

Why I’m Against AB Testing

I need to start by saying, there is value in AB testing. There are tons of resources explaining how it has been used to determine optimal button placement, language and form layout, and more. There is also a lot of discussion on it being too analytical, taking the human element out of the equation. While I can see myself on either side of this fence, my itch against AB testing stems from a project earlier in my career with an e-commerce client. Tasked to redesign a section of the site, every design decision came down to how to best test it. We were not looking at one simple item such as an image carousel or item details, we were looking at the holistic experience. When AB testing is used on localized sections in this case, it is my opinion you lose the forest for the trees and dependencies and relationships risk being overlooked. Now while this is a localized experience, it has tarnished my views of some quantitative methods.

Benefits of Qualitative Research

Qualitative methods hold more weight in my book. They allow a holistic view and broader understanding, if at the risk of less tangible and sometimes defendable data. While I can’t go into all the benefits of qualitative methods here, they offer a step back from a specific control or solution and look deeper into motivations, goals, and environment. With an HCII and Industrial Design background, I strongly prefer these methods to number crunching with SPSS and other such tools.

When AB Testing Works

AB Testing and quantitative methods can work though. The methods I love, user interviews, shadowing, workshops are invaluable in discussing larger problems and approaching design strategy. When looking at a specific problem, such as attrition rates around a controller or page, quantitative methods work. E-commerce is a prime example of this where the goal is to increase click through and adoption. And I say this with a grain of sand. You cannot AB Test every element of the page. But if you are looking to add a control, and you don’t know the best position, it becomes invaluable. Direct your customers to various options, and watch the numbers come in. This compared to a handful of user studies can give concrete evidence of an optimal solution.

The Serenity Prayer

I wrote the researchers serenity prayer as a joke that while process and methods are good, when applied blindly they lose value. A consultants job is to guide the client to the right methods, to encourage and empower them. Otherwise we are all just making guesses. This is written as a reflection of my recent experience and the need to challenge the methods we use, to break out of comfort zones, and to be ready for rejection – from our teams, our clients, our users, and even our research findings.

Advertisements

3 Responses to “Researcher’s Serenity Prayer”


  1. 1 Find out information on diamond jewellery in Australia July 20, 2013 at 5:15 am

    I comment each time I appreciate a post on a site or I
    have something to valuable to contribute to the discussion.
    It is a result of the passion displayed in the article I looked at.
    And after this post Researchers Serenity Prayer |
    Brain Farks. I was excited enough to post a comment 🙂 I actually do have a few questions
    for you if it’s okay. Could it be simply me or does it give the impression like some of these comments look like written by brain dead visitors? 😛 And, if you are posting at additional sites, I would like to follow everything new you have to post. Could you make a list the complete urls of your shared pages like your linkedin profile, Facebook page or twitter feed?

  2. 2 site August 8, 2013 at 9:29 am

    I think this is one of the most important information for me.
    And i’m glad reading your article. But wanna remark on some general things, The site style is perfect, the articles is really excellent : D. Good job, cheers

  3. 3 praying times January 2, 2018 at 11:58 am

    Hi, Neɑt post. There is an issue together with your
    site in internet expⅼorer, would checқ this? IE nonetheless
    is thе market leader and a huge part ⲟf folks will lеave out your excellent writing due to this
    рroblem.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s




My Work

What I Tweet

Error: Twitter did not respond. Please wait a few minutes and refresh this page.

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 7 other followers

Advertisements

%d bloggers like this: