What to Test

2.1 Deciding What to Test

There’s A LOT more to A/B testing than making a Mr. Potato Head out of your landing page, and testing without a game plan isn’t likely to get you ahead of the curve. Instead, you can gain clarity and direction by testing against a research hypothesis. Don’t worry. This is a lot simpler than it seems.

A hypotheses in this case merely refers to your educated guess about what will make your page perform better. The best hypotheses are based on background research and analytics, however in some unique cases you can create a hypothesis from gut feel.

The background research we’re talking about here falls into two categories.

  1. The (Quantitative) Data Approach
    • This method relies on stats like bounce rate, exits, conversion starts, and funnel analysis. Tools for this method include Clicktale, Google Analytics, Omniture, and CrazyEgg.
  2. The People Approach
    • This method involves direct, qualitative feedback from customers and page visitors. You can use tools like Qualaroo and SilverBack to ask multiple choice and long answer questions to your users. The idea is that you can then cater directly to their comments and concerns.

Of course, the most beautiful hypotheses in this world will have a combination of these two approaches.

A typical one-two punch is to take a look at your data first, then to move ahead with a people approach if you need more clarity, or if the numbers just aren’t talking to ya.

CASE STUDY

A useful case study from our own experience at Unbounce is our Pay With a Tweet vs. Pay With an Email Case Study. After creating a viral landing page for an ebook giveaway, Unbounce’s Oli Gardner added Qualaroo on the page to survey visitors on whether they’d rather get the ebook in exchange for their email address, or a tweet.

2.1-Pay-with-a-tweet

After some time, the number of visitors claiming they would rather provide their email address than a tweet in exchange for the ebook rose to 45%.

That’s when Oli decided to run an A/B test. First step; creating a test hypothesis…