For Reservation Call : 1.888.222.5847


Home » Conversational » Websites » A/B Testing

A/B Testing

I do A/B testing, or split testing, on individual pages. I like to call it page testing, just because I focus on individual pages rather than groups of pages or whole sites.

What is an A/B test?

An A/B test is when you split visitors into two groups. Each group gets a different version of your page (or site). In each version, you measure the same behavior as a goal that tells you which version performs better.

In my case, I focus on pages. I assess how the page is written currently and examine all the places where I think I can improve the message. I rewrite the content in an experimental copy and set it up for a test. Then, I run a test to see which page gets people to interact more effectively with the page based on one or more chosen goals.

Different kinds of research

This kind of testing is meant to give us an answer during causal research, but I have used it for descriptive research as well.

I learned through my business analytics certification that there are three kinds of research. Exploratory research is when you gather data. You brainstorm what might be important to gather and hope to find something that seems useful enough to analyze in more detail. Descriptive research is when you analyze data more thoroughly. It’s when you study what you’ve gathered to come up with educated guesses about what is happening and how to change it. Causal research is when you put your educated guesses to the test to confirm or dismiss each guess.

Your website visitor behavior analytics should take care of exploratory and descriptive research phases for you. However, some data can be ambiguous, or you might not have all the data you need. You can use your testing tool to create new ways to gather data or ways to clarify the data you have. Then, you can finally get to the causal phase and run a test.

Testing to get your message right

It’s important that you develop a solid hypothesis to test. That means you commit to one of your educated guesses and create an experimental design based on it. Both versions must include the same goal on the page somewhere. if your new design creates a new goal that you want to test, it won’t work, since the original can’t even measure the same goal.

It’s important to track all relevant behaviors, even if they will may not decide the outcome. I had one group question test results, because they had data outside of the test that contradicted results. It turned out there were multiple pathways to what they were tracking and the test only measured one of those pathways. If we had tracked all pathways, we might have had a more effective analysis in the end.

It’s important to consider that there may be no difference in how visitors respond or that visitors do the exact opposite of what you expect. The first is called the null hypothesis. The second, I call the antithesis. I limit my tests to 12 weeks in light of the null hypothesis. A test could run on and on and never reach a conclusion, so you have to have an end point somewhere. Even if the test doesn’t reach a reasonably sure result, the data can be helpful.

How to run an effective A/B test

I gave a presentation December 2018 about how to run an effective A/B test. It focuses on lead generation and longer term engagement cycles. However, there are plenty of lessons to share with just about anyone who is interested in A/B testing. Perhaps you might learn something that will help you with your website.

Watch my presentation on how to run an effective A/B test (Adobe Connect, 38 mins)