With the rising cost of traditional advertisements, acquiring traffic has become an expensive activity. Fortunately, there is an effective alternative: to make the most of existing traffic. To do this, AB testing is a remarkable tool. But what exactly is AB Testing and how does it work?
What is AB Testing?
What is AB Testing? Definition
AB testing (or AB test) is a scientific method for comparing two versions of a web page – A and B – and identifying the one that converts best. Version A is the current version of the page (control version), while version B is the modified page (the processing page). The performance of these two pages can be easily compared if tested simultaneously.
The modification of version A must be based on a Hypothesis. This is about how visitors use your site. It may be related to the design, structure or content of your site. By comparing the two versions (of treatment and control), you can confirm or refute your Hypothesis.
Conducting many AB tests is the best way to really understand how designing a page affects your performance. For large e-commerce companies, this process is continuous and involves many versions of each page.
A small story of AB Testing
Testing variables is not new. Some of the best-known scientific ideas have been established by statistically analyzing the effect of certain variables. However, “the examination of statistical hypotheses” in which a treatment situation is compared to a control situation was established at the beginning of the 20th century by statisticians.
1. A scientific method
Examining statistical assumptions has a major advantage. It allows us to calculate the statistical significance (or statistical representativeness) of an effect.
The use of statistical hypotheses to analyze marketing campaigns was a remarkable immediate application. In the advertising world, the first copywriters (designers-writers) like Claude Hopkins have experimented with several ways to test public engagement. Hopkins used the return rate of promotional coupons associated with different versions of an advertising text to measure the impact of different sentences. He described his techniques for testing marketing texts in his book, 1923).
2. A great tool for digital marketing
At the turn of the century, AB Testing was used as one of the main resources of software, e-commerce and online service publishers. The large samples available (offered by free traffic that seemed inextinguishable) and the automatic records led to the widespread adoption of this technique by the main companies.
In an article in the Harvard Business Review, Ron Kohavi (a Microsoft engineer) and Stefan Thomke (Business Administration Professor) highlighted the following examples:
- In 2000, engineers working at Google conducted a test to find the optimal number of results to display on a search engine page. The answer (10 results per page) has remained the same since then.
- In the early 2000s, Amazon discovered that moving credit card offers from its home page to its basket page led to an increase in revenues of tens of millions of dollars a year.
- In 2008 a Microsoft employee proposed that MSN links open in another tab so that the user session is not interrupted. The number of clicks on the MSN home page then increased by 8.9% and the practice became common on the home pages.
- Similarly, in 2012, a Bing engineer conducted a simple test to compare two ways of displaying the titles of advertisements. The winning variant increased revenues by 12%.
Today, companies such as Amazon or Booking.com each perform more than 10,000 AB tests per year.
How does the AB Testing work?
You start an AB test by deciding what you want to test, for example, the color of the Add to Basket button on a produced page.
Then you need to know how to assess performance. In this case, it can be the number of visitors who click on the button (we are talking about micro-conversions here).
To perform the test, you display the different versions (the only difference being the color of the button) to two equivalent user groups (randomly formed when they visit the site) and determine which one had the most impact on your performance indicators. In this example: Which button generated the most clicks?
In real life, many parameters can lead a person to click. This is why randomization is essential in this context, as it minimizes the chances that other factors will disrupt the results.
If the concept seems simple, its implementation requires calculating the sample size necessary to achieve (statistical representativeness).
The example of the color of the button is a very simple example. In reality, AB Testing tools allow you to modify many elements of your website and study the impact of these changes.
What is the difference between Split Testing and A / B Testing?
The Split Testing allows you to test different versions of a page hosted on different URLs, while the AB Testing uses the versions created from the graphic editor of AB Testing software.
So in AB Testing, the visitor stays on the same page which is modified dynamically, while in Split Testing the visitor is re-directed to another page in a transparent manner.
What is the difference between an AB Test, a Multivariate Test, and an A / B / n Test?
A classic AB test allows you to test only two variations of a page from a single variable. When testing multiple versions of the same page simultaneously, a Test A / B / n is performed. In each version, only one element is modified compared to version A. Imagine for example that version B changes the Call-to-Action button while version C changes an image without touching the button.
The Multivariate testing (MVT) works in the same way as the A / B / n Testing but instead of testing the variations one by one, the multivariate testing tests them separately and jointly. The objective of the multivariate testing is to determine which combination of variations is best performing for a given segment of visitors.
In the example above, the pages tested (A, B, C and D) correspond to the following combinations of variables, where X corresponds to the button and Y corresponds to the image present on the page:
A – X1 and Y1
B – X2 and Y1
C – X1 and Y2
D – X2 and Y2
By adding more variables and more versions of each variable, you quickly arrive at an exponential number of pages.
There are advantages and disadvantages with Multivariate testing (MVT), we discuss this subject in our article dedicated to Multivariate Testing.
Why do A / B Testing?
AB Testing is used by many companies to improve the marketing performance of their website:
- E-commerce sites to improve their conversion tunnel,
- Saas software and sites to improve their home page and subscription process,
- Lead generation sites to improve their landing pages.
AB Testing is also used by companies wishing to overhaul the design of their website by testing each change with their users.
How can A / B be tested on a website?
Just about all the visible elements of a website can be A / B tested:
- Titles – They are one of the first and most direct forms of communication with visitors.
- The pages produced – Your pages produced must highlight your offers and allow the user to easily access the basket with an adequate Call-to-Action button
- Texts – Tell a story, tell your visitor how you feel … words count
- Prices – You cannot offer different prices to different customers but you can change the way they are displayed. Simple changes in how you present your prices (using the effect of Serial Position) can have a real impact on the average price of your basket.
- Images – There is evidence that individuals pay more attention to images than to text: this is called the superiority effect of the image. The way a visitor reacts to an image is unpredictable; it is, therefore, necessary to collect data.
- Colors – Google colors are known to have tested the colors of its site
- Forms – Form filling rates vary much more than conversion rates so optimization is the key to success here
You can also test the appearance of your buttons, customer testimonies, the navigation menu, the elements of Social Proof, links, promotions, navigation and layout.
Conclusion
AB testing is a process similar to the experimental processes we all know. However, rather than just observing the effects of a change on a variable, the goal of AB testing is to get the greatest impact possible.
As with any other test, human error prevents convincing results. So, if you are thinking of carrying out AB tests on your website, make sure you do not fall into two very common traps:
- The classic problem of confirming your own opinions.
- Draw conclusions without having enough traffic, time or magnitude for your effect (this is the most common AB Testing error)
Make sure you have reliable and relevant AB tests.
Thrive Optimize is a simple A/B testing add-on for Thrive Architect 
(available with Thrive Suite membership or as a standalone plugin combo) that gives you the power to run fast and highly effective split tests for your landing pages — directly on your WordPress website.
