I remember a discussion with the UX director at one of Sweden's leading tech companies. We discussed products and usage. And I asked her why they had been so successful. "It's all about testing." She was obsessed with testing design and usability from step one in the development process to the final product.

Her favorite way of testing was A/B testing. It was easy to analyze the results and easy to create the tests. And by choosing the right alternatives, they had learned to extract lots of data from the analyses.

A/B testing is a simple way to compare two alternatives. From the comparisons, you can get different responses to how many respond or how quick they respond or the quality of response. The method is almost 100 years old and was at the time not designed for Internet ads ;-). Many of us have been using pairwise comparisons, QFD, or conjoint analysis. The A/B testing is the simplest case in these statistical methods.

American Express perfected their direct marketing campaigns using A/B testing in the 80s and 90s.

The fundamental math behind the model is the same, but the real-time environment makes it more powerful. Today multiple tools are available for testing websites and ads on the web, making the tests faster, cheaper, and more powerful.

Use the A/B testing when you want to: