A/B Split Testing vs. Multivariate Testing with Lance Loveday (5:16)





The interactive transcript could not be loaded.



Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Published on Aug 9, 2010

In this video interview, conversion expert Lance Loveday explains the pros and cons of simple A/B split-testing vs. the much more complex multivariate testing.

We've found, says, Loveday, that A/B testing tends to deliver the largest gain. Of course, he adds, that this could be ABCD testing, that is 4 variations of a single factor, as opposed to testing a large number of factors at one time.

The advantage of A/B testing is that it tends to cause you to rethink the entire user experience as you look for elements to test -- layout, visual hierarchy, messaging, etc. Multivariate testing, on the other hand, often leads to testing smaller incremental variations that don't make as much difference. A/B testing forces you to be more creative about what you're testing.

Loveday explains that his firm's process is to start with some A/B testing to find an effective layout for a client. Once that's established, they'll do multivariate testing to get some additional incremental gains.

A/B testing suits small businesses well because clear results often don't require as many visitors as are needed with multivariate testing. And, A/B testing is a lot easier to execute than multivariate.

With a small business with some in-house tech savvy, a company could do A/B testing itself with a tool like Google Website Optimizer -- or hire a firm to do testing for it.

Loveday encourages viewers to consider Internet sites that allow "crowdsourcing" of designs relatively inexpensively. For a couple of designs that might cost under $1,000, you could test them against your baseline and potentially make some big gains.

Lance Loveday is the president of Closed Loop Marketing, which assists clients through A/B testing, multivariate testing, end-to-end optimization, and coaching and consulting on testing, as well as search engine marketing. This interview was recorded at Conversion Conference West Coast in San Jose, California, May 2010.


When autoplay is enabled, a suggested video will automatically play next.

Up next

to add this to Watch Later

Add to

Loading playlists...