A/B Split Testing vs. Multivariate Testing with Lance Loveday (5:16)

In this video interview, conversion expert Lance Loveday explains the pros and cons of simple A/B split-testing vs. the much more complex multivariate testing.

We've found, says, Loveday, that A/B testing tends to deliver the largest gain. Of course, he adds, that this could be ABCD testing, that is 4 variations of a single factor, as opposed to testing a large number of factors at one time.

The advantage of A/B testing is that it tends to cause you to rethink the entire user experience as you look for elements to test -- layout, visual hierarchy, messaging, etc. Multivariate testing, on the other hand, often leads to testing smaller incremental variations that don't make as much difference. A/B testing forces you to be more creative about what you're testing.

Loveday explains that his firm's process is to start with some A/B testing to find an effective layout for a client. Once that's established, they'll do multivariate testing to get some additional incremental gains.

A/B testing suits small businesses well because clear results often don't require as many visitors as are needed with multivariate testing. And, A/B testing is a lot easier to execute than multivariate.

With a small business with some in-house tech savvy, a company could do A/B testing itself with a tool like Google Website Optimizer -- or hire a firm to do testing for it.

Loveday encourages viewers to consider Internet sites that allow "crowdsourcing" of designs relatively inexpensively. For a couple of designs that might cost under $1,000, you could test them against your baseline and potentially make some big gains.

Lance Loveday is the president of Closed Loop Marketing, which assists clients through A/B testing, multivariate testing, end-to-end optimization, and coaching and consulting on testing, as well as search engine marketing. This interview was recorded at Conversion Conference West Coast in San Jose, California, May 2010.

Category:

Science & Technology

Tags:

• Lance
• Loveday
• A/B
• multivariate
• testing
• small
• business
• variations
• layout
• hierarchy
• messaging
• incremental
• Google
• Website
• Optimizer
• crowdsourcing
• Ralph
• Wilson
• Web
• Marketing
• Today

[GaoShanSH]

thanks for uploading this. would you like to double-check the sound? The music is way louder than the voice. Also, the voice track doesn't seem clear. Thanks in advance!

[vshindin]

HeadMetrics - cheap and easy a/b testing tool