A/B and Multivariate Testing Landscape

In our most recent whitepaper we investigated the current state of A/B and Multivariate testing, focusing on how the practice is currently used and the obstacles companies face in implementing this type of an online strategy.

Our analysis, taken from surveys given at the eMetrics Marking Optimization Summit, found that 52% of online marketing managers are currently engaged in A/B or Multivariate testing and that an even greater percentage plan to begin testing within the next year.

A high-level overview of the findings:

  • A variety of testing platforms are considered before making a final decision
  • Experiments vary in frequency, page type and page element
  • Practitioners often use more than one platform to meet their needs
  • Pretest hypothesis often fail
  • Lack of Budget was cited as the main reason by those who currently do not do any testing
  • Complexity of Tools and lack of best practices were two top challenges faced by those engaged in Testing
  • A/B and Multivariate testing is “worth it”
  • Behavioral targeting is a small but growing practice

Surprisingly lack of budget was one of the top reasons why companies were not involved in A/B or Multivariate Testing. But if you look at the chart below you will see that companies that do conduct A/B and Multivariate testing find it that it’s worth it. Majority of the respondents who engage in testing said that either it is worth it or can’t live without it. So if you are one of those who think because of lack of budget you can’t do A/B MVT, think again? Your Return will make it worthwhile to do testing. You will be able to move budget from other areas such as Ad spend to A/B testing. Do your ROI calculations. You will be surprised with the positive results you will get with developing a culture of continuous testing.

You can download the whitepaper from ZeroDash1 website.


Looking to fill your Web Analytics or Online Marketing position? Try WebAnalytics Job Board

New Position

Web Data Analyst at Alzheimer’s Association (Chicago, IL)


Omniture Acquires Touch Clarity and WebTrends Announces Partner Integration

Yesterday Omniture announced that it will acquire Touch Clarity, a Behavioral Targeting Company. Within few minutes a WebTrends announced partner integrations with leading digital marketing vendors, including email marketing firms ExactTarget, Responsys and Silverpop; on-site behavioral targeting firms Kefta and Touch Clarity; and consumer opinion and customer voice firms ForeSee Results and OpinionLab.

This promoted me to write this article because this is exactly what I predicted.

You can read the fill press release at
Omniture Acquires Behavioral Targeting Company Touch Clarity
WebTrends Announces Marketing Lab Partners to Deliver More Measurable, Automated Relationship Marketing

Anybody who read my predictions for 2007 (http://webanalysis.blogspot.com/2007/01/my-predictions-for-2007.html) might have already seen this coming.

Here were my 3 predictions for 2007, and this acquisition proves all 3 of them true. My new comments are in Bold

1. Web Analytics won’t be standing alone – Marketers will want 360 degree view of the customers. Integration of various data sources and tools will be expected from web analytics and other supporting tool vendors. Omniture started the trend with Omniture Genesis, and this will continue, we will see more acquisitions and partnerships similar to Omnitures.

– Both the above press releases confirm that this prediction has come true. Customers are demanding this stronger integration. I wrote a follow-up on this same predictions last week, you can read it here http://webanalysis.blogspot.com/2007/02/follow-up-on-my-web-analytics.html

2. Web Analytics will be about taking actions – More and more marketers would like to take actions and not just report the findings. It just won’t be about what happened, it will be about taking action to drive sales, user satisfaction, lead generation etc. Incentives and bonuses will be tied to the online KPIs. Optimization and Behavioral Targeting will become a common term used by marketers.

– Omniture acquisition of Touch Clarity, a Behavioral Targeting company confirms this prediction has come true too. WebTrends announcing partner integration with Kefta (Optimization and Behavioral Targeting) and Touch Clarity (Behavioral Targeting) confirms that marketers are demanding Optimization and Behavioral Targeting. It is not just about reporting, it is about taking actions. In near future I am expecting to see a lot of case studies about what actions customers took and how it affected their bottom line.

3. Behavioral Targeting – Only few main behavioral network players will be left and some of the existing ones with poor networks will either go out of business or be sold. See my previous article on why size of network matter. Behavioral Targeting won’t exist in isolation. Web Analytics tool will have to support behavioral targeting and visa versa. Also, on-site behavioral targeting will become very common.

– Part of this prediction has come true. I have yet to see any consolidation in behavioral network players. But as the above two press releases confirm “On-Site Behavioral Targeting” is becoming common.

Web Analytics is maturing. Gone are the days when marketers were just satisfied by learning like
“Traffic went up”, “Traffic went down” etc. It is about deeper understanding of the visitors behavior (Web Analytics), segmenting them (Web Analytics and Behavioral Targeting), taking actions based on this deeper understanding e.g. targeting relevant content, products etc. (Email, Behavioral Targeting, Optimization).

So I am 2.5 on my 3 predictions. I have already declared 1 more prediction come true (http://webanalysis.blogspot.com/2007/02/follow-up-on-my-web-analytics_09.html). So for the year I am 3.5/5 so far and the year has just started. I feel good about my predictions.

Questions, Comments???