When Google Allows You To A/B Test Using User-Agents When Google posted a blog post on approved A/B testing methods, they specifically said do not use user-agents when determining what to A/B test. Google wrote, "make sure that you're not deciding whether to serve the test, or which content variant to serve, based on user-agent." So when someone in the Google Webmaster Help forums had a valid reason to do so, John Mueller from Google explained why it may be okay in his case. In short, he wants to test conversions between an m-dot and responsive design for iPad users only. He wants to see if the conversions will drop or increase but is worried if he does user-agent target for iPad users, it may be against Google's guidelines. John Mueller said it should be okay but added some caveats, which I will quote right there: I'd like to say that if this is within a specific category of devices, then it's not a problem doing that kind of A/B testing. That said, there are some weird edge cases that you'd need to watch out for. For example, if you're A/B testing a mobile version on separate URLs vs a non-mobile-version on the same URL, then that can get a bit confusing if things fluctuate back and forth "semi-randomly." If these tests can be completed within a reasonable length of time, or if you're able to do A/B testing within the same category of content that uses the same configuration (for example, A/B testing different m.dot versions of the site), then that's less of an issue. So be careful when doing A/B testing and the best bet, do it for a short period of time. Forum discussion at Google Webmaster Help. http://feeds.seroundtable.com/~r/SearchEngineRoundtable1/~3/XJ-OyzyfcLA/google-a-b-test-user-ag ents-19991.html

When Google Allows You To A/B Test Using User-Agents

Download PDF Report

Upload
searchinsights740
View
30
Download
1

Embed Size (px)

Citation preview

When Google Allows You To A/B Test Using User-Agents

When Google posted a blog post on approved A/B testingmethods, they specifically said do not use user-agents when determining what to A/B test. Googlewrote, "make sure that you're not deciding whether to serve the test, or which content variant toserve, based on user-agent."

So when someone in the Google Webmaster Help forums had a valid reason to do so, John Muellerfrom Google explained why it may be okay in his case.

In short, he wants to test conversions between an m-dot and responsive design for iPad users only.He wants to see if the conversions will drop or increase but is worried if he does user-agent targetfor iPad users, it may be against Google's guidelines.

John Mueller said it should be okay but added some caveats, which I will quote right there:

I'd like to say that if this is within a specific category of devices, then it's not a problem doing thatkind of A/B testing. That said, there are some weird edge cases that you'd need to watch out for. Forexample, if you're A/B testing a mobile version on separate URLs vs a non-mobile-version on thesame URL, then that can get a bit confusing if things fluctuate back and forth "semi-randomly." Ifthese tests can be completed within a reasonable length of time, or if you're able to do A/B testingwithin the same category of content that uses the same configuration (for example, A/B testingdifferent m.dot versions of the site), then that's less of an issue.

So be careful when doing A/B testing and the best bet, do it for a short period of time.

Forum discussion at Google Webmaster Help.

http://feeds.seroundtable.com/~r/SearchEngineRoundtable1/~3/XJ-OyzyfcLA/google-a-b-test-user-agents-19991.html