APPLYING THE SCIENTIFIC METHOD IN
SEO EXPERIMENTING
Branko Rihtman, June 2010
About me – Branko Rihtman SEO Scientist Warzone Experience Becks
http://www.seo-scientist.com
@neyne
http://www.whiteweb.com
1.Define the question
2.Gather information and resources (observe)
3.Form hypothesis
4.Perform experiment and collect data
5.Analyze and interpret data
6.Publish results
7.Retest (frequently done by other scientists)
The Scientific Method
@neyne
Define the question
Usually done after observation of a phenomena
• Sudden change in locations (over-optimization penalty, brand algo, etc.)• Anomalies in SERPs• Someone’s blog post (footer links, second link doesn’t count, etc.)• Social media
• Things to remember:
• You don’t have time to test everything• You don’t have the ability to test everything• What is the ROI of your test ?• Every change you do on your website is
a potential experiment!
"The most exciting phrase to hear in science -the one that heralds new discoveries- is not "Eureka!" but "That's funny...".
(I. Asimov)
@neyne
Gather information and resources • Related blog posts
• Problem: non-standardized terminology• -950 Penalty• Over-optimization penalty• OOP• Unbalanced link equation• Footer link abuse• Sidebar link abuse• Boilerplate linking
• Forums
• Problem: misinterpretation
• Social Media•
@neyne
Perform experiment and collect data
Problems: Lack of control over the system Limited output signal (ranking, title,
snippet) Non-conformity of results No reliable controls
Things to remember:
• Monitor your testing grounds• Testing keyword choice:
• [sdkfsdjwekrwle] – 0 results• [payday loans] – 5.5M results• [translational remedy] – 340K results
• Document everything!
@neyne
SolutionMulti-directional experiments
State A State B Reaction X- Reaction X
change
- change
@neyne
Interpret data and draw conclusions
Does the conclusion agree with expectations ? Does the conclusion have an alternative
explanation ? Does the conclusion agree with other existing
data ? (if not, what was different ?) Bounce the findings off someone
Important: No definite conclusions!!! Only an increasing level of certainty that yourconclusion supports (or defies) the hypothesis.
@neyne
Data Analysis Statistical analysis
Sample sizeSignificance (one and two-tailed tests)Standard DeviationCorrelation
Average is not an end-all measureGet help! Talk to a professional!
People will call you out on crap! (especially if it is self-serving)
.
@neyne
Jean Louis Théodore GéricaultThe Epsom Derby
Avoiding the personal bias
“We observe what we expect to observe, until shown otherwise”Ludwick Fleck
@neyne
Go Social!
Define the question
ForumTwitter
Gather information and resources
Social Media
Publish results
Find an independent blogger you can trust with dataMarket your findings
Interpret data and draw conclusions
Bounce the findings off someone who will not steal themMarket your findings
@neyne
The SEO Testing Secret Ingredient
@neyne
Ethanol
Thank You!
@neyne