Upload
distilled
View
3.895
Download
0
Embed Size (px)
Citation preview
@jondmyers @DeepCrawl
Mobile-First Preparedness- what we've learned from crawling the top 1 million websites.Jon Myers – Chief Growth Officer
@jondmyers @DeepCrawl
Source: http://glooadvertising.com.au/is-your-website-mobile-ready/
@jondmyers @DeepCrawl
Google is our world…
@jondmyers @DeepCrawl
Google is our world…
2 Trillion searches per year!
@jondmyers @DeepCrawl
Google is our world…
15% of searches new each day!
@jondmyers @DeepCrawl
Google is our world…
60% of search are on Mobiles!
@jondmyers @DeepCrawl
Really??...
@jondmyers @DeepCrawl
Sooooooo 2015! And before!
@jondmyers @DeepCrawl
@jondmyers @DeepCrawl
Google is thinking about it...
@jondmyers @DeepCrawl
@jondmyers @DeepCrawl
Helpful to see…
@jondmyers @DeepCrawl
@jondmyers @DeepCrawl
1 page at a time could take a while!…
@jondmyers @DeepCrawlSource: http://shop.adamjk.com/product/1-page-at-a-time
@jondmyers @DeepCrawl
History of my devices…
@jondmyers @DeepCrawl
History of Mobile…
WAP
HTML
1999 2007 2009 2014 2015 2017
Separate Mobile
Pages (Mobile Site
or Dynamic
Delivery)
Responsive
Design
Deep App
Linking
AMP &
Progressive
Web Apps
Mobile-
first
Indexing
@jondmyers @DeepCrawl
Time to get SH*T done…
@jondmyers @DeepCrawlSource: https://www.successimpulse.com/products/less-talk-more-action?variant=44575800839
@jondmyers @DeepCrawl
Heavy Data ahead…
@jondmyers @DeepCrawlSource: https://betanews.com/2015/09/14/8-early-warning-signs-of-problems-in-your-data-governance-plan/
@jondmyers @DeepCrawl
Being Majestic…
@jondmyers @DeepCrawl
Source: https://lafleur.marketing/blog/six-mad-men-marketing-tips-don-draper/
@jondmyers @DeepCrawl
What we did, why? And only we could…
@jondmyers @DeepCrawlSource: https://markfisherfitness.com/what-could-have-been/
@jondmyers @DeepCrawl
History of Mobile SEO Configuration…
Desktop
Mobile
Responsive
AMP pages
Mobile first
One version for desktop devices
One to rule them all - one version designed to work equally good on desktop and mobile
Dedicated light weight version designed for a fast loading
Dedicated mobile pages served on a separate URL e.g. m.domain or dynamically served on the same URL
Mobile becomes the PRIMARY version.
@jondmyers @DeepCrawl
Types of Mobile Configuration…
1. No Mobile Configuration (desktop only)
2. Responsive
3. Dynamic
4. Desktop + Dedicated Mobile
5. Desktop + AMP
6. Responsive/Dynamic + AMP
7. Desktop + Dedicated Mobile + AMP
8. Responsive + Dedicated Mobile + AMP
9. Responsive + Dynamic + Dedicated Mobile + AMP
10. Mobile/AMP only
11. Mobile + AMP
@jondmyers @DeepCrawl
Impact of Mobile-first…
-CRITICAL- - Desktop only (Heavily Affected)
-HIGH PRIORITY- - Desktop + Dedicated Mobile (Affected)
-HIGH PRIORITY- - Dynamic (Affected e.g. Content Issues)
-LOW PRIORITY- - Responsive (Not affected)
Impact of Mobile-first on different Mobile Configurations:
@jondmyers @DeepCrawl
Responsive Dynamic Separate
Mobile
Would expect Google to use both
user-agents on the same URL to
validate that the same content is
returned.
Google needs to crawl with both user
agents to validate the mobile version.
Hint: Use the Vary HTTP header!
Google needs to crawl the dedicated mobile URLs with a mobile user agent to validate the pages and confirm the content matches the desktop pages.
How does Google crawl different configurations?
@jondmyers @DeepCrawl
Responsive design is the optimal situation…
“Responsive design is pretty much the optimal situation because regardless of which device we use to crawl the site we will see the same thing.
We’ll get the same content and the structured data, the same videos, the same images and the same internal links. It’s all there.”
Reference: https://youtu.be/7Aq9bFdfMuE?t=54m53s
@jondmyers @DeepCrawl
How does the world look?...
Source: http://abcnews.go.com/topics/news/doomsday-clock.htm
@jondmyers @DeepCrawl
How are sites set up for mobile?...
@jondmyers @DeepCrawl
Mobile Configuration Breakdown…
Breakdown of all sites where mobile configuration could be identified
Significant no. dynamic sites – work to ensure meta data is kept consistent across different versions of page.
Responsive design dominates -Path of least resistance moving to Mobile-first Index
Separate Mobile unpopular –high maintenance to ensure desktop/mobile equivalence
15,000 sites
190,000 sites
795,000 sites
@jondmyers @DeepCrawl
Separate Mobile Sites – Are they up to scratch?...
DesktopPage
Mobile Page
Rel=canonical
61% of Separate Mobile sites have correct canonical to desktop page.
@jondmyers @DeepCrawl
Separate Mobile Sites – Are they up to scratch?...
DesktopPage
Mobile Page
No canonical
21% of Separate Mobile sites had no canonical to desktop equivalent.
@jondmyers @DeepCrawl
Separate Mobile Sites – Are they up to scratch?...
DesktopPage
Mobile Page
Other page?
18% of Separate Mobile sites had canonicals pointing to wrong page
@jondmyers @DeepCrawl
Separate Mobile Sites – Desktop/Mobile Equivalence…
Of 15,000 Separate Mobile sites returning 200 response codes:
DesktopPage
Mobile Page
Matching title & meta descriptions
DesktopPage
Mobile Page
Matching meta descriptions
DesktopPage
Mobile Page
Matching title
7.7%
7.9%
5.8%
@jondmyers @DeepCrawl
Separate Mobile Issues…
1%of separate mobile sites had noindex
meta tag
25% of all Separate Mobile sites
returned a non-200 status code
21% returned 3xx redirection
2% Separate Mobile sites returned
4xx errors
1% returned 5xx errors
@jondmyers @DeepCrawl
Google pushing for responsive web…
https://webmasters.googleblog.com/2017/09/how-to-move-from-
m-dot-urls-to.html
Separate Mobile
Google needs to crawl the dedicated mobile URLs with a mobile user agent to validate the pages and confirm the content matches the desktop pages.
@jondmyers @DeepCrawl
Order of Mobile-first Indexing…
“If you have a vary header which returns a different page for a mobile user agent, Google will use that as the mobile page instead of the responsive page.”
Reference:https://www.youtube.com/watch?v=8NyZypIfOzI&t=36m31s
@jondmyers @DeepCrawl
Dynamic configurations…
Initial analysis found ?% of websites that serve
different content based on user agent do not have
a Vary: User-agent header.
Dynamic – same URL, different HTML
dependent on user agent
@jondmyers @DeepCrawl
Let’s talk about speed…
Source: https://play.google.com/store/apps/details?id=com.ea.game.nfs14_row
@jondmyers @DeepCrawl
Fetch times – The need for speed…
Percentage of sites split by fetch time
Fetch time includes time taken to fetch URL and display the HTTP response. This doesn’t
include the time taken to request or run any associated resources (such as images or scripts) on
the page.
@jondmyers @DeepCrawl
But are the top million sites really that fast?...
@jondmyers @DeepCrawl
Fetch times – Configuration breakdown…
Dynamically servedResponsive Separate Mobile
Average fetch time by configuration (secs):
Separate mobile sites have the slowest fetch times.Responsive sites have fastest fetch times of
three mobile configurations.
@jondmyers @DeepCrawl
Fetch time with mobile desktop user agents…
19% of sites had a
fetch time between 1-
2 seconds.
68% of sites had a
fast fetch time – below
one second.
13% of sites had a
slow fetch time
exceeding 2 seconds
@jondmyers @DeepCrawl
What does Google have to say about site performance?...
Source: https://www.thinkwithgoogle.com/about/
@jondmyers @DeepCrawl
Google research – Performance per vertical…
Source: https://www.thinkwithgoogle.com/marketing-resources/data-measurement/mobile-page-speed-new-industry-benchmarks/
Consumer sites
have faster more
responsive web
servers.
Business and
finance lagging
behind.Technology my
arse!
@jondmyers @DeepCrawl
Google research – Performance per vertical…
Source: https://www.thinkwithgoogle.com/marketing-resources/data-measurement/mobile-page-speed-new-industry-benchmarks/
Google says to aim for 2.4 seconds load time for a
page!!!???!!
@jondmyers @DeepCrawl
Lets Talk Page Size…
@jondmyers @DeepCrawl
Google research – Page Size…
Source: https://www.thinkwithgoogle.com/marketing-resources/data-measurement/mobile-page-speed-new-industry-benchmarks/
According to Google, sites
across all verticals are on
average larger than the
500kb recommendation.
Again, technology sites
aren’t leading the way.
@jondmyers @DeepCrawl
What did we find out about HTML size?...
94%<=200k
5%200k - 500k
@jondmyers @DeepCrawl
Accelerated Mobile Pages…
Percentage of sites with AMP version of homepage:
Responsive0.63%
Dynamic
0.61%Separate Mobile
0.86%
@jondmyers @DeepCrawl
What about HTTPS?...
@jondmyers @DeepCrawl
HTTPS – How many have made the switch?...
HTTPS
(22%)
HTTP
(78%)
Has HTTPS reached critical mass? Our data
says there’s a long way to go for the top one
million sites.
This figure is much lower than currently reported
Mozcast – 68% of first page results have SSL
encryption.
Source: http://mozcast.com/features
@jondmyers @DeepCrawl
HTTPS – Adoption across mobile configurations…
Dynamically served sites appear
to be particularly slow to adopt
HTTPS.
Surprisingly Separate Mobile sites
leading the way for the switch to
HTTPS.
@jondmyers @DeepCrawl
HTTPS doesn’t end there…
Watch out for non-secure form fields (we
found a lot of them) as Chrome is now
flagging these with security warnings.
12.7% incorrectly configured sites with links
between protocols and some sites with mixed
content e.g. a HTTPS page with reference to a
script on an HTTP URL.
@jondmyers @DeepCrawl
Carving up Configurations by Country…
@jondmyers @DeepCrawl
Splitting the Majestic Million by TLDs…
Percentage of sites per TLD
@jondmyers @DeepCrawl
.edu & .gov are leading the way with responsive design…
@jondmyers @DeepCrawl
…and .co.uk TLDs aren’t far behind…
ResponsiveDynamic Separate
@jondmyers @DeepCrawl
What next? Plenty of issues…
Non Secure Form Fields
Pages without Valid Canonical
Tag
Failed URLs
Empty Pages
Max Fetch Time
Thin Pages
Missing H1 Tags
Non-301 Redirects
Pages with Duplicate Body
Duplicate Pages
Malformed URLs
High External Linking
Max URL Length
Missing Titles
Pages with Duplicate Titles
No Descriptions & No Snippets
Unauthorised Pages
5xx Errors
Max Links
Broken Pages (4xx Errors)
Max Description Length
Short Titles
Unlinked Paginated Pages
Hreflang to Non-200 URLs
Non-rel Alted AMP Pages
All Broken Links
Non-reciprocal Mobile/AM
Duplicate Description Sets
Max Content Size
Uncategorised HTTP Response Codes
Max Title Length
Duplicate Body Sets
Duplicate Page Sets
Pages with Duplicate Descriptions
Canonical to Non-200
Max Redirections
Max HTML Size
Short Descriptions
Redirect Loops
Duplicate Title Sets
And more…
Non-200 Mobile/AMP
Excessive Redirects In (Admin Only)
Mobile Links Out Mismatch
Mobile Links In Mismatch
Mobile Word Count Mismatch
Mobile Content Mismatch
Duplicate Pages including Primary
@jondmyers @DeepCrawl
In a nutshell…
@jondmyers @DeepCrawl
Separate Mobile sites are poorly configured
@jondmyers @DeepCrawl
Dynamic sites need to check they have Vary HTTP response header…
@jondmyers @DeepCrawl
HTTPS adoption relatively low amongst top million sites…
@jondmyers @DeepCrawl
Fetch time is surprisingly low???…
@jondmyers @DeepCrawl
Watch this space for an even deeper dive into the data…
Source: http://dive-bohol.com/
@jondmyers @DeepCrawl
But what does the future hold?...
@jondmyers @DeepCrawl
Go beyond Responsive with PWA’s…
Progressive Web Apps is:
• Progressive
• Responsive
• App-like
• And more...
Progressive Web Apps can fall back to AMP. Hint… first load as AMP.
You should think about it now.
Progressively Web App is THE FUTURE.
@jondmyers @DeepCrawl
Jon MyersDeepCrawl
Chief Growth Officer