09/04/2023 www.rba.co.uk 1
Making Google Behave
Society of Indexers Annual ConferenceRoyal Agricultural University, Cirencester
5-7th September 2014
Karen BlakemanRBA Information Services
[email protected] twitter.com/karenblakeman
Slides available on authorSTREAM, Slideshare and http://www.rba.co.uk/as/
This work is licensed under a Creative Commons Attribution-ShareAlike License.
09/04/2023 www.rba.co.uk 2
Search Engine Market Share June 2014 http://theeword.co.uk/info/search_engine_market.html
09/04/2023 www.rba.co.uk 3
2014 Financial Tables – Investor Relations – Google https://investor.google.com/financial/tables.html
09/04/2023 www.rba.co.uk 4
Google Play App Revenue Doubles in a Year http://mobilemarketingmagazine.com/google-play-app-revenue-doubles
With increased revenue, wearables focus, Google takes fast lane to mobile transition http://tech.firstpost.com/news-analysis/increased-revenue-wearables-focus-google-takes-fast-lane-mobile-transition-227605.html
09/04/2023 www.rba.co.uk 5
Google Brings More "Now" To Search With New Quick Answers http://searchengineland.com/google-brings-more-now-capabilties-to-search-with-new-quick-answers-169658
09/04/2023 www.rba.co.uk 6
Google search
– is free text search
– is not aimed at the serious, in depth researcher
– does not do Boolean
– structured search is limited even in Scholar
– is personalised
– can change minute by minute
09/04/2023 www.rba.co.uk 7
It's Over: The Rise & Fall Of Google Authorship For Search Results http://searchengineland.com/goodbye-google-authorship-201975
09/04/2023 www.rba.co.uk 8
Google Authorship
Without Google Authorship
Five things you need to know about Google search
1. Google personalises your search
Personalises search based on– location – country, town
– past search history
– past browsing activity
– your activity in other areas of Google e.g. YouTube, blogs, images
– your activity on all your devices linked to your Google account
– what other people have clicked on for similar searches
– the device you are using
09/04/2023 www.rba.co.uk 9
09/04/2023 www.rba.co.uk 10
Private browsing - quickest way “un-personalise”search
Chrome - New Incognito window Ctrl+Shift+N
FireFox Ctrl+Shift+P
Internet Explorer Ctrl+Shift+P
Opera Ctrl+Shift+N
Will not remove country personalisation
Not search engine specific, built into the browser
09/04/2023 www.rba.co.uk 11
Five things you need to know about Google search
2. Google automatically looks for variations on your search terms and sometimes drops terms from your search
– Google now tells you which terms it has ignored (some of the time)
– “..” around terms, phrases, names, titles of documents does not always work
– To force an exact match and inclusion of a term in a search prefix it with ‘intext:’
intext:agricultural occupational asthma
– Use Verbatim for an exact match search
09/04/2023 www.rba.co.uk 12
Let’s throw it into the soup and see if the cat licks it up
Google introduces the “soft AND”
“When you do a multi-term query on Google (even with quoted terms), the algorithm sometimes backs-off from hard ANDing all of the terms together.......it’s clear that people will often write long queries (with anywhere from 5 to 10 terms) for which there are no results. Google will then selectively remove the terms that are the lowest frequency to give you some results (rather than none)....Soft AND is a way to reduce the overall frustration and give the searcher something to examine (and with luck, a chance to reformulate their query).”
Dan Russell
http://www.rba.co.uk/wordpress/2011/11/08/dear-google-stop-messing-with-my-search/#comments
09/04/2023 www.rba.co.uk 13
Google – missing terms
09/04/2023 www.rba.co.uk 14
09/04/2023 www.rba.co.uk 15
Google Verbatim
Choosing your search terms
Google automatically looks for synonyms and variations on your terms
– biofuels will find biodiesel, biogas, bio-ethanol etc.
– but do not get the same results if you use biodiesel instead of biofuels
– run separate searches using alternative terms
– no information on how the synonyms are identified or implemented
The terms you use can radically change results– copper mining north wales
– copper extraction north wales
09/04/2023 www.rba.co.uk 16
Choosing your search terms
09/04/2023 www.rba.co.uk 17
09/04/2023 www.rba.co.uk 18
Five things you need to know about Google search
3. Google web search does not search everything it has in its database
– two indexes: main, default index and the supplemental index
– supplemental index may contain less popular, unusual, specialist material
– supplemental index comes into play when Google thinks your search has returned too few results
– Verbatim and some advanced search commands seems to trigger a search in the supplemental index
09/04/2023 www.rba.co.uk 19
“Normal search”1,555,500
Search after Verbatim is applied35,500,000
09/04/2023 www.rba.co.uk 20
Five things you need to know about Google search
4. Google changes its algorithms several hundred times a year How Google makes improvements to its search algorithm
- YouTube http://youtu.be/J5RZOU6vK4Q
09/04/2023 www.rba.co.uk 21
Five things you need to know about Google search
5. We are all Google’s lab rats
Just Testing: Google Users May See Up To A Dozen
Experiments
http://searchengineland.com/just-testing-google-searchers-may-see-up-to-a-dozen-experiments-141570
Mostly minor effects on search but sometimes totally bizarre
results
Google decides that coots are really lions
http://www.rba.co.uk/wordpress/2011/02/12/
google-decides-that-coots-are-really-lions/
Update on coots vs. lions
http://www.rba.co.uk/wordpress/2011/02/21/
update-on-coots-vs-lions/
Hummingbird
Not just an update but a completely new algorithm
Tries to make “sense” of your query and put it into context, natural language queries
Not just search history but also your location, device being used
Announced 26th September 2013 but had already been implemented for about a month
Many aspects had been tested over the previous months and past year
09/04/2023 www.rba.co.uk 22
Menu options change depending on your search
09/04/2023 www.rba.co.uk 23
Google rewrites page titles
09/04/2023 www.rba.co.uk 24
Google's Matt Cutts: Why Google Will Ignore Your Page Title Tag & Write Its Own http://searchengineland.com/googles-matt-cutts-look-title-match-query-190039
http://googlesystem.blogspot.co.uk/2013/11/google-knowledge-graph-gets-confused.html
09/04/2023 www.rba.co.uk 25
EU - so called “right to be forgotten” ruling
09/04/2023 www.rba.co.uk 26
Edition of Monday, January 19, 1998, page 23 - Newspaper - Lavanguardia.es http://hemeroteca.lavanguardia.com/preview/1998/01/19/pagina-23/33842001/pdf.html
EU Court of Justice ruled that Google is a “data controller” under Data Protection legislation and must remove links to information that is “inadequate, irrelevant .... or excessive” from search results on a person’s name.
Information is NOT removed from the web
Subject can apply to have links in search results that point to specific information removed from the results
Not just Google – all search engines with an EU presence
Only applies to searches conducted in the EU + Norway, Switzerland, Iceland and Lichtenstein
Not automatic – subject has to apply and request will be assessed to see if the information is “inadequate, irrelevant or no longer relevant, or excessive in relation to the purposes for which they were processed.”
09/04/2023 www.rba.co.uk 27
How to get around it?
Google now removing results (and also adding back in results!) from searches in European country versions of Google
Indicates on the results page if information has been excluded
Google adds removal statement from all results for searches on personal names even if nothing has been removed
Use non-European Google to see all results e.g. Google.com, Google.ca - but will see country biased results
09/04/2023 www.rba.co.uk 28
09/04/2023 www.rba.co.uk 29
Google Knowledge Graph and carousel
09/04/2023 www.rba.co.uk 30
Google gets it wrong again
09/04/2023 www.rba.co.uk 31
Google gets it wrong yet again
09/04/2023 www.rba.co.uk 32
Google "Henry VIII wives": Jane Seymour reveals search engine's blind spots http://www.slate.com/blogs/future_tense/2013/09/23/google_henry_viii_wives_jane_seymour_reveals_search_engine_s_blind_spots.html
Image courtesy of Will Oremus
Nutrition facts
09/04/2023 www.rba.co.uk 33
Information from Wikipedia and USDA
Compare
09/04/2023 www.rba.co.uk 34
Google Quick Answers
Some Of The Weird Issues When Google's Quick Answers Come From Random Sources http://searchengineland.com/weird-issues-googles-quick-answers-comes-random-sources-197611
“Answers” appear at top of the results page and below the ads
Try– your favourite football club
– flight times
– symptoms of a medical condition
09/04/2023 www.rba.co.uk 35
09/04/2023 www.rba.co.uk 36
09/04/2023 www.rba.co.uk 37
09/04/2023 www.rba.co.uk 38
09/04/2023 www.rba.co.uk 39
And then on another day...
09/04/2023 www.rba.co.uk 40
And one of many wrong Quick Answers submitted to me by delegates of this conference
Many thanks to Philip Stirups for the example. About 24 hours after taking this screen shot Google corrected the error.
09/04/2023 www.rba.co.uk 41
Google commands
Speech marks around phrases or titles of articles “Geochemical evaluation of flowback brine from Marcellus gas wells”
Note: if Google finds no documents, or very few documents, containing your phrase it will ignore the speech marks
Verbatim – runs your search exactly as you have typed it
Search tools, All results, Verbatim (see earlier slide)
intext: before your term – term must be present and exactly as you have typed it
UK public transport intext:biodiesel
Google commands
Think file format– PDF for research papers, lengthy documents, government
reports, industry papers
– ppt or pptx for presentations, tracking down an expert on a topic
– xls, xlsx or csv for data and statistics
filetype: command zeolites environmental remediation filetype:pdf
"north sea" deep water drilling filetype:ppt
"north sea" deep water drilling filetype:pptx
09/04/2023 www.rba.co.uk 42
Google commands
Site searchFor searching large websites, or groups of sites by type for example government, NHS, academic
Can exclude sites using -site:
agricultural occupational asthma UK site:nhs.uk
agricultural occupational asthma UK site:ac.uk
agricultural occupational asthma UK site:gov.uk
agricultural occupational asthma UK site:gov.uk –site:hse.gov.uk
09/04/2023 www.rba.co.uk 43
Google commands
Numeric range search
Anything to do with numbers and quantities: years, temperatures, weights, distances, prices etc
Use the advanced search screen or type in your two numbers separated by two full stops as part of your search
Rhodes zeolites environmental remediation 2005..2014
toblerone 1..6 kg
09/04/2023 www.rba.co.uk 44
09/04/2023 www.rba.co.uk 45
Google commands
Words in the title – can be single words or phrases
Ensures subject is the main focus of the article
Use advanced search screen or intitle:intitle:”diabetic retinopathy”
Words in the URL – can be single words or phrases
Use advanced search screen or inurl:inurl:”diabetic retinopathy”
There are allintitle and allinurl commands but they are unreliable and do not work with other commands
define
09/04/2023 www.rba.co.uk 46
09/04/2023 www.rba.co.uk 47
Google search options
Date
Restrict your results to information that has been published within the last hour, day, week, month, year or your own date range
Search tools, Any time and select an option
09/04/2023 www.rba.co.uk 48
Google search options
Reading level
Changes the type of material that is returned
Run the search and from the menu above the results select Search tools, All results and then Reading level
09/04/2023 www.rba.co.uk 49
Google search options
Google Scholar
http://scholar.google.com/
“Google Scholar provides a simple way to broadly search for scholarly literature. From one place, you can search across many disciplines and sources: articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites. Google Scholar helps you find relevant work across the world of scholarly research”.
• Search all scholarly literature from one convenient place
• Explore related works, citations, authors, and publications
• Locate the complete document through your library or on the web
• Keep up with recent developments in any area of research
• Check who's citing your publications, create a public author profile
09/04/2023 www.rba.co.uk 50
09/04/2023 www.rba.co.uk 51
Google Scholar
Does not cover all key journals in all subjects – no source list
Top publications for subjects and languages under Metrics link on home page or http://scholar.google.co.uk/citations?view_op=top_venues&hl=en
Scholar indexes the full text but you may have to pay to view the whole article
Groups different versions of an article together
09/04/2023 www.rba.co.uk 52
Google Scholar
Includes open access material, pre-prints, institutional repositories (but not necessarily author self archived repositories)
Includes material that is NOT peer reviewed but is structured and looks like an academic article (title in large font, authors, affiliations, abstract, keywords, citations)
Pre-prints and IR copies may differ from final published version – charts and images may be redacted because of copyright restrictions
09/04/2023 www.rba.co.uk 53
Google Scholar
Does NOT use the publishers’ metadata
Date and author search looks in the area of the document where those elements are usually found
Page numbers, part of an address, data item may be mistaken for publication year
Sometimes gets the author wrong
09/04/2023 www.rba.co.uk 54
Google Scholar verified author and citations
09/04/2023 www.rba.co.uk 55
Google Scholar advanced search commands
Use advanced search screen or commands as follows:
+ sign before a search term to force an exact match, for example +norne
“....” around phrases for example “environmental remediation”
intitle: to search for a single word in the title, for example intitle:zeolites environmental remediation
allintitle: to search for all of your terms in the title, for example allintitle:zeolites environmental remediation
author: to search on an author’s name, for example
zeolites environmental remediation author:rhodes
site: to limit your search to specific institution for example
marcellus shale site:psu.edu
Commands can be combined for a precise search, for example
author:wolford site:psu.edu allintitle:marcellus shale
Patents
https://www.google.com/patents
Coverage:– US
– Canada
– European Patent Office (EPO)
– Germany
– China
– World Intellectual Property Organisation (WIPO)
Patents available in original language and English (Google Translate)
09/04/2023 www.rba.co.uk 56
books.google.com
Includes magazines, journals, archives of newspapers (mainly US)
09/04/2023 www.rba.co.uk 57
Google Books Ngram viewer
http://books.google.com/ngrams/
09/04/2023 www.rba.co.uk 58
Google Books Ngram viewer
http://books.google.com/ngrams/
09/04/2023 www.rba.co.uk 59
09/04/2023 www.rba.co.uk 60
Google Trends http://www.google.com/trends/
09/04/2023 www.rba.co.uk 61
Google Trends http://www.google.com/trends/
Google Cultural Institute http://www.google.com/culturalinstitute/about/
09/04/2023 www.rba.co.uk 62
Google Art Project http://www.googleartproject.com/ http://www.google.com/culturalinstitute/project/art-project
09/04/2023 www.rba.co.uk 63
Google ArtProject museum view
09/04/2023 www.rba.co.uk 64
09/04/2023 www.rba.co.uk 65
1. Important to keep up to date with what Google and the other search engines are doing, and with regulatory changes
Search Engine Land http://searchengineland.com/ Search Engine Roundtable https://www.seroundtable.com/Search Engine Watch http://searchenginewatch.com/
2. Get to know Google commands and where the search options are hidden
3. Never, never, never trust the “answers” on Google’s results pages