How many here follow Formula 1 or another form of auto racing? In our driving, most of us do not look much beyond the hood of the vehicle that we’re driving. Race car drivers are trained to lift their eyes up and look into the distance, sometimes a couple of turns ahead (if they are smart and have studied the course).

That is my objective with this talk, to get us all to lift our gaze a bit above our past and current experiences with search engines and consider what is ahead and how we will manage this four our ourselves and our users.

1

This is some of what we will be looking at over the next hour. Please ask questions along the way.

2

Search engines do not look or act like the ones we started out with in the 1990’s. Yet, so many of the SEO practices that are still in use today come from that era. And, for some strange reason, everyone thinks that they can do SEO.

In the 90’s, search was a task.In 2011, search is an experience.

In the 90’s search was directed by the search engine.In 2011, search is directed by the user.

And we’re still not finding what we’re looking for most of the time. I believe that this is because we as information architects and user experience designers are not treating search as in experience. We are still treating it as a task managed by the machine.

““Information is like taxis in New York: it seems to be all over the place, and then you can never find it when you need it. But the problem isn’t just the raw volume; we’ve collapsed all these channels and categories that used to be distinct, so that nothing is where it’s supposed to be. It’s as if we’ve torn down the walls of the library, and now the reading room is full of street people.” Geoffrey Nunberg, New York Times, March 20, 20911

3

I always start with a review of how search engines actually work. It is a good reminder that the foundation for their functionality is quite old, dating back to the 1960’s and early document retrieval. Remarkably, the search engine stores a copy of the query terms in a searchable index and even retain a copy of the page in another index.

This was much easier when the Web was a mere 15 million pages in 1997 and considerably harder now in the dynamic Web that is estimated to have topped 1 trillion URLs last year. Google has claimed to have an index in excess of 125 billion pages, this is quite a lot considering the storage required. However, it is still less than 20% of the pages out there. Who gets into the index and why?

4

There are 2 kinds of searches, navigational and informational

How we look for information is different between people and between people and machines.

Humans are limited by their ignorance. We don’t know what we’re looking for much of the time and so do not know how to find it. We often rely on technology to provide parameters to narrow our scope and put us on the right track. Unfortunately, technology is “face value” and so does not know how to interpret our queries. Does not understand that we can have a single word mean multiple things (order a meal, put things in order) or multiple terms mean the same thing (star: celestial entity, celebrity)

5

This was recently put to the test in the US with an item that caused an uproar. A woman wants to buy designer eyeglasses and save money. She chooses the #3 result on Google. The frames that are delivered are obviously fake. When she returns them for refund, the owner of the business responds with harassment and threats.

To the customer, relevant means honest and high quality. To Google, relevant means many links and many, many social media mentions. What the search engine did not understand is that most of the mentions were warnings of bad quality and service.

When the story came to light, Google’s response was that they would “tune” their sentiment algorithm.

6

While meant as a joke, what they are referring to is a…librarian. Larry Page once said that the perfect search engine would be a reference librarian with a complete mastery of the entire corpus of human knowledge.

From the actual job description…Are you a student or a new grad? Visit our student siteAutocompleter at Google Mountain View

Autocompleter – Mountain ViewThis position is located in Mountain View, CA and obscure locations around the world

The area: Product QualityThe Product Quality team ensures that Google has the best worldwide product offerings by analyzing, positioning, packaging and promoting our solutions across a variety of countries and markets where Google does business. The team works closely with the engineeri ng group to continuously improve the search experience.

The role: AutocompleterAre you passionate about helping people? Are you intuitive? Do you often feel like you know what your friends and family are thinking and can finish their thoughts before they can? Are you an incredibly fast Google searcher? Like, so fast that you can do 20 searches before your mom does 1?

Every day people start typing more than a billion searches on Google and expect Google to predict what they are looking for. In order to do this at scale, we need your help.

Google's quality team is looking for talented, motivated, opinionated technologists to help us predict what users are looking for. If you’re eager to improve the search experience for millions of people and have a proven track record of excellence, this is a project for you!

As a Google Autocompleter, you’ll be expected to successfully guess a user’s intention as he or she starts typing instantly. In a fraction of a second, you’ll need to type in your prediction that will be added to the list of suggestions given by Google. Don’t worry, after a few million predictions you’ll grow the required reflexes.

Responsibilities:Watch anonymized search queries as they come in to Google. Predict and type completions based on your personal experience and intuition. Suggest spelling corrections when relevant. Keep updated with query trends and offer fresh suggestions. Requirements:Excellent knowledge of English and at least one other language. Excellent knowledge of grammatical rules (e.g. parts of speech, parsing). Understanding of the search engine space. Proven web search experience. Good typing skills (at least 32,000 WPM). Willingness to travel (in order to provide local autocompletions) or relocate to obscure places like Nauru and Tuvalu to develop knowledge of local news and trends. Certificate in psychic reading strongly preferred: palm, tarot, hypnosis, astrology, numerology, runes and/or auras.

7

8

Action/Interaction (behavioral)Humans are the best determinants of relevance. Our actions tell the search engine whether or not the

machine relevance matches our own. What the user clicks on in the SERP, what they do when they get there, where they go after, how they change their query based on going there, etc = information for the search engine about the quality of the result

As a result of this, Google has a strong (almost monopolistic) advantage due to lopsided user preference –Google has more data to figure out relevance because they have 3x the number of users to track

Search 2.0 is the “wisdom of crowds” Now we help each other find things. Search engines are now leveraging these forums as well as their own

extensive data collection to calculate relevance. Some believe that social media will replace search. How can your friends and followers beat a 100 billion page index? What if they don’t know?

• Online bookmarking: Delicious (recently shut down) morphed into personalized search engine pages (iGoogle)

• Community sites: Yelp, Angie’s list• Social Sharing: Facebook, Twitter (micro-blogging) among others.

9

If machines are methodical, as we’ve seen, and people are emotional, as we experience, where is the middle ground? Are we working harder to really find what we need or just taking what we get and calling it what we wanted in the first place?

Recently filed patents that leverage user behavior:•Microsoft Bing: Search manager (client-side application) that used analysis of user behavior to select the best search engine for the query •Microsoft Bing: compares snippets of Web search engine results with data collected from user behavior and client machine•Google: user bookmarks [online and client] used to construct “personalized search object” that is then used to filter Web search results

10

Developed by a computer science student, this algorithm was the subject of an intense bidding war between Google and Microsoft that Google one. The student, Ori Alon, went to work for Google in April 2006 and has not been heard from since. There is no contemporary information on the algorithm or it’s developer.

4/7/2011

11

comScore measure search engine market share December 2010

Google’s PageRank is inherently unfair because it favors Webmasters that know how to create links with a scoring that is not available to the end userGoogle’s scoring model is changing as PR is not calculated as frequently as before due to the size of the Web, now used as a factor for inclusion in index and how often to index the site and not the end all of placement due to incorporation of other factors, i.e. social indicators

12

13

14

15

Using the Internet: Skill Related Problems in User Online Behavior; van Deursen & van Dijk; 2009

There is no such thing as “advanced search” longer. We’re all lulled into the false sense that the search engine is smarter than us. Now the search engines present a mesmerizing array of choices distracting from the original intent of the search.

There are things that we can do to help…

16

Users look to search engines for guidance. We can provide similar guidance with user controls

17

Jared Spool did a site search study some time ago that found users successful 37% of the time when using site search and 50+% of the time when navigatingUsers don’t like navigation at the outset but will use it if contextual and in a form that they can influence

18

19

20

Stuart Brand in his book “How Buildings Learn” advised waiting to put in walkways around the building so that you can see where the pathways form on the grass and groundUsers will tell you how they want to get to content

21

Guided Tours: built on analysis of other user pathways and knowledge of corpusProduced Views: page of assembled content items focused on a single subjectTask List Drop Downs: “I Want To…” links to pages of assembled content focused on single common taskRelated Links: related as in “next steps” not what Marketing wants to be a next stepBest Bets: editorially assigned result that may not be chosen by the search engine

22

23

Not all links are created equal. Links between pages that share context are worth more (Hilltop and HITS algorithms)

DMOZ feeds the Google Directory and is rumored to be the ontology of the Web

24

25

26

Equal Representation By Search Engines: Vaughn & Zhang (2007)

27

We’re smart, search engines are a toolThe agenda is about money from advertising and local taggingStructured things are easier to find and the Web is not structuredAnalytics tell us what, not why – user research tells us whyNeed is an experience – need to know is a state of being

28

29

30

31

http://www.google.com/insights/search

32