Mashups: Beyond Maps

1

Mashups: Beyond Maps

Chad Dickerson

chadd@yahoo-inc.comDuke Web Community Forum

May 11, 2006

2

Coming up

• A bit about Yahoo! and what I do• Mashups: historical context and definition• Yahoo! and web services• Unavoidable mention of maps• Cool non-maps apps that you could have

built yourself, featuring services you mightnot know about

3

Background on Yahoo!

• Based in Sunnyvale, CA• ~10,000 employees• $4.6 - 4.85B annual revenue (2006 est.)• Recent acquisitions: Flickr, del.icio.us,

Upcoming.org, Webjay• 402 million use Yahoo! services each

month

Source:http://biz.yahoo.com/rb/060418/media_yahoo_earns.html?.v=6

4

What I do at Yahoo!

• Joined in August 2005

• Hack Yahoo!– Hack Days worldwide, internal programs

• Engineering SWAT team– AJAX Maps API, mobile prototypes, product

development

• Internal and external evangelism about benefitsof platforms and open APIs

• Still write a little (bad) code when needed

5

Mashups

Historical context and definition

6

A quick word about “screenscraping”

• The web was built as a “view source”environment

• Web scraping has existed since thebeginning of the web

• 1994: Weather. Today: Amtrak.• Yahoo! has always seen this kind of activity• For companies not willing to open up data,

there is the threat of the “ScrapePI”– ex. http://www.ontok.com/wiki/index.php/Wikipedia

7

One way to think about APIs(for the non-technical)

• A TV remote control is adocumented interface that letsyou do certain things

• You know “volume up” willincrease the volume, “channelup” will change the channel – themanual (i.e. documentation) saysso

• Interaction results in expectedbehavior; there is a “contract”

• Remote buttons are essentiallyan API into your TV

8

What is a “mashup?”

a website or web application that seamlesslycombines content from more than one sourceinto an integrated experience.

Content used in mashups is typically sourcedfrom a third party via a public interface or API.Other methods of sourcing content for mashupsinclude Web feeds (e.g. RSS or Atom) andJavaScript.

(Source: Wikipedia)

9

What’s driving the mashupexplosion?

• Emergence of public APIs for previouslyhidden web services

• XML, especially RSS/Atom feeds for data• an Internet social etiquette that strongly

promotes “opening up” applications andinformation for re-use and re-mixing

• increasing ease-of-use

Sources: adapted from many sources (e.g. Tim O’Reilly’s “What is Web 2.0” essay), but the words are from:

http://outsideinnovation.blogs.com/pseybold/2006/03/why_mash_ups_ma.html

10

The social etiquette codified:The Hacker Ethic

• Access to computers—and anything which might teach yousomething about the way the world works—should be unlimited andtotal. Always yield to the Hands-on Imperative!

• All information should be free.

• You can create art and beauty on a computer.• Computers can change your life for the better.

• Mistrust authority—promote decentralization.• Hackers should be judged by their hacking, not bogus criteria such

as degrees, age, race, or position.

Source: Stephen Levy, Hackers: Heroes of the Computer Revolution (http://www.gutenberg.org/dirs/etext96/hckrs10.txt)

http://en.wikipedia.org/wiki/Hacker_ethic

http://en.wikipedia.org/wiki/Hackers:_Heroes_of_the_Computer_Revolution#Hacker_ethic

Every good work of software starts by scratching a developer's personal itch.-- Eric Raymond, “The Cathedral and the Bazaar”

11

The first mashup:Eli Whitney’s “Uniformity System”

• Whitney amazed government officials in1798 when he disassembled twelvemuskets, mixed up the parts and thenreassembled the weapons in working order

• Before this, muskets were each hand-tooled; parts could not be interchanged

• Whitney's new manufacturing process wasknown as the "uniformity system"

• made it possible to create mechanicalcomponents that could be recombined toproduce new machines

• set the stage for the American IndustrialRevolution

• standard interfaces among parts (orhardware) have been extended to software

Source: http://www.sims.berkeley.edu/~hal/Articles/TheStandard/recombinant.html

12

Theory of recombinant growth

• Hal Varian wrote about “recombinant growth” in March2000 (height of NASDAQ)

• a theory that arose from the works of Stuart Kaufman, abiologist at the Santa Fe Institute, and Martin Weitzman, aHarvard economist

• innovations are broken down into separate parts• parts recombined in novel and surprising ways to create

new inventions• web is ripe ground since it is inherently open (think

TCP/IP, HTTP, etc.)• mashups illustrate the theory well

Source: http://www.sims.berkeley.edu/~hal/Articles/TheStandard/recombinant.html

13

Mashups: burst of recombinantgrowth, aka “Web 2.0”

Rise of useful web services

2001-present

The “hacker ethic”1950s-present

Mashups

14

Yahoo! and Web Services

15

Yahoo! Developer Network

• Yahoo! provides many useful and free Web servicesthat you can use in your applications and Web sitestoday (including very rich maps APIs)

• developer.yahoo.net launched in March 2005. Allservice were search-related (video, web, local, newsand image)

• In April 2005, Yahoo! started building a dedicated teamto co-ordinate and support efforts for publicly availableWeb services within Yahoo! Currently there are 6people on the team and still growing

• What we offer today is just the tip of the iceberg

16

Why web services?

• Extend the reach of Yahoo! properties– Beyond the browser: mobile, desktop, etc.– Enable combinations of properties (Flickr images on Maps!

Movie listings with restaurant reviews! Travel, Local Searchand Traffic…)

• Enable users to use Yahoo! data in their own way– We devote resources to making the “greatest good for the

greatest number” happen– But: we also recognize that today’s marginal use case can

become tomorrow’s mainstream behavior– Enabling users of Yahoo! to create applications themselves is

an important way to make this happen

Every good work of software starts by scratching a developer's personal itch.-- Eric Raymond, “The Cathedral and the Bazaar”

17

Developers! Developers!(sweating the big stuff, literally)

Steve Ballmer of Microsoft:http://www.ntk.net/media/developers.mpg

18

More than maps:Web services we provide today

• del.icio.us– Social bookmarking

• Flickr– Best way to store, sort, search and

share your photos online.

• Maps– 5 different technologies supported– Embed maps– Geocoding, Traffic Overlay, Satellite,

International

• Music– Customize YME with plug-ins or skins– Webjay - publish web playlists.

• RSS Feeds

• Search Marketing– Advertiser Web Services

• Shopping– Comparison shopping

• Search APIs– Web Search– News, Local– Audio, Image, Video– Content Analysis– My Web (bookmarks + tags + FOAF)

• Travel– Trip planner– FareChase

• Upcoming.org– Community-driven calendar

• Y! Widgets– Desktop helper applications

• Many more to come…

19

Quick technical overview

• We offer several styles of APIs– Most are REST / yREST / RESTful or RSS return XML or JSON– SOAP is on the Roadmap for Messenger and Mail. Flickr offers SOAP today– JavaScript / ActionScript APIs for Maps

• No onerous sign-up process or fees (if you have a Y! ID you’re prettymuch good to go)

• Most calls take an application ID (which is used solely to help youdetermine how popular your app is)

• Rate-limiting is done per IP address (not per app ID)– Most of the applications are Web based, and it limits per server. For client

application or ones that use JSON rate limiting is therefore done per user.

• Some APIs (Simple Maps API, RSS feeds, others) have no rate limits– Limits are noted on the documentation pages (developer.yahoo.net)

20

Maps

Obligatory mention

21

A look back at Google Maps timeline:from hacking to official API

• February 8, 2005– Google Maps product release

• February 9, 2005– detailed deconstruction of Maps product by a blogger– http://web.archive.org/web/20050211023302/http://jgwebber.blogspot.com/2005/02/mapping-google.html

– Jon Udell: “Google Maps is a web of linked XML documents”– http://weblog.infoworld.com/udell/2005/02/09.html#a1172

• February 9 – June 28, 2005: Lots of mashups– HousingMaps.com – April 2005, mashup craze begins

• Formal Google Maps API released: June 29, 2005– http://www.boingboing.net/2005/06/29/google_maps_api_rele.html

– over 5 months since useful applications were being built!

22

Yahoo! Maps

• There are lots of cool apps builton Yahoo! Maps platform.

• Maps are great.• I love maps.• Some of our best employees work

on maps.

See:http://developer.yahoo.com/maps/applications.html

23

Cool non-maps apps

24

Rollyo

• “Yahoo provides the engine and Rollyo puts youbehind the steering wheel.”

• Use Search APIs to provide vertical search

• Demo: http://www.rollyo.com/

25

TagCloud

• Uses Term Extraction API

• This API provides a list of significant words or phrasesextracted from a larger block of content

• Unfortunately, the developer took it down because itbecame too popular!

See:

http://developer.yahoo.com/search/content/V1/termExtraction.html

http://www.tagcloud.com/

26

Using Wikipedia / Yahoo Search API tomap political relationships

How do you breathe life into old rolodex-style flat databases of content usingonly free Internet APIs and content sources?

1. Gather list of names, e.g. British politicians like Margaret Thatcher, TonyBlair

2. Use Y! Search API (restrict to wikipedia.org) to find definitive Wikipediapage for Thatcher

3. Run text of that page through Term Extraction API to extract key phrasesand names

4. Do this for each of the names in your list, then you can draw a map

5. Relate this information back to your own internal databases when it makessense

See Matt Biddulph: http://www.hackdiary.com/archives/000070.html

27

The resulting map

for full map, see http://www.hackdiary.com/misc/people.png

28

QOOP:making $$ with Flickr API

• printing service for Flickr• Powered by the Flickr API• Working on commercial self-

serve option for QOOP-likecompanies

29

Checkmates:a mobile friend finder

• http://tinyurl.com/pya4x

• “failed” Hack Day project• Combines:

– Y! Maps

– Flickr social network

• Demoed at O’Reilly eTech last month

• Built by 2 engineers and 2 designers ina matter of weeks with little mobiledevelopment experience– Robust APIs promote rapid innovation

30

How Checkmates Works:A little more background

• Scroll to your location on amap

• Indicate where you are

• Your Flickr friends can thensee you on the map

31

How Checkmates works:Sub-maps at eTech

• Sub-maps of conferencesessions were available onphone

• Users were able indicatelocation within these sub-maps

• The location of friends (asrepresented in Flickr network)were displayed

32

Checkmates: Technology

• Platforms used– Flickr– Yahoo! Maps– Java phone technology

• Specific APIs used– Flickr user authentication– Other Flickr APIs (setting tags, uploading

photos, social network)– Yahoo! AJAX Maps API– Yahoo! Geocoding API– J2ME

33

YUI Library

• set of utilities and controls, written in JavaScript, for building richlyinteractive web applications using techniques such as DOMscripting, DHTML and AJAX.

• BSD license!

• Utilities: Animation, Drag and drop• Controls: Auto-complete, slider, menu, calendar

• Motivations– Web users in the mainstream are demanding more from the Web today.– A rising tide lifts all boats (inspired by Scriptaculous, Dojo, etc.)

• Use it today!

• See: http://yuiblog.com/blog/2006/02/13/the-yahoo-user-interface-library/

34

Application Gallery -gallery.yahoo.com

Unifies the many separate collections of applications

• Flickr, Search, Widgets, Maps, etc., etc.

• For Widgets, Web sites, plug-ins, etc.

Web site that enables third-party software developers to show offapplications that utilize Yahoo! technology.

• Tag-based navigation

• Integration with Y! Ratings & Reviews

• Full-text search

• Support for localization

• RSS everywhere

35

Application Gallery

36

Final word on Yahoo! APIs

• Build something• Surprise and delight us• Let us know about it by submitting it to the

Application Gallery

37

Questions?

Chad Dickersonchadd@yahoo-inc.com

http://www.chaddickerson.com/blog/