YQL AND YAHOO! APIS

Saurabh Sahni YDN Developer, Hacker, Evangelist Twitter: @saurabhsahni

Hacking together systems in 24 hours is lot of fun

Hacks = Data manipulation

+ Data visualization

The web has lot of data around

ProgrammableWeb.Com – 3517 APIs

Yahoo! has opened up its data

http://developer.yahoo.com/everything.html

THE TROUBLE WITH DATA •  You need to find data API •  Get Access – Signup for key •  Find data endpoint •  Read docs to learn what parameters you

have •  Get data in obscure format •  Use data after converting and filtering •  More APIs you use, more is your

annoyance

To make data access easy on the web, Yahoo! created YQL

YQL turns web services and data on the web into databases.

select {what} from {where} where {conditions}

You can select, filter, sort and limit

You can even insert, update and delete from it.

FINDING VIDEOS ABOUT BANGALORE

SELECT * FROM youtube.search where query='bangalore'

SELECTING PHOTOS OF HACKDAY

SELECT * FROM flickr.photos.search where text="hackday"

INSERTING DATA

INSERT INTO bitly.shorten (login, apiKey, longUrl)

VALUES ('ME', 'API_KEY', 'http://yahoo.com')

UPDATING DATA

UPDATE social.profile.status

SET status="Using YQL UPDATE”

WHERE guid="NJFIDHVPVVISDX7UKED2WHU"

RETRIEVING MY CONTACTS

SELECT * FROM social.contacts WHERE guid=me

ACCESSING PRIVATE DATA

http://query.yahooapis.com/v1/yql

Uses OAuth 1.0 for authorization

OAuth is complicated – use one of our SDKs at https://github.com/yahoo

The easiest way to start with YQL is to use the console

http://developer.yahoo.com/yql/console

YQL: http://developer.yahoo.com/yql/console

URL STRUCTURE

http://query.yahooapis.com/v1/yql?[params] http://query.yahooapis.com/v1/public/yql?[params]

Params •  q = [ YQL QUERY ] •  format = [ XML / JSON ] •  diagnostics = [ true / false ] •  debug = [ true / false ] •  callback = [ function name ]

LET’S SEE IT

QUERY EXAMPLES

Find hackday tweets: SELECT * FROM twitter.search where q='hackday’

Search Yahoo! Answers for resolved questions about cars: select * from answers.search where query="cars" and type="resolved”

Translate tweets into Hindi: select * from google.translate where q in (select text from twitter.search where q="hackday") and target='hi';

Extract important terms from top stories on Yahoo! news: select * from search.termextract where context in (select description from rss where url='http://rss.news.yahoo.com/rss/topstories')

QUERY EXAMPLES

Find distance between Bangalore and Mumbai: select * from geo.distance where place1="bangalore" and place2="mumbai”

Get stock quotes for Yahoo!, Apple, Google & Microsoft in one query: select * from yahoo.finance.quotes where symbol in ("YHOO","AAPL","GOOG","MSFT")

Shorten a URL: insert into yahoo.y.ahoo.it (url, keysize) values ('http://www.javarants.com', 5)

Search apartments in criagslist: select * from craigslist.search where location="bangalore" and type="apa" and query="indiranagar”

QUERY EXAMPLES

Scrape news from Yahoo! Finance: select * from html where url="http://finance.yahoo.com/q?s=yhoo" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a’

Select, filter data from google spreadsheets: select * from csv where url="https://spreadsheets.google.com/pub?key=0ArYndzim-lbrdF8wc3A5QWl1ZGRpdkxRZk80SU9zUXc&output=csv" and col5 like 'Bangalore%’ ;

Let’s find openhackindia photos on flickr

MAKING REQUESTS: FLICKR URLS

<photo farm="3" id="5708163920"

isfamily="0" isfriend="0" ispublic="1" owner="31832337@N04" secret="0075137487" server="2496" title="San Francisco"/>

MAKING REQUESTS: FLICKR URLS

http://farm{$farm}.static.flickr.com/{$server}/{$id}_{$secret}.jpg

Photo URL  

http://www.flickr.com/photos/{$owner}/{$id} Photo Page URL  

http://www.flickr.com/photos/{$owner} Photo Owner Profile URL  

https://github.com/saurabhsahni/Hacks/

Finding Recent Photos from flickr

Some YQL Hacks

ChromYQLip is a chrome extension for page scraping via YQL

Open  Hack  Bangalore  2010  Winner  

http://bit.ly/chromeYQL

VIDEO CLIP

http://www.webmeme.in

WEBMEME.IN

Fetch multiple feeds in different formats like atom, RSS and transform them into consistent RSS format Select * from rss where url in (‘http://feeds.feedburner.com/pluggd’, ‘http://quatrainman.blogspot.com/atom.xml’, ‘…’)

Filter news containing “india” from multiple feeds: select * from rss where url in ('http://feeds.feedburner.com/TechCrunch', 'http://www.readwriteweb.com/rss.xml','http://gigaom.com/feed/') and description like '%india%’

YQL is open – you can get your data tables in our system

All you need to do is write an XML schema and put it on Github.

http://github.com/yql/yql-tables

Here is the craigslist search table https://github.com/yql/yql-tables/tree/master/craigslist/craigslist.search.xml

USE INSTANTLY BY UPLOADING ON YOUR SITE

USE 'http://www.mysite.com/my_table.xml' AS mytable; SELECT * FROM mytable WHERE user='saurabh'

You can even write server side javascript to build a webservice or augment one. h#p://developer.yahoo.com/yql/guide/yql-­‐execute-­‐chapter.html  

There are lot of things you can do with YQL.

Play yourself http://developer.yahoo.com/yql/

RESOURCES

All Yahoo! APIs and Services http://developer.yahoo.com/everything.html

YQL Documentation http://developer.yahoo.com/yql

YQL Console http://developer.yahoo.com/yql/console

YQL Github Account (Contribute Tables) http://github.com/yql/yql-tables

ANY QUESTIONS?

Saurabh Sahni

Twitter: @saurabhsahni Github: http://github.com/saurabhsahni Web: http://www.saurabhsahni.com

http://www.slideshare.net/saurabhsahni