Preview:
DESCRIPTION
Citation preview
- 1. YQLPaul Donnelly (@pjdonnelly)YQL Team
- 2. When creating an application theheart of the project is the
data.
- 3. So how do we get data?
- 4. So how do we get data?- APIs(flickr, twitter)- Spreadsheets
(yahoo finance csv, google docs)- RSS (blogs, news)- HTML
pages
- 5. Many applications require more thenone source of data from
differentproviders.
- 6. This can get complicated very quickly.
- 7. This can get complicated very quickly.For each web service I
need to: Read API docs Figure out REST endpoints Figure out rate
limits Figure out syntax
- 8. YQL TO THE RESCUE YQL is an expressive SQL-like language
that lets you query, filter, and join data across Web services. SQL
= familiar Not limited to just Yahoo web services, but any web
service. Single access point Learn the YQL syntax to filter data
instead of learning an individual API.
- 9. YQL TO THE RESCUE Using YQL, accessing the web and its APIs
becomes as simple as SQL: select {what} from {service} where
{condition}
- 10. RSS AGGREGATIONselect * from feed where url
in("http://feeds.feedburner.com/TechCrunch/","http://allthingsd.com/feed/")
- 11. RSS AGGREGATIONselect title,description from feed whereurl
in("http://feeds.feedburner.com/TechCrunch/","http://allthingsd.com/feed/")
- 12. RSS AGGREGATIONselect title,description from feed whereurl
in("http://feeds.feedburner.com/TechCrunch/","http://allthingsd.com/feed/")
|sort(field="pubDate", descending="true)
- 13. SORT AND OTHER METHODSAnything right of the | is performed
after all selectoperations are done.Valid methods: sort tail (gets
the last count items) truncate (gets the first count items) reverse
unique (no duplicates) sanitize (emit safe html, prevents XSS)
- 14. HTML SCRAPING (SCRAY-P-I)Pinterest doesnt (currently) have
anAPI for its top 50 pins.
- 15. HTML SCRAPING (SCRAY-P-I)select * from html where
url="http://pinterest.com/popular/" and xpath
="//a[@class=PinImageImgLink]"
- 16. HTML SCRAPING (SCRAY-P-I)
- 17. FLICKRFirst get a Flickr API
Key:flickr.com/services/apps/create/apply
- 18. FLICKRQuery for pizza:select * from
flickr.photos.searchwhere text="pizza"
andapi_key="5b7b21a3d44ed659e13410edee783a3b" limit 10
- 19. FLICKRQuery for pizza:
- 20. FLICKRQuery for pizza:select source from
flickr.photos.sizeswhere photo_id in (select id
fromflickr.photos.search wheretext=pizza" and api_key=
andsort="relevance") and api_key=and label="Medium"
- 21. FLICKRQuery for pizza:
- 22. HOW DO I ACCESS YQLPublic
Endpoint:http://query.yahooapis.com/v1/public/yql?q={query}&format={format}
- 23. HOW DO I ACCESS YQLFormatsCurrent outputs are XML or JSON
orJSON-P
- 24. HOW DO I ACCESS YQLYQL
Console:developer.yahoo.com/yql/console/
- 25. YQL CONSOLE
- 26. CONSOLE PARAMETERS - DIAGNOSTICS
- 27. CONSOLE PARAMETERS - DEBUG
- 28. CONSOLE PARAMETERS DEBUG CONTD.
- 29. CONSOLE TIPS REST QUERY
- 30. CONSOLE TIPS COMMUNITY TABLES
- 31. CONSOLE TIPS PERMALINK & ALIAS
- 32. TABLE EDITOR Rapid YQL table prototyping Create YQL tables
Create YQL Environment files Create YQL Javascript execute files
Sample templates for all three Nifty drag & drop support All
the above stored in the Yahoo! Cloud
http://developer.yahoo.com/yql/editor
- 33. TABLE EDITOR SAMPLE TABLE
- 34. YQL IN YAHOO PIPES http://pipes.yahoo.com Easy way to
create RSS and createcomplex data flows using a visualeditor. NEW!!
YQL method y.pipe(pipeid,)
- 35. YQL IN YAHOO PIPES
- 36. RECAP
- 37. YQL FORMATS Accepts JSON, XML, CSV Outputs XML, JSON, JSONP
Provides data type transformers and mutators Tidies up data sources
for you (html table) Lets you spend more time building your
app
- 38. FAST, GLOBALLY AVAILABLE Deployed across the globe Both
front and back facing caches Developers can control cache behavior
_maxage= parameter _stalewhilerevalidate= parameter Rate limited
for external developers 2000 calls per hour per IP on public
endpoint 20000 on OAuth protected endpoint
- 39. APP DEVELOPMENT USING YQL Pick open tables or base YQL
tables to buildyour app. Build new ones for your API if needed Test
the tables in the YQL console Use the YUI-YQL module to get data Or
just call the REST query endpoint
- 40. TIPSSearch github.com/yql for table examples:Use:
http://developer.yahoo.com/yql/guide/Use:
http://christianheilmann.com/tag/yql/ forawesome examples and
inspiration
- 41. THANKSSite: http://developer.yahoo.com/yql/Community:
http://www.yqlblog.net/ http://www.datatables.org/
https://github.com/yql/yql-tablesEmail:
yql-questions@yahoo-inc.comTwitter: @pjdonnelly