Data, schemas and Applications UFIEKG-20-2 Chris Wallace Praminda Caleb-Solly

Data, schemas and Applications

UFIEKG-20-2

Chris Wallace

Praminda Caleb-Solly

Assessment

• 50% exam– 40% multiple choice– 2 *30% design questions

• 50% coursework – Term 1 – well-defined group project +

individual component• using Yahoo Pipes and XML with SQL

– Term 2 – open individual project • XML and XQuery

Resources

• UWE Online– Schedule– Web sites– Week by Week material

• DSA2007 blog– http://dsa2007.blogspot.com/– Informal news and commentary– please add your own comments

• del.icio.us– e.g. http://del.icio.us/morelysq/pipes– Social bookmarking site

Books

• Use free resources on the web– Wikipedia– Wiki Books

• A number of useful books on XML for a tenner.– XML - Learning by Example - by Robert Mellar,

reasonably priced at 10.49– There's another book for a tenner XML in Easy

Stages by Mike McGrath.– At the same price there is a SAMS book

• Reference book– XML in a Nutshell - Eliottte Rusty Harold

Themes

• Data (and Information)

• Schemas

• Tools and Languages

• Applications

• Learning to learn

Data

• The world is awash with data – Cheap memory means it’s possible to keep

every transaction, every document– The Web provides an infrastructure for

individuals and organisations to make huge volumes of data available

• Flickr • Wikipedia• Facebook• BBC

Information

• Data is useful for informing decisions, especially if it can be combined:– Price comparision– Nearby resources– Informed public opinion

• But– Data is hard to combine– Data is of variable and often unknown quality and

source (who, when, where, why) • The web as distributed datasets• The semantic web

Schemas

• How data is structured– CSV (Comma separated variable)– EXIF– Relational Tables– Tagged (HTML, XML)

• How we can check that data is in the right format (Validation)

• How we can describe the rules of valid data.

Tools

• A new language– RSS

• A way of disseminating News

• A new development tool– Yahoo Pipes

• A Visual programming language for ‘Mashups’

• In the future– XML, SQL, XQuery, a bit of PHP ...

Applications

• Combining data from multiple sources to increase the value of the data

• Developing our own data sources

• Seaching and visualizing complex data

Learning to learn

• Developing skills in learning new tools and languages

• New langauges and tools appear constantly– Very exciting– Very time-consuming – Very confusing (should I bother?)

• The more you learn, the quicker it gets

Learning Objectives for the next few weeks

• Data– News and weather

• Schema– RSS (and XML)

• Tools – Yahoo Pipes

• Application– A personal news feed

Spreading the news • Problem:

– I want to disseminate news about my project/company, and allow interested people to read it.

– e.g. the university wants to spread the news about successful staff

• Solution– publish a page of news on the website in HTML

• Problems– how do visitors know when its changed?– news from different universities cannot be easily

combined – (why?)

Extend HTML

• Solution– UWE makes up its own set of additional tags

to add to the HTML page• <newsitem date=“2007-10-2”>…</newsitem>

• Problem– A reader has to be able to understand this

format – and many others!

RSS

• Solution– Standardise on one (or 4!) general formats of news

feed, so that any reader can understand it– RSS

• Really Simple Syndication• Rich Site Summary

– Also ATOM

• Characteristics– Basic content are items in reverse chronological order – Some basic tags such as links back to the full story.

RSS at UWE

• The UWE site http://www.uwe.ac.uk

• Shows that there is an RSS feed

• Which is this page http://info.uwe.ac.uk/news/uwenews/downloadxml.asp

• This page appears to be a normal HTML page but underneath

• View Source

<?xml version="1.0" encoding="iso-8859-1"?> <rss version="2.0"> <channel> <title>UWE News</title> <link>http://www.uwe.ac.uk</link> <description>Latest UWE press releases</description> <image> <url>http://info.uwe.ac.uk/common/assets/2004Design/logo.gif</url> <title>University of the West of England</title> <link>http://www.uwe.ac.uk</link> </image> <pubDate>Sun, 1 Oct 2007 12:06:10 GMT</pubDate> <item> <title>UWE researcher wins funding for diabetes project</title> <link>http://info.uwe.ac.uk/news/uwenews/article.asp?item=1123</link> <description>What prevents insulin from being secreted naturally from the pancreatic beta cells of people who have diabetes? This question is key to research undertaken by Dr Aniko Varadi at the University of the...</description> </item> <item>…. </channel></rss>

RSS Feed Readers

• Problem– How to keep up with changes to multiple RSS feeds

• Solution– Application needed which

• Is stateful – remembers what items you have read• Integrates multiple feeds into one ‘magazine’• Polls RSS providers on a regular basis

– Feed integrators Bloglines, Google Reader, reduce the load on the provider and provide some filtering

– There is an RSS reader integrated into MyUWE

RSS Mashup

• Problem– I want to create my own feed which brings

together and reorganises data from several RSS feeds, web sites, search engines

– A MASHUP

• Solution– Write your own script (later in XQuery)– Use Yahoo Pipes to do the job

Yahoo Pipes

• A visual programming environment

• Allows standard components (modules) to be combined with data flows to produce a new combined component (a Pipe)

• ‘Fill in the fields’ to configuring each module

• Connect the modules with data flows

• Look at the flow in any pipe

How do you learn a new language or tool?

• What approach works best for you?

• What sources of information do you use?

• How do you record your understanding as it develops?

Pyramiding

• 5 minutes – think about these questions on your own

• 10 minutes – in pairs, look for differences and explore these

• 10 minutes - groups

Tutorials

• Week 1– Background Questionnaire– Revision

• Preparation for next week– Watch the Google Tech talk on Yahoo Pipes– Find and subscribe to one RSS feed

• Week 2– Workshop on Yahoo Pipes

Next week

• Where do RSS feeds come from?

• How does a RSS reader work?

• How to roll your own RSS feed

• Using Yahoo Pipes