Data, schemas and Applications UFIEKG-20-2 Chris Wallace Praminda Caleb-Solly

Assessment 50% exam –40% multiple choice –2 *30% design questions 50% coursework –Term 1 – well-defined group project + individual component using Yahoo Pipes and XML with SQL –Term 2 – open individual project XML and XQuery

Resources UWE Online –Schedule –Web sites –Week by Week material DSA2007 blog – –Informal news and commentary –please add your own comments del.icio.us –e.g. –Social bookmarking site

Books Use free resources on the web –Wikipedia –Wiki Books A number of useful books on XML for a tenner. –XML - Learning by Example - by Robert Mellar, reasonably priced at 10.49XML - Learning by Example –There's another book for a tenner XML in Easy Stages by Mike McGrath.XML in Easy Stages –At the same price there is a SAMS bookSAMS book Reference book –XML in a Nutshell - Eliottte Rusty HaroldXML in a Nutshell Eliottte Rusty Harold

Themes Data (and Information) Schemas Tools and Languages Applications Learning to learn

Data The world is awash with data –Cheap memory means it’s possible to keep every transaction, every document –The Web provides an infrastructure for individuals and organisations to make huge volumes of data available Flickr Wikipedia Facebook BBC

Information Data is useful for informing decisions, especially if it can be combined: –Price comparision –Nearby resources –Informed public opinion But –Data is hard to combine –Data is of variable and often unknown quality and source (who, when, where, why) The web as distributed datasets The semantic web

Schemas How data is structured –CSV (Comma separated variable) –EXIF –Relational Tables –Tagged (HTML, XML) How we can check that data is in the right format (Validation) How we can describe the rules of valid data.

Tools A new language –RSS A way of disseminating News A new development tool –Yahoo Pipes A Visual programming language for ‘Mashups’ In the future –XML, SQL, XQuery, a bit of PHP...

Applications Combining data from multiple sources to increase the value of the data Developing our own data sources Seaching and visualizing complex data

Learning to learn Developing skills in learning new tools and languages New langauges and tools appear constantly –Very exciting –Very time-consuming –Very confusing (should I bother?) The more you learn, the quicker it gets

Learning Objectives for the next few weeks Data –News and weather Schema –RSS (and XML) Tools –Yahoo Pipes Application –A personal news feed

Spreading the news Problem: –I want to disseminate news about my project/company, and allow interested people to read it. –e.g. the university wants to spread the news about successful staff Solution –publish a page of news on the website in HTML Problems –how do visitors know when its changed? –news from different universities cannot be easily combined – (why?)

Extend HTML Solution –UWE makes up its own set of additional tags to add to the HTML page … Problem – A reader has to be able to understand this format – and many others!

RSS Solution –Standardise on one (or 4!) general formats of news feed, so that any reader can understand it –RSSRSS Really Simple Syndication Rich Site Summary –Also ATOM Characteristics –Basic content are items in reverse chronological order –Some basic tags such as links back to the full story.

RSS at UWE The UWE site Shows that there is an RSS feed Which is this page oadxml.asp oadxml.asp This page appears to be a normal HTML page but underneath View Source

UWE News Latest UWE press releases University of the West of England Sun, 1 Oct :06:10 GMT UWE researcher wins funding for diabetes project What prevents insulin from being secreted naturally from the pancreatic beta cells of people who have diabetes? This question is key to research undertaken by Dr Aniko Varadi at the University of the... ….

RSS Feed Readers Problem –How to keep up with changes to multiple RSS feeds Solution –Application needed which Is stateful – remembers what items you have read Integrates multiple feeds into one ‘magazine’ Polls RSS providers on a regular basis –Feed integrators Bloglines, Google Reader, reduce the load on the provider and provide some filtering –There is an RSS reader integrated into MyUWE

RSS Mashup Problem –I want to create my own feed which brings together and reorganises data from several RSS feeds, web sites, search engines –A MASHUP Solution –Write your own script (later in XQuery) –Use Yahoo Pipes to do the job

Yahoo Pipes A visual programming environment Allows standard components (modules) to be combined with data flows to produce a new combined component (a Pipe) ‘Fill in the fields’ to configuring each module Connect the modules with data flows Look at the flow in any pipe

How do you learn a new language or tool? What approach works best for you? What sources of information do you use? How do you record your understanding as it develops?

Pyramiding 5 minutes – think about these questions on your own 10 minutes – in pairs, look for differences and explore these 10 minutes - groups

Tutorials Week 1 –Background Questionnaire –Revision Preparation for next week –Watch the Google Tech talk on Yahoo Pipes –Find and subscribe to one RSS feed Week 2 –Workshop on Yahoo Pipes

Next week Where do RSS feeds come from? How does a RSS reader work? How to roll your own RSS feed Using Yahoo Pipes