2005 Professional Excellence Symposium blogging, googling, syndication: what’s the web coming to? ian graham Enterprise IT Strategy & BMO Connect T: E: W:  With thanks to Sherri Bondy for some example weblog sites. “The race for integration”

2005 Professional Excellence Symposium what this talk is about The culture of web applications –What are the new applications about? –How are they ‘designed’ and put together –Where are they taking the ‘public’ Web (and us!) Examples (Lots) –…that help to illustrate the above Some conclusions What the talk is not about –Banking/financial services, software development, Web Services (tech details), lobsters, rodents.

2005 Professional Excellence Symposium web culture Location Connectivity Discovery Communication & Presence Community Blogging Syndication  RSS Google Jabber, ICQ Ebay, Yahoo, Slashdot, Amazon, Blogspots, … The right side lists just a few example tools & technologies & platforms there are many others … ConceptsTools / Technologies

2005 Professional Excellence Symposium a) having a home: blogging Short for ‘Weblogging’ –A ‘log’ authored on and accessible over the Web Simple Web-site generation –Personal (or group) publishing & communications: Users often publish regularly (daily, or weekly), and link to friends / colleagues / coworkers Links establish virtual ‘communities’ with shared interests Categorization for topics, etc. –Software is often free to download (and play with) Free (or near) hosting available (see list at end) –Software often supports user-level scripting to building custom functions, behaviors

2005 Professional Excellence Symposium some example blogs [1] –Blog site about blog sites (news, information) [2] –Reports from inside Iraq before and during 2003 war (later) –Internet Explorer development team member –A ‘normal’ person (purely social site) –Inventor of JavaScript – roadmap for development team –Articles on software development (professional) –Not what you were expecting !!

2005 Professional Excellence Symposium uses of weblogs Social networking –Post musings and share them with friends (Semi-) professional information sharing –Writing for richer communication (e.g. news, current affairs, professional commentary) Work-related networking –Write status reports, decisions and analyses, etc, for sharing / cross-linking with others. Popularity? –There are circa 10 million blogsites –(of course, most are started, and then abandoned!)

2005 Professional Excellence Symposium “blogspace” The collection of blog sites all across the Internet Unconnected except for hand-created links in the pages

2005 Professional Excellence Symposium b) two blogspace issues: 1.Letting people know what you have –Publishing information ‘about’ your blog postings An advertisement, or summary, about your daily postings. RSS -- An XML language for publishing summaries RSS stands for Resource Site Summary, or RDF Site Summary, or Really Simple Syndication, or …) [some examples later ] –RSS is also used by regular web sites, news sites, etc. 2.Finding interesting/relevant stuff –Aggregation sites and desktop clients Aggregate RSS info ‘about’ published blogs / other sites Headliner clients, topic-specific Web sites –Categorization and rating systems Categorize blog items, Web page/sites separate wheat from chaff

2005 Professional Excellence Symposium b.1) rss feeds from a blogsite Time-stamped ‘summary’ about a blog item Time-stamped ‘summary’ about a blog item Metadata (for cataloging, indexing) Metadata (for cataloging, indexing) Headline text Headline text A link back to the original A link back to the original … An ‘ad’ for the original content. … An ‘ad’ for the original content. Dave Massy [MSFT] Security, Security, Security /06/20/ aspx Sun, 20 Jun :56:00 GMT This topic seems to be coming up <img src =" dmassy/aggbug/ aspx" width = "1" height = "1" /> …more text … E.g.:  RSS link [3]

2005 Professional Excellence Symposium b.2) aggregators & syndication RSS or HTML Retrieves and categorizes RSS from blog (and other) sites And provides a categorized feed to users, or a Web listing: RSS feed aggregator [4] an RSS desktop client [5] a blog-RSS feed aggregator

2005 Professional Excellence Symposium rss: not just for bloggers RSS has been adopted by major media sites as a tool for “syndicating” headlines – – Blogging has moved beyond ‘cool’ – stm (political and cultural impacts)

2005 Professional Excellence Symposium api’s for accessing aggregators…. Programming interfaces to aggregation sites –Simple, easy to use, and URL-based –E.g. Meerkat API -- [6] (documentation) –API based on arguments in the URL (above explains how this works). For example: [6.1] RSS feed of items in profile category 1065 Same, but use different RSS version for feed

2005 Professional Excellence Symposium and embeddable in web pages: (... demo.html) [6.2] Example of RSS feed inclusion using JavaScript Heading For the Example Here is some regular HTML -- ok, i didn't have time to make it pretty And here is the JavaScript example code: Includes script sent by Meerkat; script generates HTML summaries and writes them out as HTML document

2005 Professional Excellence Symposium summary Blogging is for individuals or groups. RSS (and syndication) help build connectivity, and communities Aggregators support simple APIs so it’s easy to build functions that integrate these tools with others (blogsites, portals, other..) –Richer communities via richer functionality –No conscious high-level architecture: people just try new ideas, and adopt what works (and drop what doesn’t) –A heterogeneous collection of apps, connected using simple protocols, APIs. Each application and protocol is simple (limited scope, easy to use) and open (encourages reuse, extension) Key idea: URL as the reference for everything

2005 Professional Excellence Symposium c. googling Google -- A web search engine – –Other stuff: blogger.com; google groups; news, special search services; maps; gmail ( ) … Some numbers: –Estimated space> 3,000,000 GB –Number of servers> 100,000 (Linux) Access to the index: –Via web pages, and several APIs -- some simple, some complex

2005 Professional Excellence Symposium google as a service Via APIs that let you reuse the google engine as part of your own application: –A URL-based interface (like Meerkat) URL says where to go Stuff in URL says what you want back –A full-blown Web services interface (yes, I know I promised not to mention web services!) –Both let you, essentially, use Google as the search engine / database within your own applications (subject to restrictions)

2005 Professional Excellence Symposium Google’s URL-based API (... demo-google.html) [7.1] <input type="hidden" name="cof" value="LW:600;LH:58;L: <input type="hidden" name="domains" value=“bmo.com"> <input type="hidden" name="sitesearch" value=“bmo.com"> Info – How it Works

2005 Professional Excellence Symposium google’s web services api Experimental interface, but widely used –Requires registration for access key (simple, and free) –Provides library and tools for building simple applications, using SOAP-based web services. Java and.NET examples provided API covers –Ways to query the google database –Response formats / error responses

2005 Professional Excellence Symposium Google Example [7.2] public class GoogleAPIDemo { public static void main(String[] args) { … // Create a Google Search object, set our authorization key GoogleSearch s = new GoogleSearch(); s.setKey(clientKey); // do search, then print out result try { if (directive.equalsIgnoreCase("search")) { s.setQueryString(directiveArg); GoogleSearchResult r = s.doSearch(); System.out.println("Google Search Results:"); System.out.println(r.toString()); } else if (directive.equalsIgnoreCase("cached")) { …. } else if (directive.equalsIgnoreCase("spell")) { … } else { printUsageAndExit(); } } catch (GoogleSearchFault f) { System.out.println("The call to the Google Web APIs failed:"); System.out.println(f.toString()); } } private static void printUsageAndExit() { … } }}

2005 Professional Excellence Symposium Google Summary Simple API focused on single function –Querying the google database. Loose coupling –Google doesn’t care what you’re doing. Coarse-grained –Here’s a query, and there’s your answer. No details beyond those needed for business function. These two characteristics are core to almost all ‘Web’ development models.

2005 Professional Excellence Symposium d. jabber: messaging & presence Jabber server Jabber server Jabber clients Presence User directory Proxys to Yahoo, ICQ Other services Jabber servers

2005 Professional Excellence Symposium Jabber Example Jabber server Jabber server Jabber client Connect register presence Lookup user contact database Send text message contact database Jabber client Requests and responses all sent in XML Generic XML protocol for exchanging messages, plus some services. Can be extended to non-text messaging applications Now used for virtual presence: lets you know who else is at a blog or other web site Requests and responses all sent in XML Generic XML protocol for exchanging messages, plus some services. Can be extended to non-text messaging applications Now used for virtual presence: lets you know who else is at a blog or other web site Jabber client [8.1]

2005 Professional Excellence Symposium Virtual Presence (Jabbber extension: [8.2]) 1. Web Server registers with virtual presence (VP) server, using a URL 2. VP client gets name of VP server (URL) from Web server. 3. VP client registers with VP server. 4. VP server tells VP client who else is there. 5. VP client provides chat service to other users of the site. VP server WS 1 VP client

2005 Professional Excellence Symposium Key features Simple, single-purpose URL-based referencing scheme –Underlying protocol uses XML Loose coupling between components –servers, clients – failure is expected Less-simple protocol than RSS … –May explain why jabber (and family) isn’t as popular as other technologies.

2005 Professional Excellence Symposium And many other examples Bittorrent - file sharing / transfers – 30% of all Internet traffic is generated by BitTorrent Wikis -- collaborative web authoring – -- a collaboratively edited ‘encyclopedia’ Over 500,000 entries (English) –

2005 Professional Excellence Symposium An example application Web page (from web server) that includes data from an aggregator, –uses google to provide searching –publishes new content info using RSS –Uses jabber to provide presence functionality to visitors –Provides bittorrent links to downloadable data All integrated using URLs, and simple web protocols.

2005 Professional Excellence Symposium Conclusions Social  focus on community, networking, connection, sharing Technical  Software design focused on –Simplicity of interfaces, and applications –URL as the key reference for an application, function, method –Loose coupling between applications Construction of Web applications by integrating these different ‘application services’ together. Nice essay on this at: – -- “Small pieces loosely Joined”

2005 Professional Excellence Symposium Conclusion II “ A large part of how we think about music is influenced by the methods by with which it has conventionally been distributed. We think of pop songs as being three or four minutes long because 40 years ago that was all that could fit on one side of a vinyl single.” Moby We often think of Internet-based computing is the same way -- in terms of what we know -- not what it is becoming. It is different, and challenges our thinking! Couldn’t be a more fun time to be doing software!

2005 Professional Excellence Symposium blogging, googling, syndication: what’s the web coming to? ian graham Enterprise IT Strategy & BMO Connect T: E: W: “The race for integration” --- The End ---

2005 Professional Excellence Symposium Some Tools and Sites Blogging software – – – – Free blogging sites –  (Ian’s site) – RSS / Atom Aggregation sites – – – – (Open Wire Service) API Documentation: RSS / Atom Desktop clients – –

2005 Professional Excellence Symposium Some Tools and Sites Google tools and APIs –Google Hacks, Calishain and Dornfest, O’Reilly and Associates, – – (Free websearch) – (custom search) – – (Google Web service APIs) Jabber – – (virtual presence extension) Bittorrent – Wikis – (all about Wikis) Java Development Kit –