End User Mashup Programming Environments Oleg Beletski HUT, Telecommunications Software and Multimedia Laboratory

Content Introduction Approach – mashup methods and ways to compare Case study – overview of mashup environments Results + Discussions Conclusions

Introduction Mashup – web application that combines data from online sources of data and perform new tailored task Mashup creation –Hard way – programmer skills required –Easy way – use end user programming tools and visual environments

Approach Find environments for a comparison Define criteria for comparison Analyze each site Sum-up and analyze results (drop if environment does not fit well to our scope) Make conclusions

Approach – criteria for comparison Lifecycle Richness of functionality Usability Target type of users Publishing formats Technologies Data sources

Case study – list of environments Yahoo! Pipes Microsoft’s Popfly MarmiteDapper Google mashup editor MashmakerQEDWiki

Case study – Yahoo! Pipes Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web. Visual drag and drop environment Does not require knowledge of programming languages Requires good understanding of a data formats. Sample application: –Combine many feeds into one, then sort, filter and translate it. –Geo code your favorite feeds and browse the items on an interactive map (use location extractor) –Power widgets/badges on your web site. –Grab the output of any Pipes as RSS, JSON, KML, and other formats.

Case study – Yahoo! Pipes

Case study – Microsoft Popfly Microsoft® Popfly™ is a web site and tool to help people create and share web sites, mashups, and other kinds of experiences. Popfly is the fun and easy way to build and share mashups, gadgets, and Web pages. It’s made up of online visual tools for building Web pages and mashups and a social network where you can host, share, rate, comment and even remix creations from other Popfly users. It is fun, it is simple Uses Silverlight runtime Has a lot of predefined modules for existing services and social network Cool visualizations

Case study – Microsoft Popfly

Case study – Marmite Marmite, a tool that will let everyday end-users create mashups by making it easy to extract content from web pages, process it in a data-flow manner, integrate it with other data sources, and direct it to a variety of useful sinks, such as databases, map services, and compilable source code that can be further customized. This proposal focuses on three high-risk issues: –making it easy to select what content to crawl –developing a hybrid dataflow / spreadsheet UI that shows what content has been extracted and how that content is transformed –developing techniques for handling exceptions in the dataflow. Structure comprises of sources, processors, sinks Sources: add data into Marmite by querying databases, extracting information from web pages, and so on. Processors: modify, combine, or delete existing rows/columns (geocoding, filtering). Sinks: redirect the flow the data out of Marmite. (showing data on a map, saving it to a file/web page) Ownership: Jason I. Hong Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Human Computer Interaction Institute School of Computer Science Carnegie Mellon University

Case study – Marmite

Case study – Dapper Dapper stands for data mapper. The main purpose of the service is to convert any type of content into standard form that can be reused (RSS, XML). It also has the set of publishing features that turn that content into Google Gadget, Netvibes Module, iCalendar, Flash widgets and so on. Visually it run is a wizard mode Results is to create a dapp (data imported). Dapps can be shared and there are many of then for popular services Dapp often is sufficient and can be tuned in map mashup or image loop Output or visualization is defined by user Next level of development is to combine those dapps into aggregator service. The typical example is to combine search result from several search engines or video clips from alternative video services similar to magg movie aggregator (

Case study – Dapper

Case study – GME Google Mashup Editor is an AJAX development framework and a set of tools that enable developers to quickly and easily create simple web applications and mashups with Google services like Google Maps and Google Base. Google Mashup Editor is a great tool for grabbing information from feeds and letting users see and manipulate it. Creating applications with Google Mashup Editor is “simple” for developer This is a software developer tools and not an end user mashup environment. Google mashup editor will be excluded from the section where we compare different mashup environment because it did not quite fit to out purpose.

Case study – GME

Case study – Mashmaker Intel® Mash Maker is an extension to your existing web browser that allows you to easily augment the page that you are currently browsing with information from other websites. As you browse the web, the Mash Maker toolbar suggests Mashups that it can apply to the current page in order to make it more useful for you. For example: plot all items on a map, or display the leg room for all flights. Community can contribute to mashup DB. Any user can teach Mash Maker new mashups, using a simple copy and paste interface, New user created mashups are suggested to other users. Relies on the community to teach it about the structure and semantics of web pages, using a built in structure editor. There is no dedicated page on the web where to have to go and construct the mashup application. All what user has to do is to install the toolbar and start browsing pages. So the paradigm could be called “annotate and mix while browsing”. Users themselves tell to Mashmaker servers about the page structures that they visit and purpose content from other sites to mix with current page. The service is in beta and not available to wide public at the moment.

Case study – Mashmaker

Case study – QEDWiki QEDWiki is a browser-based assembly canvas used to create simple mashups. The same steps: Assemble, Wire, Share A mashup maker is an assembly environment in which the creator of a mashup uses software components (or services) made available by content providers. Wiki framework in that it provides both Web users and developers with a single Web application framework for hosting and developing a broad range of Web 2.0 Sample applications –Web content management for a typical collection of Wiki pages –traditional form processing for database-oriented CRUD (Create/Read/Update/Delete) applications –document-based collaboration –rich interactive applications that bind together disparate services –situational applications (or mashups).

Case study – QEDWiki

Results + Discussions Programming paradigm Richness of functionality Technology Ease of use Openness for integration

Results table 1/2 PipesPopFlyMarmiteDapperQEDWikiMashmaker ParadigmData flow Flow + spreadsheetFlow + wizardApplication on canvas Browsing enrichment MaturityMatureMature(beta)Research projectMature Close, beta FunctionalityRich but techy Rich, integrated with services PoorLimitedRichN/A Easy of useFor coders, not for an end users Simple and friendly, makes a suggestion about linking parts together CrypticEasy, but still not 100% end user tool Easy, lazy developer tool Toolbar is simple to use if site structure know TechnologyStandard web, YUI Silverlight, proprietaryBrowser plug-in, runs in own windows, allows point at the parts of web pages to scrap Standard and/or popular WEB technologies StandardIn browser mashing up, needs a toolbar, enabled by community maintained DB of sites structure

Results table 2/2 Openness for integration Html code to embed in blog, RSS, JSON HTML code to embed n/aNew API creator. Export to Google gadget, RSS, XML, Flash Widget Can embed public WEB services and widgets N/A ExtensibilityYesYes, user can contribut e n/aDapps can be reused by other dapps New sites and widgets can be added Advanced featuresExtension by JavaScri pt ? OtherClaim to have unique web page scrapping approach

Results – programming paradigm Data flow with various visual coding –Wizard based Dapper –Visual connected functional PipesPopfly –Spreadsheet Marmite Visual widget assembly MashmakerQEDWiki

Results – richness of functionality Data sources –RSS, ATOM feeds (low level that is supported by all tools) –Page scrapping (Most tools support, except Mashmaker & QEDWiki) –Service wrappers (Pipes, POPfly) Data processing options –Feed merging –Location extraction –Numeric operation –String manipulation Visualization of results and output formats –Multiple visualization alternatives (POPFly only) –Visual appeal –Export to external services via RSS/ATOM feeds, XML, JSON –Blog integration

Results - technology AJAX – standard web technologies, runs in any browser (Pipes) Silverlight – proprietary but highly visual technology from Microsoft (Popfly) Browser add-on, dependent on browser (Mashmaker, Marmite)

Results – ease of use User roles –Mashup author –Mashup user –Environment developer Mashup author is the user of tools we analyze –Average user that uses provided blocks –Advanced user that adds modules and new services to the environment Goal – make system as simple as possible for use by average user

Results – openness for integration Can results of the mashup be reused? Popfly – supports only html code for adding to external blogs Pipes are more open -RSS, KML, JSON Dapper is designed for integration - XML, RSS, Google Gadget, Flash, Google Map, Image Loop, iCalendar Mashmaker and QEDWiki is a bit out of scope

Conclusions Mashup environments target different user groups – programmers, technology enthusiasts, non technical users Best visual tools for non technical users hide data exchange formats and provide blocks for every service that user is likely to integrate Advanced functionality will keep environments on the float by letting users provide support for new services Standard technologies are preferred New direction – enhancing browsing experience