Smart Style on the Semantic Web Jacco van Ossenbruggen Lynda Hardman CWI Amsterdam

2/20 Talk overview Requirements for 2nd & 3rd generation multimedia –why multimedia is different from text Smart style –a document engineer meets a graphic designer Overview of CWI’s Cuypers engine –testbed for multimedia transformations Future directions –towards 3rd generation multimedia

3/20 The Web in three generations 1.Hand-coded (HTML) Web content –easy access through uniform interface –huge authoring and maintenance effort –hard to deal with dynamically changing content 2.Automated on-the fly content generation –based on templates filled with database content –later extended with XML document transformations 3.Automated processing of content –The Semantic Web –explicit meta-data instead of “screen scraping” –agreed upon semantics (RDFS, DAML+OIL)

4/20 Multimedia on the Web Real multimedia Web content is still rare –Mostly bells & whistles to enhance HTML text … –… or mono-media AV-streams Virtually all presentations are hand-authored –proprietary formats that are hard to generate –limited support for dynamic content and multichanneling –most Web technology is text/page-oriented … –… with SMIL as one of the few exceptions Conclusion: Multimedia has hardly caught up with the 1st generation Web!

5/20 2 nd generation multimedia Adapts to end-user’s platform capabilities –multichanneling: PC, PDA, mobile, voice-only,... Adapts to the network resources available –bandwidth and other QoS parameters Personalization –language, abilities, level of expertise,...

6/20 But multimedia is not text... Different document and presentation abstractions –hard to separate style from structure Formatting is not based on text flow –no pages or scrollbars, no line-breaking or hyphenation –templates often do not work well either Feedback from the formatting back-end required –need to check whether proposed layout is feasible –layout of media items is less flexible than text-based layout Transformations are hard in a functional language –need to try out designs and backtrack when necessary

7/20 Problem Current document transformation and style languages are insufficiently powerful They rely on flexibility of text: re-flow, scrollbars, pagination, etc. They are "template-based” and thus cannot cater for wide variations in: dynamic & media-centric content device characteristics user preferences

8/20 Solution (our approach) The creation of a web-based infrastructure that is –an extension of the current document engineering perspective –taking into account the graphic design perspective

9/20 Document Engineering Perspective Content is selected & structured Mappings are defined to a new presentation structure Styles (such as color and font) can be applied The transformation process is linear Assume that –Content/document structure, –presentation structure and, –style are independent of each other.

10/20 Document Engineering Perspective Document structure Presentation structure Styled presentation Final form Transformation Sheet (XSLT) Stylesheet (CSS) Rendering (Web-device) HTML WML Post Script

11/20 Graphic Design Perspective Basically: presentation structure, content and style depend on one another. In multimedia presentations, spatio- temporal layout gives meaning to the presentation (in contrast to the "linear" nature of text-flow).

12/20 Design dependencies Presentation structure Style Content Grouping deter- mines selection of media items Grouping depends on semantic relations among media items Style determined by selected media items Style determines selection of media items Top-level order/grouping affects local arrangements Overall style determines local style of media items Grouping deter- mines layout style Layout aesthetics determines grouping

13/20 Example

14/20 External Forces on Design Process Content provider –Mission (make profit, promote image) –Limited resources (cost) –Preferences (company colors) End-user –Goals, needs –Delivery context limitations (time, environment) –Preferences (images vs text, audio vs visual) Designer –Design experience –Resource limitations

15/20 Requirements for Smart Style Integrate Semantic Web technology with current Web engineering technology: Communicating delivery contexts Supporting metadata for content and description of its function Encapsulating explicit design knowledge Processing delivery context, metadata and design knowledge within the document engineering pipeline.

16/20 Cuypers multimedia transformation engine Cuypers testbed is based on –media independent presentation abstractions –transformation rules with built-in backtracking and constraint solving

17/20 Cuypers Web Embedding Prolog-based engine embedded in Apache XML-based input/output stream Java servlets for XML to Prolog translation

18/20 Cuypers Web Embedding Prolog-based engine embedded in Apache XML-based input/output stream Java servlets for XML to Prolog translation … but all knowledge is implicit and hidden in the rules lost in the generated Web presentation not reusable for other Web sites

19/20 Potential Knowledge Sources

20/20 Conclusions Multimedia is mostly first generation –hand-authored presentations –common Web-tools are too text-centric Cuypers system realises second generation –on-the-fly multimedia generation –media-centric transformations Third generation is around the corner –generating annotated multimedia –reusing knowledge available on the Semantic Web