Of 50 E GOV Universal Access Ahmed Gomaa CIMIC Rutgers University

Of 50 2 Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions

Of 50 3 Universal Access Any MM InformationOver Any Network To Any One with varying expertise, capabilities, and preferences To Any Device

Of 50 4 E GOV Universal Access Example The Entrepreneur can view on the E GOV portal a set of information resources specific to his type of business on any type of internet appliance with customizable preferences based on his appliance and his own preferences and capabilities.

Of 50 5 Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions

Of 50 6 Issues and Problems Issue: –Incorporate Temporal and spatial Synchronization and User interactivity. Problems: –What constraints shall be added to insure presentation integrity? ( Temporal and Spatial constraints ) –How to synchronize between different components in a spatio-temporal fashion? *In the user interaction cases:Forward / Rewind / Pause. – How to make sure that the synchronization constraints are still valid? –How to visualize and analyze the effect of the user interaction on the spatio-temporal constraints?

Of 50 7 Issues and Problems Issue: –Automatic detection of desynchronization and minimize the desynchronization periods automatically Problems: –Some of the multimedia object playback period is determined on the runtime (Video). So even if there are constraints on the presentation, desynchronization will appear. How can we automatically detect that desynchronization and put it to a minimum.

Of 50 8 Issues and Problems Issue: –We need to make the different networks transparent to the user with no dead time, and making the user feel as if he is watching a local presentation. Which we call Fine and Coarse grain network delay manipulation. Problems: –How to estimate the bandwidth? (Known at runtime) –How to make the network delay seamless between different networks (wireless – mobiles- cables …) can we develop a generic model that automatically adapt for changes between different networks?

Of 50 9 Issues and Problems Issue: –Adaptation of Multimedia Presentations on different hardware and software. ( OS / MM Boards / Devices ). Problems: –Resource constraints and device diversity. How can we manage that the user device will be able to handle the Incoming presentation? –How are we going to deal with the client resources? Buffer problem? Different board latency time? –The Problem will be more complex if we want to support user interaction ( FF- REW – Pause)

Of Issues and Problems Issue: –We need to make the MM information accessible based on user preferences, expertise and capabilities. Problems: –The user might need to convert some MM formats to others based on his preferences and appliance capabilities ( text to audio) –Personalize the Multimedia presentations based on user preferences, expertise and capabilities. The broker server issue arise for converting to different formats. –How to select the route for converting the MM presentation? – What will be the cost function?

Of Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions

Of Extended E GOV Universal Access PETRI-NET Modeling for validation and analysis Temporal and spatial Synchronization. User interactivity Automatic detection of desynchronization Minimize the desynchronization periods Fine and Coarse grain network delay Resource constraints and device diversity Personalization and accessibility Temporal and spatial Synchronization. User interactivity Automatic detection of desynchronization Minimize the desynchronization periods Fine and Coarse grain network delay Resource constraints and device diversity Personalization and accessibility Implementation with user/web friendly with flexibility and interoperability (SMIL 2.0) Temporal and spatial Synchronization. User interactivity Automatic detection of desynchronization Minimize the desynchronization periods Fine and Coarse grain network delay Resource constraints and device diversity Personalization and accessibility Temporal and spatial Synchronization. User interactivity Automatic detection of desynchronization Minimize the desynchronization periods Fine and Coarse grain network delay Resource constraints and device diversity Personalization and accessibility

Of Why Use Petri-Nets? The Above mentioned problems can be implemented by hard coded solution. ( SMIL 2.0) BUT How can we identify if there is a temporal or spatial conflict in case of normal playback or even in Forward or Rewind? How can we visualize the runtime of our presentation to prevent any deadlocks, resource constraints, or network constraints? SO We need a model to analyze and abstract the problems mentioned. Petri- Net has been used in the academic field as well as the industrial field to model different systems.

Of Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions

Of A medical Digital Library Object The video must start immediately after the image has been displayed. The text must be displayed simultaneously with the image and the video. [Adam et al,2001]

Of The digital library object is downloaded to the client machine.

Of Implementation

Of Implementation

Of Object Plan as MOPN Adam et al, 2001

Of DL Object Plan for the above medical object Adam et al, 2001

Of The modified plan (with only audio and text capabilities) Adam et al, 2001

Of Parallel and Sequential Adam et al, 2001

Of An Adjusted plan Adam et al, 2001

Of Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions

Of Server Oblet Adjusted Object Plan Multimedia Boards Audio/Video N/W delay Synchronization: Fine and coarse-grain Temporal Navigation + Spatial Constraints Client Different Devices User Capabilities and Accessibility Spatial& Temporal Dynamic presentation Extensions Interpretable & easy to maintain Object Plan(SMIL) Automatic adjustment of Desync. Accessibility Cost Function Resource Constraints N/W Constraints Access Control for Timesheets Server

Of Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions

Of Why SMIL XML Based. 2.Allows you to use clips in different locations. (Separate URL for each clip) 3.Time and control a presentation. (Different approaches for timing) 4.Lay out a presentation. When your presentation includes multiple clips, such as a Real Video clip playing simultaneously with text captions or banner graphics, you use SMIL to define the layout. 5.Content Control module provides alternate presentations ( multiple languages, or different bandwidths. )

Of Timing Styled TimingStyled Timing CSS or XSL stylesheets used to applytiming to a language- can be used for SMIL Timesheets Separate timing from both content and presentation style - can be used for SMIL Styled TimingStyled Timing CSS or XSL stylesheets used to applytiming to a language- can be used for SMIL Timesheets Separate timing from both content and presentation style - can be used for SMIL Inline syntaxInline syntax -Attributes added to language elements-Attributes added to language elements -Approach used in SMIL Inline syntaxInline syntax -Attributes added to language elements-Attributes added to language elements -Approach used in SMIL

Of SMIL Document Timing principals Interne t Timing is a relation between the 2 other entities rather than a property to them Contents Provides structure to the content and the links Formatting style and (spatial) layout of presenting the content Timing describes the temporal relations between the elements in the content and formatting section

Of Relation Versus Property The objective is let the list element appear each one after another. - In assign property, the order is derived from the order of the list. - In assign relation, the order has been reserved from the order of the list. This shows immediately the independence created by the principle [kate et al, 2000]

Of Timesheets may be used –Document structure and timing structure do not align –Synchronization spans multiple documents –Inline syntax impractical or illegal Copyright restrictions Digital Talking Books

Of SMIL2.0 To Petri Net Equals ( SYNC): | | image | | text | | Audio 30s After ( Before): | | 6 sec | | 4sec | | 11 seconds

Of Temporal relations and corresponding OCPN [Little et al, 1990]

Of Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions

Of Data Retrieving Engine [Yang, 2001]

Of RTSM [Yang, 2001]

Of Of 50 [Yang, 2001]

Of Fast Forward operation Of 50 [Yang, 2001]

Of Pre- Fetch table Retrieval time = (Playback period *play rate) / Estimated BW + RDdelay Of 50 [Yang, 2001]

Of Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions Of 50

41 Model Global time

Of User Interaction Modeling

Of Absolute time line The main deficiency in the absolute time line is the indeterminism of continuous objects. It might take more or less than 10 seconds for the audio file to play. – How to insure synchronization? Indeterminism problem arise. Of 50

44 Indeterminism and absolute time line Of 50 [ Layaïda et al, 2002]

Of Indeterminism Modeling Of 50

46 Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions Of 50

47

Of Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions Of 50

49 Accessibility How to render a different type of media to the client? For example, a blind person do not need to download text files, instead he needs to convert it to audio format. Three cases: Converters on the Server. Converters on the Client. Converters with middleman server Cost function? Cost function?

Of Conclusion Converting SMIL 2.0 to Petri-net for interoperability purposes. Timesheets use in SMIL2.0. User input into the Petri net model. Forward / Rewind / Pause in a PN model. Presenting Indeterminism and how to minimize desynchronization for uncontrollable objects in a PN model. Presenting both Spatial and temporal constraints in one Petri net model ( In progress) Accessibility and cost function. Of 50