Of 50 E GOV Universal Access Ahmed Gomaa CIMIC Rutgers University

Of 50 2 Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of 50 3 Universal Access Any MM InformationOver Any Network To Any One with varying expertise, capabilities, and preferences To Any Device

Of 50 4 E GOV Universal Access Example The Entrepreneur can view on the E GOV portal a set of information resources specific to his type of business on any type of internet appliance with customizable preferences based on his appliance and his own preferences and capabilities.

Of 50 5 Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of 50 6 A medical Digital Library Object The video must start immediately after the image has been displayed. The text must be displayed simultaneously with the image and the video. [Adam et al,2001]

Of 50 7 The digital library object is downloaded to the client machine.

Of 50 8 Implementation

Of 50 9 Implementation

Of Object Plan as MOPN Adam et al, 2001

Of DL Object Plan for the above medical object Adam et al, 2001

Of The modified plan (with only audio and text capabilities) Adam et al, 2001

Of Parallel and Sequential Adam et al, 2001

Of An Adjusted plan Adam et al, 2001

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Issues and Problems Issue: –Synchronization and User interactivity. Problems: *In the user interaction cases:Forward / Rewind / Pause. – How to make sure that the synchronization constraints are still valid? –How to visualize and analyze the effect of the user interaction on the temporal constraints?

Of Issues and Problems Issue: –Automatic detection of desynchronization and minimize the desynchronization periods automatically Problems: –Some of the multimedia object playback period is determined on the runtime (Video). So even if there are constraints on the presentation, desynchronization will appear. How can we automatically detect that desynchronization and put it to a minimum?

Of Issues and Problems Issue: –We need to make the MM information accessible based on user preferences, expertise and capabilities. Problems: –The user might need to convert some MM formats to others based on his capabilities ( text to audio) –Personalize the Multimedia presentations based on user preferences, expertise and capabilities. The broker server issue arise for converting to different formats. –How to select the route for converting the MM presentation? – What will be the cost function?

Of Presentation outline Universal access definition and example Issues and problems Methodology Previous work Extensions proposed SMIL and timing SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Conclusions

Of Extended E GOV Universal Access PETRI-NET Modeling for validation and analysis Temporal Synchronization and User interactivity Automatic detection of desynchronization Minimize the desynchronization periods Personalization and accessibility Temporal Synchronization and User interactivity Automatic detection of desynchronization Minimize the desynchronization periods Personalization and accessibility Implementation with user/web friendly with flexibility and interoperability (SMIL 2.0) Temporal Synchronization with User interactivity Automatic detection of desynchronization Minimize the desynchronization periods Personalization and accessibility Temporal Synchronization with User interactivity Automatic detection of desynchronization Minimize the desynchronization periods Personalization and accessibility

Of Why Use Petri-Nets? The Above mentioned problems can be implemented by hard coded solution. ( SMIL 2.0) BUT How can we identify if there is a temporal or spatial conflict in case of normal playback or even in Forward or Rewind? How can we visualize the runtime of our presentation to prevent any deadlocks, resource constraints, or network constraints? SO We need a model to analyze and abstract the problems mentioned. Petri- Net has been used in the academic field as well as the industrial field to model different systems.

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Server Oblet Adjusted Object Plan N/W delay Synchronization: Fine and coarse-grain Temporal Navigation ( User Interaction) Client Temporal Dynamic presentation Extensions Interpretable & easy to maintain Object Plan(SMIL) Automatic adjustment of Desync. Accessibility Cost Function

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Why SMIL XML Based. 2.Allows you to use clips in different locations. (Separate URL for each clip) 3.Time and control a presentation. (Different approaches for timing) 4.Lay out a presentation. When your presentation includes multiple clips, such as a Real Video clip playing simultaneously with text captions or banner graphics, you use SMIL to define the layout. 5.Content Control module provides alternate presentations ( multiple languages, or different bandwidths. )

Of SMIL2.0 To Petri Net Equals ( SYNC): | | image | | text | | Audio 30s After ( Before): | | 6 sec | | 4sec | | 11 seconds

Of Temporal relations and corresponding OCPN [Little et al, 1990]

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Data Retrieving Engine [Yang, 2001]

Of RTSM [Yang, 2001]

Of Of 50 [Yang, 2001]

Of Fast Forward operation Of 50 [Yang, 2001]

Of Pre- Fetch table Retrieval time = (Playback period *play rate) / Estimated BW + RDdelay Of 50 [Yang, 2001]

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Model Global time

Of User Interaction Modeling

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Absolute time line The main deficiency in the absolute time line is the indeterminism of continuous objects. It might take more or less than 10 seconds for the audio file to play. – How to insure synchronization? Indeterminism problem arise. Of 50

39 Indeterminism and absolute time line Of 50 [ Layaïda et al, 2002]

Of Indeterminism Modeling Of 50

41

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Accessibility How to render a different type of media to the client? For example, a blind person do not need to download text files, instead he needs to convert it to audio format. Three cases: Converters on the Server. Converters on the Client. Converters with middleman server Cost function? Cost function?

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Future Work Issue: –Incorporate Temporal and spatial constraints Problems: –What constraints shall be added to insure presentation integrity? ( Temporal and Spatial constraints ) –How to synchronize between different components in a spatio-temporal fashion?

Of Future Work Issue: –We need to make the different networks transparent to the user, and making the user feel as if he is watching a local presentation. Which we call Fine and Coarse grain network delay manipulation. Problems: –How to estimate the bandwidth? (Known at runtime) –How to make the network delay seamless between different networks (wireless – mobiles- cables …) can we develop a generic model that automatically adapt for changes between different networks?

Of Future Work Issue: –Adaptation of Multimedia Presentations on different hardware and software. ( OS / MM Boards / Devices ). Problems: –Resource constraints and device diversity. How can we manage that the user device will be able to handle the Incoming presentation? –How are we going to deal with the client resources? Buffer problem? Different board latency time?

Of Server Oblet Adjusted Object Plan Multimedia Boards Audio/Video N/W delay Client Different Devices Future Work Interpretable & easy to maintain Object Plan(SMIL) Automatic adjustment of Desync. Resource Constraints N/W Constraints Access Control for Timesheets Server Synchronization: Fine and coarse-grain Temporal Navigation + Spatial Constraints Spatial & Temporal Dynamic presentation Accessibility Cost Function

Of Presentation outline Universal access definition and example Previous work Issues and problems Methodology Extensions SMIL SMIL engine PETRI-NET model for user interaction PETRI-NET model for automatic detection and minimization of desynchronization periods Accessibility Future Work Conclusions

Of Conclusion Converting SMIL 2.0 to Petri-net for interoperability purposes. Timesheets use in SMIL2.0. User input into the Petri net model. Forward / Rewind / Pause in a PN model. Presenting Indeterminism and how to minimize desynchronization for uncontrollable objects in a PN model. Presenting both Spatial and temporal constraints in one Petri net model ( In progress) Accessibility and cost function. Of 50