CS 414 - Spring 2010 CS 414 – Multimedia Systems Design Lecture 35 – Synchronization (Part 3) Klara Nahrstedt Spring 2010.

Slides:



Advertisements
Similar presentations
Multimedia Synchronization Brian P. Bailey Spring 2006.
Advertisements

CS Spring 2009 CS 414 – Multimedia Systems Design Lecture 28 – Media Server (Part 3) Klara Nahrstedt Spring 2009.
Multimedia Presentation and Delivery Ch.13 Principles of Multimedia Database Systems. V.S. Subrahmanian, 1998.
Chapter 7 Structuring System Process Requirements
1 Synchronization TTM4142, 2007 Harald Øverby/Leif Arne Rønningen.
Page 15/20/2015 CSE 40373/60373: Multimedia Systems Notion of Synchronization  Sync in correspondence to  Content relation  Spatial relation  Temporal.
PTIDES: Programming Temporally Integrated Distributed Embedded Systems Yang Zhao, EECS, UC Berkeley Edward A. Lee, EECS, UC Berkeley Jie Liu, Microsoft.
Do You Have the Time? Composition and Linking in Time-based Hypermedia Lynda Hardman, Jacco van Ossenbruggen, K. Sjoerd Mullender, Lloyd Rutledge, and.
© Katz, 2003 Formal Specifications of Complex Systems-- Real-time 1 Adding Real-time to Formal Specifications Formal Specifications of Complex Systems.
SCA Introduction to Multimedia
© Katz, 2007 Formal Specifications of Complex Systems-- Real-time 1 Adding Real-time to Formal Specifications Formal Specifications of Complex Systems.
Introduction to Databases Transparencies
Multimedia Authoring Tools Pertemuan 8 Matakuliah: T0732 / Sistem Multimedia Tahun: 2007.
© Copyright Eliyahu Brutman Programming Techniques Course.
Petri Net Modeling for dynamic MM composite Object.
Mahapatra-A&M-Sprong'021 Co-design Finite State Machines Many slides of this lecture are borrowed from Margarida Jacome.
CS Spring 2012 CS 414 – Multimedia Systems Design Lecture 34 – Media Server (Part 3) Klara Nahrstedt Spring 2012.
Today’s Lecture Process model –initial & always statements Assignments –Continuous & procedural assignments Timing Control System tasks.
Department of Computer Science 1 CSS 496 Business Process Re-engineering for BS(CS)
Design Synopsys System Verilog API Donations to Accellera João Geada.
Chapter 7 Structuring System Process Requirements
CS Spring 2011 CS 414 – Multimedia Systems Design Lecture 28 – Media Server (Part 4) Klara Nahrstedt Spring 2011.
Multimedia Authoring Tools Pertemuan 15&16 Matakuliah: O Computer / Multimedia Tahun: Feb
CS Spring 2011 CS 414 – Multimedia Systems Design Lecture 34 – Synchronization (Part 2) Klara Nahrstedt Spring 2011.
CS370 Spring 2007 CS 370 Database Systems Lecture 2 Overview of Database Systems.
BPMN By Hosein Bitaraf Software Engineering. Business Process Model and Notation (BPMN) is a graphical representation for specifying business processes.
DEVSView: A DEVS Visualization Tool Wilson Venhola.
Integrating Timing into XML Documents Patrick Schmitz MS Research BARC Telepresence.
Page 19/21/2015 CSE 40373/60373: Multimedia Systems Reference Models  We need reference models to  Understand various requirements for multimedia sync.
1 Object-Oriented Modeling Using UML (2) CS 3331 Fall 2009.
©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 10Slide 1 Architectural Design l Establishing the overall structure of a software system.
Requirement Engineering. Review of Last Lecture Problems with requirement Requirement Engineering –Inception (Set of Questions) –Elicitation (Collaborative.
CS4730 Real-Time Systems and Modeling Fall 2010 José M. Garrido Department of Computer Science & Information Systems Kennesaw State University.
CS Spring 2012 CS 414 – Multimedia Systems Design Lecture 26 – Synchronization Issues Klara Nahrstedt Spring 2012.
CHAPTER TEN AUTHORING.
Timed Use Case Maps Jameleddine Hassine Concordia University, Montreal, Canada URN Meeting, Ottawa, January 16-18, 2008.
1 Constraints for Multimedia Presentation Generation Joost Geurts, Multimedia and Human-Computer Interaction CWI Amsterdam
©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 10Slide 1 Architectural Design l Establishing the overall structure of a software system.
Chapter 10 Analysis and Design Discipline. 2 Purpose The purpose is to translate the requirements into a specification that describes how to implement.
Mahapatra-A&M-Fall'001 Co-design Finite State Machines Many slides of this lecture are borrowed from Margarida Jacome.
1 Structuring Systems Requirements Use Case Description and Diagrams.
MACROMEDIA DIRECTOR - LECTURE NOTES -. INTRODUCTION Macromedia Director 8.5 is the best selling multimedia authoring program and leading tool for creating.
Sketchmate: A Computer-Aided Sketching and Simulation Tool for Teaching Graph Algorithms Dissertation Proposal Kristy VanHornweder April 11, 2011.
Of 50 E GOV Universal Access Ahmed Gomaa CIMIC Rutgers University.
CS Spring 2012 CS 414 – Multimedia Systems Design Lecture 25 – Synchronization Issues Klara Nahrstedt Spring 2012.
CS Spring 2011 CS 414 – Multimedia Systems Design Lecture 33 – Synchronization (Part 1) Klara Nahrstedt Spring 2011.
SOFTWARE DESIGN AND ARCHITECTURE LECTURE 13. Review Shared Data Software Architectures – Black board Style architecture.
System Testing Beyond unit testing. 2 System Testing Of the three levels of testing, system level testing is closest to everyday experience We evaluate.
Introduction to Interactive Media Interactive Media Tools: Authoring Applications.
CS4730 Real-Time Systems and Modeling Fall 2010 José M. Garrido Department of Computer Science & Information Systems Kennesaw State University.
Course: COMS-E6125 Professor: Gail E. Kaiser Student: Shanghao Li (sl2967)
Reasoning about the Behavior of Semantic Web Services with Concurrent Transaction Logic Presented By Dumitru Roman, Michael Kifer University of Innsbruk,
Lesson 3-Multimedia Skills. Overview Members of a multimedia team. Roles and responsibilities in a multimedia team.
CS Spring 2009 CS 414 – Multimedia Systems Design Lecture 31 – Process Management (Part 1) Klara Nahrstedt Spring 2009.
Multimedia Synchronization I. Fatimah Alzahrani. Definitions Multimedia System : A system or application that supports the integrated processing of several.
Multimedia Retrieval Architecture Electrical Communication Engineering, Indian Institute of Science, Bangalore – , India Multimedia Retrieval Architecture.
Chapter 7 Part II Structuring System Process Requirements MIS 215 System Analysis and Design.
Appendix Object-Oriented Analysis and Design: Use Cases and Sequence Diagrams Modern Systems Analysis and Design Fifth Edition Jeffrey A. Hoffer Joey F.
CS Spring 2009 CS 414 – Multimedia Systems Design Lecture 27 – Media Server (Part 2) Klara Nahrstedt Spring 2009.
CS Spring 2009 CS 414 – Multimedia Systems Design Lecture 34 – Synchronization (Part 2) Klara Nahrstedt Spring 2009.
Klara Nahrstedt Spring 2009
Business System Development
Klara Nahrstedt Spring 2009
Klara Nahrstedt Spring 2009
Intelligent Information System Lab
Multimedia Authoring Tools
Data Model.
Implementation support
COT 5611 Operating Systems Design Principles Spring 2014
Implementation support
Presentation transcript:

CS Spring 2010 CS 414 – Multimedia Systems Design Lecture 35 – Synchronization (Part 3) Klara Nahrstedt Spring 2010

Administrative MP4 is due May 1, competition May 3 Peer evaluation material is on the website (see  Deadline: May 7 Homework 2 will be out on Monday, April 26  Deadline, May 5, 11:59pm. CS Spring 2010

Outline Synchronization Specification Methods  Requirements  Interval-based Specification  Axes-based Specification  Control Flow-based Specification hierarchical approach Timed Petri nets  Event-based Specification CS Spring 2010

Sync Specification Methods - Requirements Object consistency and maintenance of sync specifications  Media objects should be kept as one LDU in spec Temporal relations must be specify-able Easy Description of Sync Relations Definition of QoS requirements Integration of time-dependent and independent media Hierarchical levels of synchronization CS Spring 2010

Models Interval Timeline Hierarchical Reference points Petri net Event-based Common threads  provide language to express relationships  runtime system to monitor relationships  policies to enforce relationships CS Spring 2010

Interval-based Specification (1) Presentation duration of an object is specified as interval Types of temporal relations:  A before B, A overlaps B, A starts B, A equals B, A meets B, A finishes B, A during B Enhanced interval-based model includes 29 interval relations, 10 operators handle temporal relations (e.g., before(δ1),…) CS Spring 2010

Interval Model (2) 13 relationships between two intervals Before A B Meets A B During A B Overlaps A B Starts A B Ends A B Equal A B CS Spring 2010

Example (3) Audio1 while(0,0) Video Audio1 before(0) RecordedInteraction RecordedInteraction before(0) P1 P1 before(0) P2 P2 before(0) P3 P3 before(0) Interaction P3 before(0) Animation Animation while(2,5) Audio2 Interaction before(0) P4 CS Spring 2010

Interval-based Specification (4) Advantages:  Easy to handle open LDUs (i.e., user interactions)  Possible to specify additional non- deterministic temporal relations by defining intervals for durations and delays  Flexible model that allows specification of presentations with many run-time presentation variations CS Spring 2010

Interval-based Specification (5) Disadvantages:  Does not include skew spec  Does not allow specification of temporal relations directly between sub-units of objects  Flexible spec leads to inconsistencies Example: A NOT in parallel with B A while(2,3) I I before(0) B CS Spring 2010

Timeline Axis-based Specification Presentation events like start and end of presentation are mapped to axes that are shared by presentation objects All single medium objects are attached to time axis that represents abstraction of real-time This sync specification is very good for closed LDUs CS Spring 2010

Timeline Model (2) Uses a single global timeline Actions triggered when the time marker reaches a specific point along timeline CS Spring 2010

Example (3) Define a timed sequence of images, each image has a caption that goes with it I1I1 C1C1 t1t1 I2I2 C2C2 t2t2 I3I3 C3C3 t3t3 CS Spring 2010

Example (4) Rule language  At (t1), show (I1, C1)  At (t2), show (I2, C2)  At (t3), show (I3, C3) Visual environment CS Spring 2010

Control Flow-based Spec - Hierarchical Model (1) Possibility to specify concurrent presentation threads at predefined points of presentation Basic hierarchical spec types:  Serial synchronization  Parallel synchronization of actions Actions: atomic or compound  Atomic action handles presentation of single media object, user input, delay  Compound actions are combinations of sync operators and atomic actions  Delay is atomic action – allows modeling of delays in serial presentations CS Spring 2010

Hierarchical Model (SMIL) Based on sequential and parallel Apply operators to only the start/end points of each media object I1I2I3I1T1 CS Spring 2010

Example (3) Narrated slide show  image, text, audio on each slide  select link to move to the next slide S1A1T1 … I1 S2A2T2I2 CS Spring 2010

Example (4) (and Comparison with Interval- based Spec) CS Spring 2010 Audio1 while(0,0) Video Audio1 before(0) RecordedInteraction RecordedInteraction before(0) P1 P1 before(0) P2 P2 before(0) P3 P3 before(0) Interaction P3 before(0) Animation Animation while(2,5) Audio2 Interaction before(0) P4

Control Flow-based Spec – Hierarchy (5) Advantages  Easy to understand  Natural support for hierarchies  Integration of interactive object easy Disadvantage  Need additional descriptions of skews and QoS  No duration description CS Spring 2010

Control Flow-based Spec – Reference Points (1) Time-dependent single medium objects are regarded as sequences of closed LDUs Start/stop times of object presentation are reference points Connected reference point is synchronization point Temporal relations specified between objects without explicit reference to time CS Spring 2010

Example (2) CS Spring 2010

Control Flow-based Spec – Reference Points (3) Advantages:  Sync at any time during presentation of objects  Easily integrated object presentation with unpredictable duration  Intuitive type of synchronization spec Disadvantages:  Not easy way to detect inconsistencies  Cannot specify delays in presentation CS Spring 2010

Event-based Specification Presentation actions initiated by synchronization events Example:  Start presentation  Stop presentation  Prepare presentation Events initiating presentation  External or internal CS Spring 2010

Event-based Spec Advantage:  Easily extended to new sync types  Easy integration of interactive objects Disadvantage:  Difficult to handle in case of realistic scenarios  Too complex specification  Need separate description of skew/QoS  Difficult use of hierarchies CS Spring 2010

Associate actions with expressions Expressions may contain scalars, clocks, variables, relations, and connectives When the expression becomes TRUE, invoke associated action When “Time > Q.end + 5 && !Response” Answer=WRONG Event Model (Nsync) Source: B. Bailey et al. “Nsync- A Toolkit for Building Interactive Multimedia Presentations”, ACM Multimedia 1998 CS Spring 2010

Background and Time Model Each media object attached to a clock Clock implements logical time  Media-time = Speed * Real-Time + Offset Speed (S) – ratio of media-time progression to that of real-time  E.g., a speed of 2.0 for cont. media indicates that the media is being played at twice its normal playout rate Express temporal behavior as relationships among clocks Interactive events tied to variables CS Spring 2010

Graph Demonstrating Media- time Mapping CS Spring 2010

Overview No Yes MoreInfo? DetailedNarration More Info Example: Delayed Transition CS Spring 2010

Model Specification When “Narration >= Overview && ! MoreInfo” NextSlide When “Narration >= Overview && MoreInfo” PlayDetails When “Narration >= Overview + Details” NextSlide Narration : narration’s logical timeline Overview : normal transition point Details : additional narrative details MoreInfo : records kitchen info status CS Spring 2010

Reactive Interface CS Spring 2010

Model Specification When “Video >= 0 && Video < T1” Select Kitchen When “Video >= T1 && Video < T2” Select Deck When “Video >= T2 && Video <= T3” Select Yard CS Spring 2010

Example (Virtual Murder Mystery) CS Spring 2010

Conclusion Synchronization Specifications  Important for different authoring tools for complex presentation  Be careful as you go from one spec to another  Carefully consider which spec closest allows you to specify sync requirements in your application CS Spring 2010