1. Clarke, R. J (2001) L213-08: 1 Multimedia in Organisations BUSS 213 Lecture 8 Modelling Multimedia Applications

2. Clarke, R. J (2001) L213-08: 2 Notice Assignment 1 Results Pick up your assignment 1 if you haven’t already done so: check the marks sheet and make sure that marks have been included for all categories (where relevant) and that it is added up correctly make sure that the total shown on the cover is the same as that shown on the results list (on my office door) the final grade for the assignment will be rounded to an integer

3. Clarke, R. J (2001) L213-08: 3 Agenda (1) in previous lectures we have considered the different types of media used in multimedia systems and the types of transformations performed on them we have also considered multimedia systems from the point of view of the kind of design considerations used when building the interface this lecture we consider what is behind the interface, the software architecture (abstractions) for multimedia implementation…

4. Clarke, R. J (2001) L213-08: 4 Agenda (2) we consider the design of multimedia applications from an object-oriented approach (OOHDM) then consider some of the types of building blocks or classes used in multimedia frameworks these frameworks consist of media types and their properties and operations are also viewed using object-oriented programming

5. Clarke, R. J (2001) L213-08: 5 Object-Orientation

6. Clarke, R. J (2001) L213-08: 6 Object-orientation In this lecture we consider the use of so- called ‘object-oriented’ development techniques as the basis for: Design of multimedia applications Implementation of Multimedia Applications we describe both the design and implementation aspects of multimedia as ‘modelling’ for our purposes in this lecture we emphasis design

7. Clarke, R. J (2001) L213-08: 7 Object-orientation thankfully however the implementation details of multimedia systems are hidden by the high-level tools we use during development (Director etc) in this section we describe the general characteristics of object-orientation- these are broadly applicable in other development contexts as well object-oriented approaches are a natural fit with multimedia systems- there are several reasons for this

8. Clarke, R. J (2001) L213-08: 8 Object-orientation Main Properties (after Borland 1991) there are three main properties which characterise object-oriented development and languages which support object-orientation: Encapsulation: combining a record with the procedures and functions that manipulate it to form a new data type- an object Inheritance: defining an object and then using it to build a hierarchy of descendant objects, with each descendant inheriting access to all its ancestors’ code and data Polymorphism: given an action one name that is shared up and down an object hierarchy, with each object in the hierarchy implementing the action in a way appropriate to itself

9. Clarke, R. J (2001) L213-08: 9 Object-orientation Inheritance (after Borland 1991)… as with other sciences and technologies a great deal of analysis and development is about classification- the creation of distinctions and there organisation into taxonomies consider the category of ‘insect’, which has two divisions: Insects with visible wings, and Insects with hidden or no wings

10. Clarke, R. J (2001) L213-08: 10 Object-orientation … Inheritance (after Borland 1991)… ‘winged insects’ contain a large number of categories including moths, butterflies, flies etc. and each category has many subcategories a scientist asks several questions when trying to classify a new animal or object How is it similar to the others of its general class? How is it difficult?

11. Clarke, R. J (2001) L213-08: 11 Object-orientation … Inheritance (after Borland 1991) … each different class has a set of behaviours and characteristics that define it a scientist begins at the top of the tree and asks questions along the way eg. ‘wings’ or ‘no wings’ from the most general to the most specific once a characteristic is defined all the categories beneath that definition include that characteristic…

12. Clarke, R. J (2001) L213-08: 12 Object-orientation … Inheritance (after Borland 1991) once a new insect has been identified as a member of ‘flies’ you don’t need to identify your fly as having wings! the new insect is said to inherit that characteristic from ‘flies’ insects winglesswinged butterfliesfliesmoths new insect

13. Clarke, R. J (2001) L213-08: 13 Object-orientation Polymorphism (after Borland 1991) … polymorphism is based on the Greek meaning ‘many shapes’- a way of giving an action one name that is shared up and down an object hierarchy with each object in the hierarchy implementing the action in a way appropriate to itself the winged insects taxonomy is a good example of polymorphism in action- we would like each insect to be drawn on the screen for example- in fact this is a desirable property for all of our electronic insects…

14. Clarke, R. J (2001) L213-08: 14 Object-orientation … Polymorphism (after Borland 1991) …but what is different for each object type is the way it must show itself on the screen for example bees have stripes on their body, but dragonflies don’t part of the basic definition of insect would also implement so called virtual method called ‘Show’- which is resolved at program execution to display on the screen a bee rather than a dragonfly

15. Clarke, R. J (2001) L213-08: 15 Multimedia Design

16. Clarke, R. J (2001) L213-08: 16 Multimedia Design Object Oriented Hypermedia Design Method an Object Oriented Hypermedia Design Method (OOHDM) has evolved over recent years which applies a set of object- oriented design primitives to hypermedia design Comprises four different activities: Conceptual Design Navigation Design Abstract Interface Design, and Implementation

17. Clarke, R. J (2001) L213-08: 17 Multimedia Design OOHDM Activities … each activity is performed as a mix of incremental, iterative and prototype-based development styles during each activity a set of object- oriented models describing particular design concerns are built from previous iterations Conceptual, navigation and interface design are separated in order to allow developers to concentrate on each one at a time

18. Clarke, R. J (2001) L213-08: 18 Multimedia Design … OOHDM Activities the result of this separation is modular, reusable designs and produces a framework for reasoning about the design process while encapsulating design experience specific to each activity design primitives can be mapped directly into non-object-oriented implementation languages or environments (eg. HTML) or into pure object- oriented environments we describe each OOHDM’s activity in detail (we exclude implementation- considered latter)

19. Clarke, R. J (2001) L213-08: 19 Multimedia Design OOHDM Conceptual Modelling (1) Goal is to build a model of the application domain using object- oriented modelling principles and notations(like UML) Output is a class schema where conceptual classes are built using aggregation and hierarchies like the taxonomy shown previously

20. Clarke, R. J (2001) L213-08: 20 Multimedia Design OOHDM Conceptual Modelling (2) Modelling conceptual entities as objects allows behaviour to be encapsulated inside them and therefore improves the design of complex applications Conceptual models don’t just model the ‘domain’ of users and tasks but also can include complex algorithms and transactions applied to computational entities- permits a broad range of applications to be modelled

21. Clarke, R. J (2001) L213-08: 21 Multimedia Design OOHDM Navigation Design OOHDM does not prescribe any particular procedure to create navigation designs- but it does provide a method to iteratively develop them Uses scenarios as a means to do this… 1. Determine user profiles (types of users), identify user tasks 2. Scenario collection 3. Analysis of scenarios- create a simple representation of the navigation path described by the scenario (the designer will have to fill in incomplete information, such as implied indexes, missing orderings, exception handling, etc..) 4. Synthesis of a partial context diagram, specifying navigation in contexts that support the task described in the scenario 5. Synthesis of final context diagrams through a process of union and amalgamation of partial schemas, produced in the analysis of each scenario (in Step 4)

22. Clarke, R. J (2001) L213-08: 22 Multimedia Design OOHDM Abstract Interface Design in the stage the interface objects that will be perceived by the user are defined there is a distinction between navigation operations and interface operations- not everything in an interface is navigation oriented it it useful to design interfaces in an abstract way- to achieve for example independence in the implementation environment

23. Clarke, R. J (2001) L213-08: 23 Multimedia Design OOHDM Design Methods (Rossi et al 1999) Conceptual Model Navigation Views Object Oriented Views of Conceptual Model Interface Model (Presentation)

24. Clarke, R. J (2001) L213-08: 24 Multimedia Framework

25. Clarke, R. J (2001) L213-08: 25 Multimedia Framework multimedia represents a challenging type of programming with respect to detailed technical implementation because of: the variety of different types of media that have to be handled and the fact that the characteristics of the hardware change all the time (non-standardisation as well as technical development) an advantage of OOP is that it can encapsulate information that is hide details of the particulars of a given media and the specifics of particular hardware programmers only need to know what is referred to confusingly as the interface (object NOT user interface) rather than the details of the object implementation

26. Clarke, R. J (2001) L213-08: 26 Multimedia Framework Digital Content Workflows and Modularity the workflows used in developing multimedia content are increasingly becoming digital (as we have seen in previous lectures for example when we discussed VR) modern digital content workflows now include what used to be standalone and isolated devices- these devices are becoming virtually integrated together at the desk top or are now available as software rather than hardware OOP can handle the complex interfacing needed to support interconnectivity and modularity between functions in digital content workflows- the interface between for example an audio effects processor (Sound Forge) and a digital sequencer (CakeWalk)

27. Clarke, R. J (2001) L213-08: 27 Multimedia Framework Extensibility & Portability the requirements of multimedia systems are constantly evolving and relatively non-standard- this means that there will always be a need to extend applications in ways which are not always predictable OOP supports this so called extensibility by providing standard ways in which programs can be enhanced multimedia systems should be cross-platform as much as this is possible- in OOP the platform dependencies are made explicit in order for the code to be made portable - this means that the application need not be completely rewritten as demonstrated in the following slides…

28. Clarke, R. J (2001) L213-08: 28 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Platform 2 Abstract classes Concrete classes Platform 3 Abstract classes Concrete classes Platform 1 Abstract classes Specialisation Environments Framework Systems programming interfaces Application programming interface (API)

29. Clarke, R. J (2001) L213-08: 29 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Platform 2 Abstract classes Concrete classes Platform 3 Abstract classes Concrete classes Platform 1

30. Clarke, R. J (2001) L213-08: 30 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Abstract classes Concrete classes Abstract classes Concrete classes Platform 2Platform 3 Platform 1

31. Clarke, R. J (2001) L213-08: 31 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Abstract classes Concrete classes Abstract classes Concrete classes Platform 2Platform 3 Platform 1

32. Clarke, R. J (2001) L213-08: 32 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Abstract classes Concrete classes Abstract classes Concrete classes Platform 2Platform 3 Platform 1

33. Clarke, R. J (2001) L213-08: 33 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Abstract classes Concrete classes Abstract classes Concrete classes Platform 2Platform 3 Platform 1

34. Clarke, R. J (2001) L213-08: 34 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Abstract classes Concrete classes Abstract classes Concrete classes Platform 2Platform 3 Platform 1

35. Clarke, R. J (2001) L213-08: 35 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Abstract classes Concrete classes Abstract classes Concrete classes Platform 2Platform 3 Platform 1

36. Clarke, R. J (2001) L213-08: 36 Object-Oriented Multimedia Specialising a Framework Abstract classes Concrete classes Abstract classes Concrete classes Abstract classes Concrete classes Platform 2Platform 3 Platform 1

37. Clarke, R. J (2001) L213-08: 37 Object-Oriented Multimedia Specialising a Framework Classes Platform 2Platform 3 Platform 1

38. Clarke, R. J (2001) L213-08: 38 Object-Oriented Multimedia Specialising a Framework Platform 2Platform 3 Platform 1 Classes

39. Clarke, R. J (2001) L213-08: 39 Object-Oriented Multimedia Specialising a Framework Platform 2Platform 3 Platform 1 Classes

40. Clarke, R. J (2001) L213-08: 40 Object-Oriented Multimedia Specialising a Framework Platform 2Platform 3 Platform 1 Classes

41. Clarke, R. J (2001) L213-08: 41 Object-Oriented Multimedia Specialising a Framework Classes Platform 2Platform 3 Platform 1

42. Clarke, R. J (2001) L213-08: 42 Object-Oriented Multimedia Specialising a Framework Classes Platform 2Platform 3 Platform 1

43. Clarke, R. J (2001) L213-08: 43 Application Object-Oriented Multimedia Specialising a Framework Classes Platform 2Platform 3 Platform 1

44. Clarke, R. J (2001) L213-08: 44 Media Classes

45. Clarke, R. J (2001) L213-08: 45 Media Classes (1) the low-level implementation of distinct types of media- that is the computer-based representation of media- can also described in object-oriented terms in this section we consider how to model different media types and there properties and operations using object-oriented programming the resulting building blocks are referred to as classes which can then be directly included into programs or further extended

46. Clarke, R. J (2001) L213-08: 46 Media Classes (2) Gibbs and Tsichritzis (1995) in a now ‘classic work’ showed how media could be considered in detail using a multimedia framework which includes: Media classes Transform Classes Format Classes Component Classes here we only briefly describe media classes in object oriented terms (in the following slides) however the student with a technical background and interest is referred to Gibbs and Tsichritzis (1995) for further discussion

47. Clarke, R. J (2001) L213-08: 47 Media Classes (3) Objects, Instance Variables, Methods … Objects are a programming constructs that encapsulate or contain its state- a collection of data items- and its behaviour- collection of operations the object’s data items are called instance variables and its operations are called its methods Object: myMp3Player Instance Variable: currentTrack Method: PlayTrack(trackNumber)

48. Clarke, R. J (2001) L213-08: 48 Media Classes (4) … Objects, Instance Variables, Methods … invoking or using a method can be thought of as sending a message to an object this message contains both the method name and any arguments needed by the method the syntax used to invoke the method method differs from language to language (and the stuff of vigorous debate) but the following form of object name followed by method- is commonly used: myMp3Player.PlayTrack(4)

49. Clarke, R. J (2001) L213-08: 49 Media Classes (5) … Objects, Instance Variables, Methods … the result of invoking a method often leads to a change of state of the object for example currentTrack instance variable is now 4 in fact in many OOP languages, the methods of objects are often the only interface by which objects are manipulated- you cannot modify an objects instance variables directly

50. Clarke, R. J (2001) L213-08: 50 Media Classes (6) Class Hierarchy Diagrams following the work of Gibbs and Tsichritzis (1995) classes can be shown as class hierarchy diagrams ovals are classes lines depict superclass/subclass relationships the line points from superclass to the subclass classes (ovals) in dark blue are abstract classes while those in pale pink are concrete classes media are divided into types- each type is represented by a class…

51. Clarke, R. J (2001) L213-08: 51 Media Classes (7) …Class Hierarchy Diagrams Media TextGraphicImage TemporalMedia AudioVideoMusicAnimation

52. Clarke, R. J (2001) L213-08: 52 Media Classes (8) Media Objects… instances of media classes are called media objects a media object consists of a descriptor and and a media value descriptors bundle together attributes of media objects such as their values correspond to the actual data used to represent artifacts

53. Clarke, R. J (2001) L213-08: 53 Media Classes (9) … Media Objects… 010010 010110 010001 Media value (media data) Media artifact Media descriptor represents describes Media Object

54. Clarke, R. J (2001) L213-08: 54 Media Classes (10) General Categories of Methods most general media classes have two general categories of methods: querying- allows media to determine the media types that are present in an environment editing- cut, copy and paste operations for any general media type

55. Clarke, R. J (2001) L213-08: 55 Media Classes (11) Video Classes & Sequence Elements RawVideo VideoCompressedVideo MpegVideo Frame VideoElementCompressedFrame MpegFrame Instances of Classes Corresponding Classes

56. Clarke, R. J (2001) L213-08: 56 Media Classes (12) Audio Classes & Sequence Elements RcmAudio Audio CompressedAudio AdpcmAudio Sample AudioElement CompressedSample AdpcmSample Instances of Classes Corresponding Classes Adpcm and AdpcmSample are abstract classes

57. Clarke, R. J (2001) L213-08: 57 Media Classes (13) Animation Classes & Sequence Elements Instances of Classes Corresponding Classes Adpcm and AdpcmSample are abstract classes Animation EventBasedAnimationSceneBasedAnimation AnimationElement AnimationEventScene

58. Clarke, R. J (2001) L213-08: 58 Media Classes (14) Music Classes & Sequence Elements EventBasedMusic MidiMusic ScoreElement MusicElement MidiEvent Instances of Classes Corresponding Classes ScoreBasedMusicMusic MusicEvent

59. Clarke, R. J (2001) L213-08: 59 References Gibbs, S. J. and D. C.Tsichritzis (1995) Multimedia Programming: Objects, Environments and Frameworks Chapter 2: Media Types, ACM Press Books/Addison-Wesley Publishing Company, 15-78 Rossi, G.; Schwabe, D. and F. Lyardet (1999) “Integrating Patterns into the hypermedia development process” The New Review of Hypermedia and Multimedia 1999 (3), 59-80 Borland, Inc. (1991) Turbo Pascal for Windows: User Guide Borland International