Media Frameworks CS 445/656 Computer & New Media

Defining New Media “New media” suggests something less settled, known, identified Changing set of formal and technological experiments Complex set of interactions between new technologies and established media forms

Kinds of New Media New textual experiences New ways of representing the world New relationships between subjects and media technologies New experiences of the relationship between embodiment, identity, and community New conceptions of the biological body’s relationship to the technological media New patterns of organization and production

Characteristics of New Media Digitality Interactivity Hypertextuality Dispersal Virtuality

Digitality Digital signals and objects deal in the realm of the discrete or finite, meaning there is a limited set of values they can be. We live in an analog world. The common theme among all of these analog signals is their infinite possibilities There are an infinite amount of colors to paint an object (even if the difference is indiscernible to our eye), There are an infinite number of tones we can hear, and There are an infinite number of smells we can smell.. Data input converted to numbers Can be output to both online sources or “hard copy” Analog - all input data is converted to another physical object

Interactivity Instrumental - users’ ability to directly intervene in and change the images and texts that they access. Hypertextual navigation Immersive navigation - visual and sensory spatial exploration Registration interactivity - users’ ability to register their own messages; bulletin bds, MUDs, MOOs Interactive Communication - ability of communication to emulate face-to-face

Hypertext Discrete units of material in which each one carries a number of pathways to other units. A Web of connections in which the user controls the navigation Vannevar Bush - As We May Think Ted Nelson - A New Home for the Mind Marshall McLuhan – Understanding Media: Extensions of Man

Dispersal Consumption - large number of highly differentiated texts; no longer simultaneity and uniformity of messages received by mass audience Production - craft skills of production becoming more dispersed, less specialized Media production processes become closer to habits of everyday life - PowerPoint, desktop publishing, Web design, photo manipulation, etc. Concept of prosumer

Virtuality Immersion - environment of computer graphics and digital video in which user has some degree of interaction Visual, tactile experiences felt to be in one place, while the body is in physical space Space - way of imagining the invisible space of communication networks Adopt different identities; new associations and communities Cyberspace - questions of embodiment

Generalizing New Media: Frameworks for Discussion and Comparison

Exploring the Design Space for Interactive Scholarly Communication Motivation for a community developed framework for interactive scholarly communication Seven dimensions of interactive communication Previous work in the context of the seven dimensions Open research questions Conclusions/Goals

Brief timeline for improved scholarly communication 1940’s Vannevar Bush 1960’s & 1970’s Nelson, Engelbart, Licklider, van Dam 1980’s Hypertext research field coalesces –Authoring and accessing hypertextual writing and literary studies of hypertextual structures 1990’s Digital libraries and interactive digital storytelling research fields coalesce –Creation, maintenance, preservation, access

Vannevar Bush (1890 – 1974) Vannevar Bush is the pivotal figure in hypertext research. His conception of the Memex introduced the idea of an easily accessible, individually configurable storehouse of knowledge. Douglas Engelbart and Ted Nelson were directly inspired by his work, and, in particular, his article, "As We May Think."Memex Douglas EngelbartTed Nelson In 1919, he joined MIT's Department of Electrical Engineering, where he stayed for twenty-five years. In 1932, he was appointed vice-president and dean. At this time, Bush worked on optical and photocomposition devices, as well as a machine for rapid selection from banks of microfilm. Further positions followed: president of the Carnegie Institute in Washington, DC (1939); chair of National Advisory Committee for Aeronautics (1939); director of Office of Scientific Research and Development. During World War II, Bush worked on radar antenna profiles and the calculation of artillery firing tables. The mathematics involved was complicated and repetitive. Bush proposed the development of an analogue computer; this became the Rockefeller Differential Analyser. Unfortunately, his research was rendered obsolete by 1950 with the invention of the digital computer. Bush is most famous for his Memex, publicized in the aforementioned article in Atlantic Monthly (1945) and most readily available in Nyce and Kahn. Yet this same article also contained descriptions of devices rarely cited. These include the Cyclops Camera: "worn on forehead, it would photograph anything you see and want to record. Film would be developed at once by dry photography;" advances in microfilm; a thinking machine (actually a mathematical calculator); and a vocoder, "a machine which could type when talked to".

“What Dr. Bush Foresees” Cyclops Camera Worn on forehead, it would photograph anything you see and want to record. Film would be developed at once by dry photography. Microfilm It could reduce Encyclopaedia Britannica to volume of a matchbox. Material cost: 5¢. Thus a whole library could be kept in a desk. Vocoder A machine which could type when talked to. But you might have to talk a special phonetic language to this mechanical supersecretary. Thinking machine A development of the mathematical calculator. Give it premises and it would pass out conclusions, all in accordance with logic. Memex An aid to memory. Like the brain, Memex would file material by association. Press a key and it would run through a “trail” of facts.

Memex

Cyclops Camera

Current practices of scholarly communication Focus on text and continuance of existing methods of writing the scientific record Restructuring old media via point-to-point conversions from the static physical world to a part of the digital world that is also static The way we make the record is essentially unchanged from Vannevar Bush’s time

A new approach to scholarly communication A wide-open exploration of the design space created by new media for writing the scientific record Focus on interactive authoring tools and systems that will help scholars record the record of their ideas and scientific contributions Authoring tools for the digital libraries of tomorrow

Why new forms of scholarly communication are needed Infrastructure is available: –Internet for dissemination –Digital Libraries for archival storage Interactive faction is not keeping up with results from interactive fiction Scholarly communication is already broken Existing forms may not be the most efficient New media may be more immersive and engaging

Research agenda Design new systems for making and consulting the scientific record Evaluate and disseminate the results of interactive media studies on scholarly communication Generate and distribute new interactive media, authoring tools, and storytelling engines Improve the general framework for interactive scholarly communication

Initial framework Interactive media tend to change the relationship between the reader and the author A simple model will suffice to discuss the design space of interactive scholarly communication

Consider the ACM DL Consider a personalized news reader Consider a MMORPG

Dimensions of Interest Roles – are there separate author/reader (creator/consumer) roles or are they merged? Voices – how many voices are normal in the medium? Interaction – do users get to interact with the content? Indirection – does the reader see what the author created?

Dimensions of Interest (cont.) History – does the medium preserve the authoring process or interaction? Narrative – do normal examples bind the contents into a single (or multiple) narrative? Media – does the medium build on top of a variety of component media?

Seven dimensions of interactive communication Roles Voices Interaction Indirection Indirection History History Narrative Narrative Media Media

Prior systems Spatial hypertext (VKB) Digital Scholarship and Publishing (Synchrony) Metadocuments (Walden’s Paths) For each system: –Brief review –Locate in design space provided by the seven dimensions

VKB Spaces as media for interactive scholarly communication Publishing unit is an evolutionary space Authors construct the space over time through direct manipulation of visual representations Readers explore the space to understand its story Existing media types: text, images, music files, internal and external links Constructed media types: classes, lists, collections

VKB Spaces in the design space Multiple roles Multiple voices Moderate level of interaction Low level of indirection High level of support for history VKB spaces are most often non-narrative Low to moderate level of media use

Synchrony PADLs as media for interactive scholarly communication Publishing units: structured presentations of streaming video segments and text (transcripts, original writing, annotations) Authoring through direct manipulation Readers watch streaming video and read text Existing media types: streaming video, text Constructed media types: presentations

Synchrony PADLs in the design space Multiple roles One voice Low level of interaction Low level of indirection No history Highly narrative Moderate level of media use

Walden’s Paths as media for interactive scholarly communication Publishing unit: annotated paths Authoring via a path authoring tool Readers browse paths linearly, jump between pages of a path, or navigate off the path Existing and constructed media are those offered by the web

Walden’s Paths in the design space Separate author and user roles Multiple voices due to component pages Medium level of interaction Medium level of indirection No history Medium level of narrative Moderate level of media use

Characteristics of communication supported by ends of spectrum RolesAuthorityDiscussion Voices Consistent presentation Many perspectives InteractionImmersionEngagement Indirection Author control Applicability to diverse situations HistoryPrivacy Understanding authoring process Narrative Facts, maps, emergent relations Comprehension of complex reasoning Media Easy distribution Multiple comprehension strategies